Happened to me in work once… I was connected via SSH to one of our test machines, so I could test connection disruption handling on a product we had installed.
I had a script that added iptables rules to block all ports for 30 seconds then unblock them. Of course I didn’t add an exception for port 22, and I didn’t run it with nohup, so when I ran the script it blocked the ports, which locked me out of SSH access, and the script stopped running when the SSH session ended so never unblocked the ports. I just sat there in awe of my stupidity.
Happened to me in work once… I was connected via SSH to one of our test machines, so I could test connection disruption handling on a product we had installed.
I had a script that added iptables rules to block all ports for 30 seconds then unblock them. Of course I didn’t add an exception for port 22, and I didn’t run it with nohup, so when I ran the script it blocked the ports, which locked me out of SSH access, and the script stopped running when the SSH session ended so never unblocked the ports. I just sat there in awe of my stupidity.
We’ve all experienced the walk of shame to the server room to hook up a monitor keyboard.
Ah, if only it was a server room and not a customer 3 hours drive away. And he’d closed and gone home for the night.
Fortunately it just needed a reboot, and I was able to talk him through that in the morning.
Oof… well you can just say “it must be some hardware problem or something… maybe a reboot will fix it.”
tmux
Always use tmux when possible for remote connections.
What does it do in this case?
Tmux essentially creates a pseudo-shell that persists between sessions.
So you can start a process, detach the session, start something else, disconnect, come back next week, and check on it.
It does other things too. Like console tiling.