ssh-Fu: the scATAC Edition

The problem

We want to transfer a large amount of data from Jason’s server at Stanford to the MGH Erisone cluster. The problem is that both the Stanford server and the MGH server require a VPN to access. We can’t be VPN’d to two different connections at once, right? RIGHT?

SSH Tunnel

Login into Jason’s VPN server (su-vpn.stanford.edu).
ssh to the server of interest (e.g. ssh jason@greenseq) with TWO terminal windows
In one terminal window, we’re going to expose the erisone server to port 2222 from the greenseq server like so:
```
ssh -L 2222:erisone.partners.org:22 cl322@ssh.research.partners.org
```
Alternatively, we can open the port in the background using these additional flags:
```
ssh -f -N -L 2222:erisone.partners.org:22 cl322@ssh.research.partners.org
```
After password authentication, this terminal window will bridge the erisone cluster and the greenseq cluster via port 2222
Now, we can ssh or scp from the greenseq server using this exposed port (and thus without the need to VPN)
To ssh, we have to remove the strict RSA key checking since these are a dynamic process and point our connection to the exposed port via local host:
```
ssh -o StrictHostKeyChecking=no -p 2222 cl322@localhost
```

Moving Files

In general, we don’t care as much about being able to ssh (it may be nice to do this in a third terminal window though to verify the files are transfering). Instead, we want to scp files, using the following command:
```
scp -o StrictHostKeyChecking=no -P 2222 `find ./ -name "singles*.st.bam"` cl322@localhost:/data/aryee/caleb/scATAC/
```
An important thing to not here is that the scp synatx requires a captial “P”, but otherwise, the typical scp syntax is retained.
Rather than watching all of the files transfer, we may want to throw the command in the background. This way, we can close our windows and do something else. See the discussion here. Thus, the final command that was executed with port 2222 still exposed in another command window was
```
nohup scp -o StrictHostKeyChecking=no -P 2222 `find ./ -name "singles*.st.bam"` cl322@localhost:/data/aryee/caleb/scATAC/ > nohub.out 2>&1
*ctrl + z*
bg
```
Note: if you want to nohup the file transfer, you’ll also have to have the ssh connection in the background as well (with the -f -N flags).
Note: read this discussion on closing out ssh ports.
Note: there’s nothing special about the port number 2222. If it’s closed when you try to establish the connection in the first execution, you will get notified of this. Simply replace 2222 with some other port number.

Important

You’ll want to close the ssh port before exiting whatever server you’re on. This isn’t a big deal if you ran the port connection in the foreground. However, if you ran it in the background (i.e. the -f -N flags), then you’ll need to kill the connection to make your server access play friendly in the future. From the post linked above, see which connection is responsible for the port connection:
```
ps aux | grep 2222
```
and then look for the second column that has the pid. Say this number is 14651. Close it off using the following:
```
kill -9 14651
```

Acknowledgements

Thanks to Martin Aryee for the discussion on ssh-fu’ing