Transferring large amount of data from external server to compute node

[Predrag Punosevac]

Jayanth Koushik <jkoushik at cs.cmu.edu> wrote:

> Hey Predrag
> 
> For a new project, I need to transfer about 75 gigs of data (around 300 files) to a GPU node on Auton Lab from another server. But I???m not quite sure how to do this without involving the NFS at some point. Is there anyway to transfer the data directly to a compute node without going through one of the gateway nodes?
> 

I can transfer entire directory test (recursively) to my gpu1 scratch
folder using shell.autonlab.org as a proxy by executing the single
command from my home desktop

scp -o ProxyCommand="ssh -W %h:%p shell.autonlab.org" -r test predrag at gpu1.int.autonlab.org:/home/scratch/predrag/

NFS is not involved at all.

Cheers,
Predrag

> I hope I???ve not misunderstood the architecture here.
> 
> Thank you!
> Jayanth