Uploading Large Files

Whenever I have to upload a large file to a server—large meaning over 20GB—I try using a simple scp, forgetting that in Argentina we have a terrible internet connection that cuts out all the time and has very low bandwidth.

Therefore, the ideal approach is to use something that can upload the file and, if the connection fails, cancel and resume from where it left off. This is where rsync comes to the rescue.

The following commands support resume:

To upload:

rsync --partial --progress --rsh=ssh local_file user@host:remote_file

To download

rsync --partial --progress --rsh=ssh user@host:remote_file local_file

A tip from an old mentor that is still useful today.