Sep 27 2010

High Performance & Multi-threaded SCP Using RSYNC

by Ash • System Administration • Tags: Backup, File System, Rsync, SCP, SSH 0

Recently, I had the somewhat laborious job to backup a stack of websites and blogs from my remote web server.  Initially I tried do it with a simple SCP command; but after letting it run for about an hour, it was obvious that it was just too slow and taking too long downloading each file one at a time.

After talking complaining to a friend he suggested using the RSYNC command. Surprisingly it was incredibly very easy to get it working, you simply issue:

 rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

Obviously changing the appropriate parts for your case.

I found it to be at least 10 fold faster (or more) than SCP on it's own, and better still, RSYNC will resume when SCP wont!  Try it and see for yourself.