If you ever need to copy a large amount of data over a network (especially if its a huge number of small files) you can pipe a tar command through a ssh connection, and because tar copies whole blocks at a time, it will be far, far, faster than using SCP.
To execute it, simply:
$ tar czf - <files> | ssh user@host "cd /wherever; tar xvzf -"
Forget what I said. Install Homebrew (really, do it now) and use HomeBrew to install it.
Pipe Viewer (pv) – is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion. However, it isn’t included by default in Mac OSX.
The good news is that there are several ports of pv, you just need to go and grab one:
- HomeBrew: Run “
brew install pv” to get the latest version.
- MacPorts: Run “
port install pv” to get the latest version.
- Or (recommended) install the Rudix pv port (a simple package installer)
Pv allows you to get a really awesome progress of your terminal commands to see how things are going; especially useful for long operations (such as cp or tar etc) so you know everything is ticking over time (and perhaps even giving an ETA for completion):
13.2GiB 1:33:17 [3.57MiB/s] [================================> ] 67% ETA 0:44:4
I highly recommend this for anyone doing long, large or complicated terminal commands. It’s outstanding! To learn more about using Pipe Viewer, this is a great resource.