recursively use scp but excluding some folders
Although scp
supports recursive directory copying with the -r
option, it does not support filtering of the files. There are several ways to accomplish your task, but I would probably rely on find
, xargs
, tar
, and ssh
instead of scp
.
find . -type d -wholename '*bench*/image' \| xargs tar cf - \| ssh user@remote tar xf - -C /my/dir
The rsync
solution can be made to work, but you are missing some arguments. rsync
also needs the r
switch to recurse into subdirectories. Also, if you want the same security of scp
, you need to do the transfer under ssh
. Something like:
rsync -avr -e "ssh -l user" --exclude 'fl_*' ./bench* remote:/my/dir
You can specify GLOBIGNORE
and use the pattern *
GLOBIGNORE='ignore1:ignore2' scp -r source/* remoteurl:remoteDir
You may wish to have general rules which you combine or override by using export GLOBIGNORE
, but for ad-hoc usage simply the above will do. The :
character is used as delimiter for multiple values.
Assuming the simplest option (installing rsync on the remote host) isn't feasible, you can use sshfs to mount the remote locally, and rsync from the mount directory. That way you can use all the options rsync offers, for example --exclude
.
Something like this should do:
sshfs user@server: sshfsdirrsync --recursive --exclude=whatever sshfsdir/path/on/server /where/to/store
Note that the effectiveness of rsync (only transferring changes, not everything) doesn't apply here. This is because for that to work, rsync must read every file's contents to see what has changed. However, as rsync runs only on one host, the whole file must be transferred there (by sshfs). Excluded files should not be transferred, however.