Downloading all the files in a directory with cURL

If you're not bound to curl, you might want to use wget in recursive mode but restricting it to one level of recursion, try the following;

wget --no-verbose --no-parent --recursive --level=1\--no-directories --user=login --password=pass ftp://ftp.myftpsite.com/

• --no-parent : Do not ever ascend to the parent directory when retrieving recursively.
• --level=depth : Specify recursion maximum depth level depth. The default maximum depth is five layers.
• --no-directories : Do not create a hierarchy of directories when retrieving recursively.

OK, considering that you are using Windows, the most simple way to do that is to use the standard ftp tool bundled with it. I base the following solution on Windows XP, hoping it'll work as well (or with minor modifications) on other versions.

First of all, you need to create a batch (script) file for the ftp program, containing instructions for it. Name it as you want, and put into it:

curl -u login:pass ftp.myftpsite.com/iiumlabs* -Oopen ftp.myftpsite.comloginpassmget *quit

The first line opens a connection to the ftp server at ftp.myftpsite.com. The two following lines specify the login, and the password which ftp will ask for (replace login and pass with just the login and password, without any keywords). Then, you use mget * to get all files. Instead of the *, you can use any wildcard. Finally, you use quit to close the ftp program without interactive prompt.

If you needed to enter some directory first, add a cd command before mget. It should be pretty straightforward.

Finally, write that file and run ftp like this:

ftp -i -s:yourscript

where -i disables interactivity (asking before downloading files), and -s specifies path to the script you created.

Sadly, file transfer over SSH is not natively supported in Windows. But for that case, you'd probably want to use PuTTy tools anyway. The one of particular interest for this case would be pscp which is practically the PuTTy counter-part of the openssh scp command.

The syntax is similar to copy command, and it supports wildcards:

pscp -batch login@mysshsite.com:iiumlabs* .

If you authenticate using a key file, you should pass it using -i path-to-key-file. If you use password, -pw pass. It can also reuse sessions saved using PuTTy, using the load -load your-session-name argument.

What about something like this:

for /f %%f in ('curl -s -l -u user:pass ftp://ftp.myftpsite.com/') do curl -O -u user:pass ftp://ftp.myftpsite.com/%%f