Copying hdfs file to remote linux server using scp directly? Copying hdfs file to remote linux server using scp directly? hadoop hadoop

Copying hdfs file to remote linux server using scp directly?


You can stream the data using linux pipes if ssh to server is available

hdfs dfs -cat my-file.txt | ssh myserver.com 'cat > /path/to/my-file.txt'

First command reads the file data as a stream, second one redirects it to remote server. Worked for me. Take into account that ssh sometimes timeout if there's nothing on the wire for a long time.

Credits to this answer: https://superuser.com/questions/291829