copy files from FTP to HDFS copy files from FTP to HDFS hadoop hadoop

copy files from FTP to HDFS


Have you tried this? FTP TO HDFS ... You can modify this code and change arguments for File to be FTP a agrs[0] and HDFS Paht to args[1]. and than run it as hadoop jar. hope this'll help...


Have you looked at WebHDFS (http://hadoop.apache.org/docs/r1.0.4/webhdfs.html) or HttpFS (http://hadoop.apache.org/docs/r2.2.0/hadoop-hdfs-httpfs/index.html)

The services need access to the Hadoop cluster, then you could expose the HttpFS port to a server with access to the FTP server.


Try this:

hadoop fs -get ftp://uid:password@server_url/file_path temp_file | hadoop fs -moveFromLocal tmp_file hadoop_path/dest_file