Use FTP Over Hadoop for Large Files Use FTP Over Hadoop for Large Files hadoop hadoop

Use FTP Over Hadoop for Large Files


Yes, Hadoop Framework is suitable for large size files ( probably where the ingested file size is more than the block size of 256 MB ).

The main advantage of Hadoop is that it could be run on cheap hardware. Hadoop has some problems regarding handling of small files ( in Kb's ) .

Hadoop provides fault - tolerance as the files would be replicated in the cluster 3 times ( default ), could be increased or decreased.

Hadoop allows distributed processing of the data across the nodes.

It also scales easily.

If you want to copy in between 2 HDFS cluster, there is Distcp process, youcan look into the following link

https://www.cloudera.com/documentation/enterprise/5-13-x/topics/cdh_admin_distcp_data_cluster_migrate.html