Upload file to HDFS using DFSClient on Java Upload file to HDFS using DFSClient on Java hadoop hadoop

Upload file to HDFS using DFSClient on Java


Can't say anything about copying files with DFSClient but you can use FileSystem's methods for that purposes:

  • copyFromLocalFile(Path src, Path dst) - copy file from local filesystem to HDFS
  • moveFromLocalFile(Path src, Path dst) - move file fromlocal file system to HDFS

For example:

FileSystem fs = FileSystem.get(conf);fs.copyFromLocalFile(new Path("/home/user/test.txt"), new Path("/hadoop/test.txt"));

Also you can write files via output stream:

FSDataOutputStream outStream = fs.create(new Path("/hadoop/test.txt"));outStream.write(buffer);outStream.close();

Futhermore there are many useful methods for file copying between local and distributed file systems in classes FileSystem and FileUtil.


Change the finally block order

finally {    if (out != null) {        out.close();    }    if (in != null) {        in.close();    }    if (client != null) {        client.close();    }}