How to programmatically create/touch a file in hdfs? How to programmatically create/touch a file in hdfs? hadoop hadoop

How to programmatically create/touch a file in hdfs?


The java hadoop FileSystem api provides these types of helpers.

Here is a way to replicate a classic touch for hdfs:

import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FileSystem;import org.apache.hadoop.fs.Path;import org.apache.hadoop.fs.FSDataOutputStream;import java.io.IOException;public static void touch(String filePath) throws IOException {  FileSystem hdfs = FileSystem.get(new Configuration());  Path fileToTouch = new Path(filePath);  FSDataOutputStream fos = null;  // If the file already exists, we append an empty String just to modify  // the timestamp:  if (hdfs.exists(fileToTouch)) {    fos = hdfs.append(new Path(filePath));    fos.writeBytes("");  }  // Otherwise, we create an empty file:  else {    fos = hdfs.create(new Path(filePath));  }  fos.close();}

This creates an empty file if the file doesn't already exist:

hdfs.create(new Path(filePath)).close();

And appends an empty String to the file if it already exist, in order to modify the timestamp:

hdfs.append(new Path(filePath)).writeBytes("");


Configuration configuration = new Configuration();FileSystem hdfs = FileSystem.get( new URI( "hdfs://localhost:54310" ), configuration );Path file = new Path("hdfs://localhost:54310/s2013/batch/table.html");if ( hdfs.exists( file )) { hdfs.delete( file, true ); } OutputStream os = hdfs.create( file,    new Progressable() {        public void progress() {            out.println("...bytes written: [ "+bytesWritten+" ]");        } });BufferedWriter br = new BufferedWriter( new OutputStreamWriter( os, "UTF-8" ) );br.write("Hello World");br.close();hdfs.close();