Writing output in multiple files in Hadoop [duplicate] Writing output in multiple files in Hadoop [duplicate] hadoop hadoop

Writing output in multiple files in Hadoop [duplicate]


Use the MultipleOutputFormat class, the output file name can be deduced from the key and the reducer output value from the reducer. MultipleOutputFormat#generateFileNameForKeyValue has to be implemented in the user defined OutputFormat class.

static class MyMultipleOutputFormat extends MultipleOutputFormat<Text, Text> {    protected String generateFileNameForKeyValue(Text key, Text value, String name) {        String keyString = key.toString();        String valueString = value.toString();        #return a combination of keyString and valueString     }}