Hadoop Map-Reduce , Need to combine two mapper with one common Reducer Hadoop Map-Reduce , Need to combine two mapper with one common Reducer hadoop hadoop

Hadoop Map-Reduce , Need to combine two mapper with one common Reducer


MultipleInputs.addInputPath is what you are looking for. This is how your configuration would look like. Make sure both AnyMapper1 and AnyMapper2 write the same output expected by MergeReducer

JobConf conf = new JobConf(Merge.class);conf.setJobName("merge");conf.setOutputKeyClass(IntWritable.class); conf.setOutputValueClass(Text.class); conf.setReducerClass(MergeReducer.class);conf.setOutputFormat(TextOutputFormat.class);MultipleInputs.addInputPath(conf, inputDir1, SequenceFileInputFormat.class, AnyMapper1.class);MultipleInputs.addInputPath(conf, inputDir2, TextInputFormat.class, AnyMapper2.class);FileOutputFormat.setOutputPath(conf, outputPath);


You can create a custom writable. You can populate the same in the Mapper. Later in the Reducer you can get the Custom writable Object and do the necessary business operation.