Same key different Reducers (HADOOP)? Same key different Reducers (HADOOP)? hadoop hadoop

Same key different Reducers (HADOOP)?


Data with the same key will always go to the same reducer. But you can choose whatever key you want, so if you want them to go to different reducers, then just choose different keys.

If you want to do an additional combination based on the output from your reducers, then you must do another MapReduce job, with the output from the first job as the input to the next one. This can get ugly fast, so you may wish to look at Cascading, Pig, or Hive to simplify things.


You can write a Custom Partitioner for your case, which overrides the default partitioning functionality of Hadoop MR job.

More details here: http://developer.yahoo.com/hadoop/tutorial/module5.html#partitioning