How to Sort Reducer Output? [duplicate] How to Sort Reducer Output? [duplicate] hadoop hadoop

How to Sort Reducer Output? [duplicate]


you can use the composite key and the composite key comparator

create a class e.g.

class Pair(){    String key    String value;}

and use it in your reducer output like this

context.write(new Pair(key.toString(), Float.toString(result)), null);

then create a comparator

public class PairComparator extends WritableComparator {    protected PairComparator() {        super(Pair.class, true);    }       @Override    public int compare(WritableComparable w1, WritableComparable w2) {        Pair k1 = (Pair)w1;        Pair k2 = (Pair)w2;                 return  k1.getValue().compareTo(k2.getValue());    }}

and then use the comparator in your job definition
job.setSortComparatorClass(PairComparator.class);

I didn't check the code above. It just the idea.

I hope it will help