How to select the optimal key in map reduce? How to select the optimal key in map reduce? hadoop hadoop

How to select the optimal key in map reduce?


So, for each line of input, you're going to query a database and then perform benchmark calculations for each line separately. After you finish the benchmark calculations, you are going to output each line with the benchmark value.

In this case, you can either not use a reducer at all, or use an identity reducer.

So your map function will read in a line, then it will fire a query to the Sybase database for the standard values, and then perform benchmark calculations. Since you want to output each line with the benchmark value, you could have the Map function output the line as key and benchmark value as value, i.e <line, benchmark value>

Your map function would look something like this: (I'm assuming your benchmark value is an integer)

public void map(Text key, IntWritable value, Context context) throws Exception {    String line = value.toString();   //this will be your key in the final output     /*          Perform operations on the line      */      /*          standard values = <return value from sybase query.>;      */      /*Perform benchmark calculations and obtain benchmark values */      context.write(line,benchmarkValue);     }