What is the proper value of HADOOP_HOME and PIG_CLASSPATH for APACHE HADOOP version 2.8.0? What is the proper value of HADOOP_HOME and PIG_CLASSPATH for APACHE HADOOP version 2.8.0? hadoop hadoop

What is the proper value of HADOOP_HOME and PIG_CLASSPATH for APACHE HADOOP version 2.8.0?


Alex!Unfortunately, it's not related to Pig paths (tried it on my configured hadoop cluster) with same result. The error you get refers to the fact that Physical plan compiler has a bug in compile method. So in order to make your attempt work you have two possibilities

  1. Run native MR job using hadoop and after it finishes process it's results in pig

  2. Edit pig source code and compile your own version. You'll need to editorg.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler#compile method and replace

    for (PhysicalOperator op : leaves) {    if (!(op instanceof POStore)) {        int errCode = 2025;        String msg = "Expected leaf of reduce plan to " +            "always be POStore. Found " + op.getClass().getSimpleName();        throw new MRCompilerException(msg, errCode, PigException.BUG);    }}

with

    for (PhysicalOperator op : leaves) {        if (!(op instanceof POStore) && !(op instanceof PONative)) {            int errCode = 2025;            String msg = "Expected leaf of reduce plan to " +                "always be POStore. Found " + op.getClass().getSimpleName();            throw new MRCompilerException(msg, errCode, PigException.BUG);        }    }