Python how to resolve the Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 2
Use the answer by Harishanker in the post - How to resolve java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 2?
Make sure that the both the mapper and the reducer files are executable using chmod. (Eg: 'chmod 744 mapper.py')
Then make the streaming command as such:
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar \-D mapreduce.job.reduces=5 \-files lr \-mapper lr/mapper.py -n 5 -r 0.4 \-reducer lr/reducer.py -f 3618 \-input training \-output models Now it should work. Please comment if it doesn't.