MapReduce job in headless environment fails N times due to AM Container exception from container-launch

The problem is caused because YARN is using different path for JAVA executable different then you have in your OS.

The hardcoded path to check for java is /bin/java however if you don't have /bin/java as your Java executable the YARN job will fail. Like in OSX I have Java 1.7 running at /usr/bin/java as below:

$java -version   java  version "1.7.0_45"  Java(TM) SE Runtime Environment (build 1.7.0_45-b18)  Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

To solve this problem in OSX I created a link from /bin/java to /usr/bin/java as below:

$ sudo ln -s /usr/bin/java /bin/java                                                                          Password: *****

After then the job ran successfully.

If sudo ln -s /usr/bin/java /bin/java action not permitted

System Integrity Protection (SIP,1 sometimes referred to as rootless2[3]) is a security feature of OS X El Capitan, the operating system by Apple Inc.

It is a new feature of OS X 10.11 and you don't want to disable it, you have to edit hadoop-env.sh and yarn-env.sh, set JAVA_HOME and JAVA to exact path.

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/HomeJAVA=/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/bin/java

Thnaks