/bin/bash: /bin/java: No such file or directory error in Yarn apps in MacOS

This answer is applicable for Hadoop version 2.6.0 and earlier. Disabling SIP and creating a symbolic link does provide a workaround. A better solution is to fix the hadoop-config.sh so it picks up your JAVA_HOME correctly

In HADOOP_HOME/libexec/hadoop-config.sh look for the if condition below # Attempt to set JAVA_HOME if it is not set

Remove extra parentheses in the export JAVA_HOME lines as below. Change this

if [ -x /usr/libexec/java_home ]; then    export JAVA_HOME=($(/usr/libexec/java_home))else    export JAVA_HOME=(/Library/Java/Home)fi

to

if [ -x /usr/libexec/java_home ]; then    // note that the extra parentheses are removed    export JAVA_HOME=$(/usr/libexec/java_home)else    export JAVA_HOME=/Library/Java/Homefi

Restart yarn after you have made this change.

More detailed info can be found here https://issues.apache.org/jira/browse/HADOOP-8717 and seems that Hadoop 3.0.0-alpha1 is the first release with the fix.

Starting YARN using following commands instead of $HADOOP_PREFIX/sbin/start-yarn.sh fixed this issue for me.

$HADOOP_PREFIX/sbin/yarn-daemon.sh start resourcemanager;

$HADOOP_PREFIX/sbin/yarn-daemon.sh start nodemanager;

as mentioned at following link

https://issues.apache.org/jira/browse/HADOOP-8717

None of the above helped me. What I fixed for me was:

Setting the hadoop-env.sh in /usr/local/Cellar/hadoop/etc/hadoop/hadoop-env.sh

Changed this:

export JAVA_HOME=${JAVA_HOME}

To this:

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home

All the credit to this user: https://stackoverflow.com/a/39676720/992347

Cheers