Hadoop : start-dfs.sh Connection refused Hadoop : start-dfs.sh Connection refused hadoop hadoop

Hadoop : start-dfs.sh Connection refused


There is a problem the way pdsh works by default (see edit), but Hadoop can go without it. Hadoop checks if the system has pdsh on /usr/bin/pdsh and uses it if so. An easy way get away from using pdsh is editing $HADOOP_HOME/libexec/hadoop-functions.sh

replace the line

if [[ -e '/usr/bin/pdsh' ]]; then

by

if [[ ! -e '/usr/bin/pdsh' ]]; then

then hadoop goes without pdsh and everything works.

EDIT:

A better solution would be use pdsh, but with ssh instead rsh as explained here, so replace line from $HADOOP_HOME/libexec/hadoop-functions.sh:

PDSH_SSH_ARGS_APPEND="${HADOOP_SSH_OPTS}" pdsh \

by

PDSH_RCMD_TYPE=ssh PDSH_SSH_ARGS_APPEND="${HADOOP_SSH_OPTS}" pdsh \

Obs: Only doing export PDSH_RCMD_TYPE=ssh, as I mention in the comment, doesn't work. I don't know why...

I've also opened a issue and submitted a patch to this problem: HADOOP-15219


I fixed this problem for hadoop 3.1.0 by adding

PDSH_RCMD_TYPE=ssh

in my .bashrc as well as $HADOOP_HOME/etc/hadoop/hadoop-env.sh.


check if your /etc/hosts file contains the hostname stretch and localhost mapping or not

my /etc/hosts file