HDFS datanode not starting with kerberos HDFS datanode not starting with kerberos hadoop hadoop

HDFS datanode not starting with kerberos


It seems the JSVC package is not available for Windows. After taking a long time, I figured out a solution for this problem. Instead of using JSVC we can use SASL.

Hadoop-2.6.0 supports SASL for authenticating the data transfer protocol. Refer to the documentation: Secure Datanode.

To quote that link:

As of version 2.6.0, SASL can be used to authenticate the data transfer protocol. In this configuration, it is no longer required for secured clusters to start the DataNode as root using jsvc and bind to privileged ports. To enable SASL on data transfer protocol, set dfs.data.transfer.protection in hdfs-site.xml, set a non-privileged port for dfs.datanode.address, set dfs.http.policy to HTTPS_ONLY and make sure the HADOOP_SECURE_DN_USER environment variable is not defined. Note that it is not possible to use SASL on data transfer protocol if dfs.datanode.address is set to a privileged port. This is required for backwards-compatibility reasons.