HDFS datanode not starting with kerberos
It seems the JSVC package is not available for Windows. After taking a long time, I figured out a solution for this problem. Instead of using JSVC we can use SASL.
Hadoop-2.6.0 supports SASL for authenticating the data transfer protocol. Refer to the documentation: Secure Datanode.
To quote that link:
As of version 2.6.0, SASL can be used to authenticate the data transfer protocol. In this configuration, it is no longer required for secured clusters to start the DataNode as root using jsvc and bind to privileged ports. To enable SASL on data transfer protocol, set
dfs.data.transfer.protection
inhdfs-site.xml
, set a non-privileged port fordfs.datanode.address
, setdfs.http.policy
toHTTPS_ONLY
and make sure theHADOOP_SECURE_DN_USER
environment variable is not defined. Note that it is not possible to use SASL on data transfer protocol ifdfs.datanode.address
is set to a privileged port. This is required for backwards-compatibility reasons.