Hadoop IOException failure to login

I've encountered the same issue when running an hbase client from a docker container with Java 8. It is apparently caused by class com.sun.security.auth.module.UnixLoginModule which uses a native call to get the unix username. In my case, it is not mapped in docker, and the class throws a NullPointerException. It is not a bug in hadoop per se.

To instruct hadoop to bypass the lookup of the OS username, I was able to add the following line of code before all initialization:

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("hduser"));

In your case, you are running the server, so your options of injecting the code are limited. Instead there are two options:

1. Try the IBM JDK instead
2. Try to debug the OS user setup on the workers ( $ whoami). If it says something like 'cannot find name for user ID XXXX', then check the /etc/passwd setup

Faced same error while running spark code from IDE.

Exception in thread "main" java.io.IOException: failure to login        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:822)        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:774)        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:647)        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)        at scala.Option.getOrElse(Option.scala:121)        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)        at org.apache.spark.SparkContext.<init>(SparkContext.scala:292)        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2486)        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)        at scala.Option.getOrElse(Option.scala:121)        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)        at SimpleApp$.main(SimpleApp.scala:17)        at SimpleApp.main(SimpleApp.scala)Caused by: javax.security.auth.login.LoginException:java.lang.NullPointerException: invalid null input: name

Resolution added following statement in main code.

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("vyxx"))

I found that I don't have to take any special measures in code if I ensure that the docker image is properly configured for the jenkins user. The code I use to setup the image for the jenkins user for Debian/Ubuntu based images is:

# Add Jenkins usergroupadd --gid 1000 jenkinsuseradd --uid 1000 --gid jenkins --shell /bin/bash --home-dir /var/jenkins_home jenkinsmkdir /var/jenkins_homechown 1000:1000 /var/jenkins_homeecho 'jenkins ALL=NOPASSWD: ALL' >> /etc/sudoers.d/50-jenkinsecho 'Defaults    env_keep += "DEBIAN_FRONTEND"' >> /etc/sudoers.d/env_keep