Can't get Master Kerberos principal for use as renewer for Talend Batch Jobs

You should include the hadoop config in the classpath (:/path/hadoop-configuration). You should include all configuration files in that hadoop configuration directory, not only the core-site.xml and hdfs-site.xml files. It happened to me and that solved the problem.

the same problem when I start spark on k8s,

py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.             : java.io.IOException: Can't get Master Kerberos principal for use as renewer                                                                                       at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:133)                                                                                                                                                                                                                 at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)                                                                                                                                                                                                                 at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)                                   at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243)                                     at org.apache.spark.input.WholeTextFileInputFormat.setMinPartitions(WholeTextFileInputFormat.scala:52)                                                      at org.apache.spark.rdd.WholeTextFileRDD.getPartitions(WholeTextFileRDD.scala:54)                                                                           at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273)                                                            at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269)                                                                                      at scala.Option.getOrElse(Option.scala:121)                                                                                                                 at org.apache.spark.rdd.RDD.partitions(RDD.scala:269)

and I just add yarn-site.xml to the HADOOP_CONFIG_DIR.

the yarn-site.xml only contains yarn.resourcemanager.principal

<?xml version="1.0" encoding="UTF-8"?><configuration> <property>    <name>yarn.resourcemanager.principal</name>    <value>yarn/_HOST@DM.COM</value>  </property></configuration>

this working for me.