You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Gerard Casey <ge...@me.com> on 2016/11/11 17:48:15 UTC

RDD to HDFS - Kerberos - authentication error - RetryInvocationHandler

Hi all,

I have an RDD that I wish to write to HDFS.

data.saveAsTextFile("hdfs://path/vertices")

This returns: WARN RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.getFileInfo over null. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]

I have checked KERBEROS and it is properly authenticated. The Available:[TOKEN, KERBEROS] seems to show that the spark conf file is setup correctly. 

How do I fix this?! 

Thanks! 

Geroid