You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Bansal, Jaimita" <Ja...@gs.com> on 2023/01/19 23:35:29 UTC
[Spark Standalone Mode] How to read from kerberised HDFS in spark standalone mode
Hi Spark Team,
We are facing an issue when trying to read from HDFS via spark running in standalone cluster. The issue comes from the executor node not able to authenticate. It is using auth:SIMPLE when actually we have setup auth as Kerberos. Could you please help in resolving this?
Caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:778) ~[hadoop-common-3.1.1.7.1.7.1000-141.jar:na]
18:57:44.726 [main] DEBUG o.a.spark.deploy.SparkHadoopUtil - creating UGI for user: <user>
18:57:45.045 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login
18:57:45.046 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login commit
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - using kerberos user:<user> @GS.COM
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - Using user: "<user>@GS.COM" with name <user>@GS.COM
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - User entry: "<user> @GS.COM"
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - UGI loginUser:<user>@GS.COM (auth:KERBEROS)
18:57:45.056 [main] DEBUG o.a.h.security.UserGroupInformation - PrivilegedAction as:<user> (auth:SIMPLE) from:org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:61)
18:57:45.078 [TGT Renewer for <user>@GS.COM] DEBUG o.a.h.security.UserGroupInformation - Current time is 1674068265078
18:57:45.079 [TGT Renewer for <user>@GS.COM] DEBUG o.a.h.security.UserGroupInformation - Next refresh is 1674136785000
18:57:45.092 [main] INFO org.apache.spark.SecurityManager - Changing view acls to: root,<user>
18:57:45.092 [main] INFO org.apache.spark.SecurityManager - Changing modify acls to: root,<user>
18:57:45.093 [main] INFO org.apache.spark.SecurityManager - Changing view acls groups to:
18:57:45.093 [main] INFO org.apache.spark.SecurityManager - Changing modify acls groups to:
Thanks,
Jaimita
Vice President, Data Lake Engineering
Goldman Sachs
________________________________
Your Personal Data: We may collect and process information about you that may be subject to data protection laws. For more information about how we use and disclose your personal data, how we protect your information, our legal basis to use your information, your rights and who you can contact, please refer to: www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
Fwd: [Spark Standalone Mode] How to read from kerberised HDFS in spark standalone mode
Posted by Wei Yan <ni...@gmail.com>.
Glad to hear that!
And hope it can help any other guys facing the same problem.
---------- Forwarded message ---------
发件人： Bansal, Jaimita <Ja...@gs.com>
Date: 2023年2月1日周三 03:15
Subject: RE: [Spark Standalone Mode] How to read from kerberised HDFS in
spark standalone mode
To: Wei Yan <ni...@gmail.com>
Cc: Chittajallu, Rajiv <Ra...@gs.com>,
Abner.Espinoza@ny.email.gs.com <Ab...@gs.com>
Hey Wei,
This worked! Thank you so much.
Thanks,
Jaimita
*From:* Wei Yan <ni...@gmail.com>
*Sent:* Thursday, January 19, 2023 7:08 PM
*To:* Bansal, Jaimita [Engineering] <Ja...@ny.email.gs.com>
*Subject:* Re: [Spark Standalone Mode] How to read from kerberised HDFS in
spark standalone mode
Hi!
You can use the Delegation Token.
In the spark standalone mode ,the simple way to use the Delegation Token is
set an environment variable in every node include master nodes and work
nodes, and the content of this environment variable is the path of the
Delegation Token file.
You should renew this file at a fixed time interval.
hdfs fetchdt -renewer hive /opt/spark/conf/delegation.token
Bansal, Jaimita <Ja...@gs.com> 于2023年1月20日周五 07:46写道：
Hi Spark Team,
We are facing an issue when trying to read from HDFS via spark running in
standalone cluster. The issue comes from the executor node not able to
authenticate. It is using auth:SIMPLE when actually we have setup auth as
Kerberos. Could you please help in resolving this?
Caused by: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN, KERBEROS]
at
org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:778)
~[hadoop-common-3.1.1.7.1.7.1000-141.jar:na]
18:57:44.726 [main] DEBUG o.a.spark.deploy.SparkHadoopUtil - creating UGI
for user: <user>
18:57:45.045 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login
18:57:45.046 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
login commit
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - using
kerberos user:<user> @GS.COM
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - Using user:
"<user>@GS.COM" with name <user>@GS.COM
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - User entry:
"<user> @GS.COM"
18:57:45.047 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
loginUser:<user>@GS.COM (auth:KERBEROS)
18:57:45.056 [main] DEBUG o.a.h.security.UserGroupInformation -
PrivilegedAction as:<user> (auth:SIMPLE)
from:org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:61)
18:57:45.078 [TGT Renewer for <user>@GS.COM] DEBUG
o.a.h.security.UserGroupInformation - Current time is 1674068265078
18:57:45.079 [TGT Renewer for <user>@GS.COM] DEBUG
o.a.h.security.UserGroupInformation - Next refresh is 1674136785000
18:57:45.092 [main] INFO org.apache.spark.SecurityManager - Changing view
acls to: root,<user>
18:57:45.092 [main] INFO org.apache.spark.SecurityManager - Changing
modify acls to: root,<user>
18:57:45.093 [main] INFO org.apache.spark.SecurityManager - Changing view
acls groups to:
18:57:45.093 [main] INFO org.apache.spark.SecurityManager - Changing
modify acls groups to:
Thanks,
Jaimita
*Vice President, Data Lake Engineering*
*Goldman Sachs*
------------------------------
Your Personal Data: We may collect and process information about you that
may be subject to data protection laws. For more information about how we
use and disclose your personal data, how we protect your information, our
legal basis to use your information, your rights and who you can contact,
please refer to: www.gs.com/privacy-notices
------------------------------
Your Personal Data: We may collect and process information about you that
may be subject to data protection laws. For more information about how we
use and disclose your personal data, how we protect your information, our
legal basis to use your information, your rights and who you can contact,
please refer to: www.gs.com/privacy-notices