You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Maxim Buslovaev <m....@gmail.com> on 2023/06/20 07:41:13 UTC
Help with Spark and Ozone
Dear Community,
I would greatly appreciate it if you could assist me with my concern about
Apache Ozone setup.
Let me introduce myself in a few words. I am a Senior MLOps who works for a
worldwide enterprise with 50K+ employees on board. The project is a ML
Platform for Data Scientists (under NDA). We use a cutting edge setup:
Apache Ozone 1.3.0 + Spark 3.2.3 (cluster mode) + Kerberos + Kubernetes.
The primary concern is Spark workers can't authenticate on Ozone Manager.
In a mean time, Spark driver alone can do it.
Please, find meaningful logs attached. The interesting timestamp is
04:35:18 in both files where SaslRPCServer could not send tokens for
SaslRPCClient.
I look forward to receiving your reply as soon as possible. Thanking you in
anticipation of your kind cooperation.
P.S.:
I would appreciate if we could collaborate on the above topics.