You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Maxim Buslovaev <m....@gmail.com> on 2023/06/20 07:41:13 UTC

Help with Spark and Ozone

Dear Community,
I would greatly appreciate it if you could assist me with my concern about
Apache Ozone setup.

Let me introduce myself in a few words. I am a Senior MLOps who works for a
worldwide enterprise with 50K+ employees on board. The project is a ML
Platform for Data Scientists (under NDA).  We use a cutting edge setup:
Apache Ozone 1.3.0 +  Spark 3.2.3 (cluster mode) + Kerberos + Kubernetes.

The primary concern is Spark workers can't authenticate on Ozone Manager.
In a mean time, Spark driver alone can do it.

Please, find meaningful logs attached. The interesting timestamp is
04:35:18 in both files where SaslRPCServer could not send tokens for
SaslRPCClient.

I look forward to receiving your reply as soon as possible. Thanking you in
anticipation of your kind cooperation.

P.S.:

I would appreciate if we could collaborate on the above topics.