Re: Upgrading spark history server, no logs showing.

[bbarks <br...@pm.me>]

I finally circled back and tinkered with this, eventually finding the
solution. It turned out to be HDFS permissions on the history files.

For whatever reason, our HDFS perms worked fine with spark 2.0.2 and 2.1.2,
but when we ran spark 2.3.0 it wouldn't load any history in the UI.

I found out the history files are written with perms 0770, which is
hard-coded in the spark source. I just had to chown the history directory to
the same group the history service user is in, then set the facl so the
group has default permissions on all files added to the directory.




--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org