reflection error in spark yarn cluster mode

[Michal Vince <vi...@gmail.com>]

Hi guys,

after few years of working via spark livy I wanted to test out spark via
Yarn in cluster mode

I`m using HDP 2.6.5.x version of hadoop with spark 2.3.0, my HDP is
kerberized

I builded zeppelin from master branch and updated my interpreter
settings (see the attachment)
after running any spark paragraph I`m getting    

org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: Fail to open SparkInterpreter

...

Caused by: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.


I also attached logs from yarn and zeppelin to keep this mail


Also, when I switch from cluster mode to client mode, everything works
as expected...


If you have any idea, what I did wrong configuring this, please let me
know as I`m getting super frustrated here :(


Thanks a million

best regards

Miso