You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Bulldog20630405 <bu...@gmail.com> on 2022/03/08 19:37:54 UTC

spark master/worker logs after job completes

coming from a yarn background; log files can be found after job finishes...

with spark master/workers how to configure to get logs after job finishes?

we have setup our spark history server and spark-defaults include:

spark.eventLog.enabled         true
spark.eventLog.dir             file:///shared_mount/spark/events/
spark.history.fs.logDirectory  file:///shared_mount/spark/events/

the events are there; but no log files anways
what am i missing?