You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Kelvin Chu <2d...@gmail.com> on 2015/12/09 23:01:18 UTC

Is Spark History Server supported for Mesos?

Spark on YARN can use History Server by setting the configuration
spark.yarn.historyServer.address. But, I can't find similar config for
Mesos. Is History Server supported by Spark on Mesos? Thanks.

Kelvin

Re: Is Spark History Server supported for Mesos?

Posted by Steve Loughran <st...@hortonworks.com>.

On 9 Dec 2015, at 22:01, Kelvin Chu <2d...@gmail.com>> wrote:

Spark on YARN can use History Server by setting the configuration spark.yarn.historyServer.address.


That's the stuff in SPARK-1537 which isn' actually built in yet.

But, I can't find similar config for Mesos. Is History Server supported by Spark on Mesos? Thanks.



you set up your jobs to log to HDFS and run the history server to poll and replay the logs that appear there.

see: http://spark.apache.org/docs/latest/monitoring.html