[jira] [Resolved] (SPARK-6751) Spark History Server support multiple application attempts

["Thomas Graves (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Graves resolved SPARK-6751.
----------------------------------
    Resolution: Duplicate

Dup of SPARK-4705

> Spark History Server support multiple application attempts
> ----------------------------------------------------------
>
>                 Key: SPARK-6751
>                 URL: https://issues.apache.org/jira/browse/SPARK-6751
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core, YARN
>    Affects Versions: 1.3.0
>            Reporter: Thomas Graves
>
> Spark on Yarn supports running multiple application attempts (configurable number) in case the first (or second..) attempts fail.  The Spark History server only supports one history file though.  Under the default configs it keeps the first attempts history file. You can set the undocumented config spark.eventLog.overwrite to allow the follow on attempts to overwrite the first attempts history file.
> Note that in spark 1.2 not having the overwrite config set causes any following attempts to actually fail to run, in spark 1.3 they run and you just see a warning at the end of the attempts.
> It would be really nice to have an option that keeps all the attempts history files.  This way a user can go back and look at each one individually.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org