[jira] [Created] (SPARK-43861) History Server delete event log of running Spark applicaiton

["shengkui leng (Jira)" <ji...@apache.org>]

shengkui leng created SPARK-43861:
-------------------------------------

             Summary: History Server delete event log of running Spark applicaiton
                 Key: SPARK-43861
                 URL: https://issues.apache.org/jira/browse/SPARK-43861
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.4.0
            Reporter: shengkui leng


When enable Spark history cleaner:

{noformat}
spark.history.fs.cleaner.enabled=true
spark.history.fs.cleaner.interval=1d
spark.history.fs.cleaner.maxAge=7d
spark.history.fs.cleaner.maxNum=10
{noformat}

Spark will delete history for running Spark applications, following is some message picked from the spark log:

{noformat}
153 23/05/26 14:19:43 INFO FsHistoryProvider: Parsing /tmp/spark-events/local-1685081895370 to re-build UI...
154 23/05/26 14:19:43 INFO FsHistoryProvider: Finished parsing /tmp/spark-events/local-1685081895370
155 23/05/28 13:09:53 INFO FsHistoryProvider: Deleting expired event log for app-20230524141002-0000
156 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for local-1684999095807
157 23/05/29 13:09:53 INFO FsHistoryProvider: Deleting expired event log for app-20230524171730-0000.inprogress
158 23/05/29 13:34:24 INFO ApplicationCache: Failed to load application attempt app-20230524171730-0000/
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org