You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2019/10/23 23:27:00 UTC

[jira] [Created] (SPARK-29581) Enable cleanup old event log files

Jungtaek Lim created SPARK-29581:
------------------------------------

             Summary: Enable cleanup old event log files 
                 Key: SPARK-29581
                 URL: https://issues.apache.org/jira/browse/SPARK-29581
             Project: Spark
          Issue Type: Sub-task
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Jungtaek Lim


This issue can be start only with SPARK-29579 is addressed properly.

After SPARK-29579 Spark would guarantee strong compatibility on both live entities and snapshots, which means snapshot file could replace older origin event log files. This issue tracks the efforts on automatically cleaning up old event logs if snapshot file can replace them, which lets overall size of event log on streaming query to be manageable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org