You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Maximilian Michels (JIRA)" <ji...@apache.org> on 2015/04/08 11:29:13 UTC

[jira] [Created] (FLINK-1843) Job History gets cleared too fast

Maximilian Michels created FLINK-1843:
-----------------------------------------

             Summary: Job History gets cleared too fast
                 Key: FLINK-1843
                 URL: https://issues.apache.org/jira/browse/FLINK-1843
             Project: Flink
          Issue Type: Bug
          Components: JobManager
    Affects Versions: 0.9
            Reporter: Maximilian Michels
             Fix For: 0.9


As per FLINK-1442, the JobManager stores the archived ExecutionGraph behind a SoftReference. At least for local setups, this mechanism doesn't seem to work properly. There are two issues:

- The history gets cleared too fast
- The history gets cleared in a non-sequential fashion, i.e. arbitrary old ExecutionGraph are discarded

To solve these problems we might

- Store the least recent ExecutionGraph behind a SoftReference
- Store the most recent ExecutionGraphs without a SoftReference

That way, we can save memory but have the latest history available to the user. We might introduce a configuration variable where the user can specify the number of ExecutionGraphs that should be held in memory. The remaining can be stored behind a SoftReference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)