You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Stephan Ewen (JIRA)" <ji...@apache.org> on 2015/05/12 21:57:01 UTC
[jira] [Created] (FLINK-2004) Memory leack in presence of failed
checkpoints in KafkaSource
Stephan Ewen created FLINK-2004:
-----------------------------------
Summary: Memory leack in presence of failed checkpoints in KafkaSource
Key: FLINK-2004
URL: https://issues.apache.org/jira/browse/FLINK-2004
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 0.9
Reporter: Stephan Ewen
Assignee: Robert Metzger
Priority: Critical
Fix For: 0.9
Checkpoints that fail never send a commit message to the tasks.
Maintaining a map of all pending checkpoints introduces a memory leak, as entries for failed checkpoints will never be removed.
Approaches to fix this:
- The source cleans up entries from older checkpoints once a checkpoint is committed (simple implementation in a linked hash map)
- The commit message could include the optional state handle (source needs not maintain the map)
- The checkpoint coordinator could send messages for failed checkpoints?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)