[jira] [Created] (FLINK-2317) Stream Jobs with Windows Leave Lingering State

["Aljoscha Krettek (JIRA)" <ji...@apache.org>]

Aljoscha Krettek created FLINK-2317:
---------------------------------------

             Summary: Stream Jobs with Windows Leave Lingering State
                 Key: FLINK-2317
                 URL: https://issues.apache.org/jira/browse/FLINK-2317
             Project: Flink
          Issue Type: Bug
          Components: Streaming
    Affects Versions: 0.9, 0.9.1
            Reporter: Aljoscha Krettek
            Priority: Blocker


The problem is that some components in the windowing system create Threads that are not properly cleaned up after a Job finished (is canceled). Specifically these are the {{CentralCheck}} thread of the {{GroupedActiveDiscretizer}} and the {{TimeCheck}} thread in {{TimeTriggerPolicy}}. These threads have references to other parts of the windowing system, such as the window buffers. This then means that the complete state of the buffers at the time a job is canceled stays around indefinitely. Over time this will fill up the memory and make successive windowing jobs incredibly slow or lead to GC overheat exceptions.

Also, to prevent stuff like this from happening in the future we should put all streaming Tests on a common {{TestBase}} that verifies that we have no lingering threads after job execution.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)