You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Stephan Ewen (JIRA)" <ji...@apache.org> on 2014/12/15 12:44:13 UTC

[jira] [Created] (FLINK-1326) The output serializers may consume excessive memory

Stephan Ewen created FLINK-1326:
-----------------------------------

             Summary: The output serializers may consume excessive memory
                 Key: FLINK-1326
                 URL: https://issues.apache.org/jira/browse/FLINK-1326
             Project: Flink
          Issue Type: Bug
          Components: Distributed Runtime
    Affects Versions: 0.8-incubating
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen


The SpanningRecordSerializer creates large byte arrays to buffer records during serialization. Since they never release the records, they may occupy a large amount of memory. Currently, we use one per output partition, which is due to the original architecture.

There are two ways to fix this:

 1. Use only one serializer per output, rather than one per output partition. This is the preferable way in the long run anyways.

 2. Release the arrays in the serializer (quick fix).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)