[jira] [Resolved] (FLINK-1326) The output serializers may consume excessive memory

["Stephan Ewen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephan Ewen resolved FLINK-1326.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 0.9

Fixed in bc69b0bb530a6daeea2c18b1ff9edbb4d1faa48c

> The output serializers may consume excessive memory
> ---------------------------------------------------
>
>                 Key: FLINK-1326
>                 URL: https://issues.apache.org/jira/browse/FLINK-1326
>             Project: Flink
>          Issue Type: Bug
>          Components: Distributed Runtime
>    Affects Versions: 0.8
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>             Fix For: 0.9
>
>
> The SpanningRecordSerializer creates large byte arrays to buffer records during serialization. Since they never release the records, they may occupy a large amount of memory. Currently, we use one per output partition, which is due to the original architecture.
> There are two ways to fix this:
>  1. Use only one serializer per output, rather than one per output partition. This is the preferable way in the long run anyways.
>  2. Release the arrays in the serializer (quick fix).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)