[jira] [Closed] (FLINK-19023) Remove pruning of Record Serializer Buffer

["Stephan Ewen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-19023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephan Ewen closed FLINK-19023.
--------------------------------

> Remove pruning of Record Serializer Buffer
> ------------------------------------------
>
>                 Key: FLINK-19023
>                 URL: https://issues.apache.org/jira/browse/FLINK-19023
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Network
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
>             Fix For: 1.12.0
>
>
> Currently, the {{SpanningRecordSerializer}} prunes its internal serialization buffer under special circumstances:
>   - The buffer becomes larger than a certain threshold (5MB)
>   - The full record end lines up exactly with a full buffer length (this change got introduced at some point, it is not clear what the purpose is)
> This optimization virtually never kicks in (because of the second condition) and also seems unnecessary. There is only a single serializer on the sender side, so this will not help to reduce the maximum memory footprint needed in any way.
> NOTE: A similar optimization on the reader side ({{SpillingAdaptiveSpanningRecordDeserializer}}) makes sense, because multiple parallel deserializers run in order to piece together the records when retrieving buffers from the network in arbitrary order. Truncating buffers (or spilling) there helps reduce the maximum required memory footprint.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)