[jira] [Created] (FLINK-19023) Remove pruning of Record Serializer Buffer

["Stephan Ewen (Jira)" <ji...@apache.org>]

Stephan Ewen created FLINK-19023:
------------------------------------

             Summary: Remove pruning of Record Serializer Buffer
                 Key: FLINK-19023
                 URL: https://issues.apache.org/jira/browse/FLINK-19023
             Project: Flink
          Issue Type: Sub-task
          Components: Runtime / Network
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 1.12.0


Currently, the {{SpanningRecordSerializer}} prunes its internal serialization buffer under special circumstances:

  - The buffer becomes larger than a certain threshold (5MB)
  - The full record end lines up exactly with a full buffer length (this change got introduced at some point, it is not clear what the purpose is)

This optimization virtually never kicks in (because of the second condition) and also seems unnecessary. There is only a single serializer on the sender side, so this will not help to reduce the maximum memory footprint needed in any way.

NOTE: A similar optimization on the reader side ({{SpillingAdaptiveSpanningRecordDeserializer}}) makes sense, because multiple parallel deserializers run in order to piece together the records when retrieving buffers from the network in arbitrary order. Truncating buffers (or spilling) there helps reduce the maximum required memory footprint.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)