You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Nico Kruber (JIRA)" <ji...@apache.org> on 2018/10/31 17:25:00 UTC
[jira] [Updated] (FLINK-10356) Add sanity checks to
SpillingAdaptiveSpanningRecordDeserializer
[ https://issues.apache.org/jira/browse/FLINK-10356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nico Kruber updated FLINK-10356:
--------------------------------
Issue Type: Sub-task (was: Improvement)
Parent: FLINK-3462
> Add sanity checks to SpillingAdaptiveSpanningRecordDeserializer
> ---------------------------------------------------------------
>
> Key: FLINK-10356
> URL: https://issues.apache.org/jira/browse/FLINK-10356
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.6.0, 1.6.1, 1.7.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
> Labels: pull-request-available
>
> {{SpillingAdaptiveSpanningRecordDeserializer}} doesn't have any consistency checks for usage calls or serializers behaving properly, e.g. to read only as many bytes as available/promised for that record. At least these checks should be added:
> # Check that buffers have not been read from yet before adding them (this is an invariant {{SpillingAdaptiveSpanningRecordDeserializer}} works with and from what I can see, it is followed now.
> # Check that after deserialization, we actually consumed {{recordLength}} bytes
> ** If not, in the spanning deserializer, we currently simply skip the remaining bytes.
> ** But in the non-spanning deserializer, we currently continue from the wrong offset.
> # Protect against {{setNextBuffer}} being called before draining all available records
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)