You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/01/19 16:19:26 UTC
[jira] [Commented] (FLINK-5576) extend deserialization functions of
KvStateRequestSerializer to detect unconsumed bytes
[ https://issues.apache.org/jira/browse/FLINK-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830180#comment-15830180 ]
ASF GitHub Bot commented on FLINK-5576:
---------------------------------------
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/3174
[FLINK-5576] extend deserialization functions of KvStateRequestSerializer to detect unconsumed bytes
`KvStateRequestSerializer#deserializeValue()` deserializes a given byte array. This is used by clients and unit tests and it is fair to assume that these byte arrays represent a complete value since we do not offer a method to continue reading from the middle of the array anyway. Therefore, we can treat unconsumed bytes as errors, e.g. from a wrong serializer being used, and throw an `IOException` with an appropriate failure message.
Similarly, this adds a better failure message to exceptions in `KvStateRequestSerializer#deserializeList()` by wrapping the original `IOException` into a new one with an appropriate error message just as in #3172.
The new unit tests require #3171 to be accepted first on which this PR is also based.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-5576
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3174.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3174
----
commit e29d5ef3099d5f5ffc817f3e5cdbb79c523b34c7
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-18T17:52:57Z
[FLINK-5561] fix DataInputDeserializer#available() 1 smaller than correct
commit e589058e92d1f2a57d242987327777baa1fe64d4
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-19T16:06:00Z
[FLINK-5576] let KvStateRequestSerializer#deserializeValue() detect unconsumed bytes
KvStateRequestSerializer#deserializeValue deserializes a given byte array. This is used by clients and unit tests and it is fair to assume that these byte arrays represent a complete value since we do not offer a method to continue reading from the middle of the array anyway. Therefore, we can treat unconsumed bytes as errors, e.g. from a wrong serializer being used, and throw a IOException with an appropriate failure message.
commit 12d2108333b23e01b7b978a08e868a5f888506f9
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-19T16:10:54Z
[FLINK-5576] add a better failure message to exceptions in KvStateRequestSerializer#deserializeList()
As in FLINK-5559, wrap the original IOException into a new one with an
appropriate error message to better diagnose it.
commit 9770e7f3269adfa5b80dd2125bcdbf8cebc4dab8
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-19T16:12:02Z
[FLINK-5576] add more unit tests for KvStateRequestSerializer#deserializeList()
These tests ensure that some special cases not properly tested before are
handled correctly in future.
----
> extend deserialization functions of KvStateRequestSerializer to detect unconsumed bytes
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-5576
> URL: https://issues.apache.org/jira/browse/FLINK-5576
> Project: Flink
> Issue Type: Improvement
> Components: Queryable State
> Affects Versions: 1.2.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Minor
>
> KvStateRequestSerializer#deserializeValue and KvStateRequestSerializer#deserializeList both deserialize a given byte array. This is used by clients and unit tests and it is fair to assume that these byte arrays represent a complete value since we do not offer a method to continue reading from the middle of the array anyway. Therefore, we can treat unconsumed bytes as errors, e.g. from a wrong serializer being used, and throw a IOException with an appropriate failure message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)