You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2017/01/19 14:10:48 UTC
[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/3172
[FLINK-5559] let KvStateRequestSerializer#deserializeKeyAndNamespace() throw a proper IOException
This adds the hint that a deserialisation failure probably results from a `"mismatch in the key/namespace serializers used by the KvState instance and this access"` to all thrown exceptions and throws proper `IOException` instances instead of `IllegalArgumentException`.
The new unit tests require #3171 to be accepted first on which this PR is also based.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-5559
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3172.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3172
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on a diff in the pull request:
https://github.com/apache/flink/pull/3172#discussion_r97100430
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java ---
@@ -377,22 +376,24 @@ public static Throwable deserializeServerFailure(ByteBuf buf) throws IOException
0,
serializedKeyAndNamespace.length);
- K key = keySerializer.deserialize(dis);
- byte magicNumber = dis.readByte();
- if (magicNumber != 42) {
- throw new IllegalArgumentException("Unexpected magic number " + magicNumber +
- ". This indicates a mismatch in the key serializers used by the " +
- "KvState instance and this access.");
- }
- N namespace = namespaceSerializer.deserialize(dis);
+ try {
+ K key = keySerializer.deserialize(dis);
+ byte magicNumber = dis.readByte();
+ if (magicNumber != 42) {
--- End diff --
This was an historic artifact of the Flink 1.1 key and namespace serialization with RocksDB. I think we can remove this completely in a different issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on a diff in the pull request:
https://github.com/apache/flink/pull/3172#discussion_r97099962
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java ---
@@ -377,22 +376,24 @@ public static Throwable deserializeServerFailure(ByteBuf buf) throws IOException
0,
serializedKeyAndNamespace.length);
- K key = keySerializer.deserialize(dis);
- byte magicNumber = dis.readByte();
- if (magicNumber != 42) {
- throw new IllegalArgumentException("Unexpected magic number " + magicNumber +
- ". This indicates a mismatch in the key serializers used by the " +
- "KvState instance and this access.");
- }
- N namespace = namespaceSerializer.deserialize(dis);
+ try {
+ K key = keySerializer.deserialize(dis);
+ byte magicNumber = dis.readByte();
+ if (magicNumber != 42) {
+ throw new IOException("Unexpected magic number " + magicNumber + ".");
+ }
+ N namespace = namespaceSerializer.deserialize(dis);
- if (dis.available() > 0) {
- throw new IllegalArgumentException("Unconsumed bytes in the serialized key " +
- "and namespace. This indicates a mismatch in the key/namespace " +
- "serializers used by the KvState instance and this access.");
- }
+ if (dis.available() > 0) {
+ throw new IOException("Unconsumed bytes in the serialized key and namespace.");
+ }
- return new Tuple2<>(key, namespace);
+ return new Tuple2<>(key, namespace);
+ } catch (IOException e) {
--- End diff --
Do you think it makes sense to directly throw the final exception in the method body? The reported stack traces are usually already very deep.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/3172#discussion_r97279222
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java ---
@@ -377,22 +376,24 @@ public static Throwable deserializeServerFailure(ByteBuf buf) throws IOException
0,
serializedKeyAndNamespace.length);
- K key = keySerializer.deserialize(dis);
- byte magicNumber = dis.readByte();
- if (magicNumber != 42) {
- throw new IllegalArgumentException("Unexpected magic number " + magicNumber +
- ". This indicates a mismatch in the key serializers used by the " +
- "KvState instance and this access.");
- }
- N namespace = namespaceSerializer.deserialize(dis);
+ try {
+ K key = keySerializer.deserialize(dis);
+ byte magicNumber = dis.readByte();
+ if (magicNumber != 42) {
+ throw new IOException("Unexpected magic number " + magicNumber + ".");
+ }
+ N namespace = namespaceSerializer.deserialize(dis);
- if (dis.available() > 0) {
- throw new IllegalArgumentException("Unconsumed bytes in the serialized key " +
- "and namespace. This indicates a mismatch in the key/namespace " +
- "serializers used by the KvState instance and this access.");
- }
+ if (dis.available() > 0) {
+ throw new IOException("Unconsumed bytes in the serialized key and namespace.");
+ }
- return new Tuple2<>(key, namespace);
+ return new Tuple2<>(key, namespace);
+ } catch (IOException e) {
--- End diff --
it may, but there are other `IOException`s in deeper code, e.g. in `dis.readByte();` and I did not want any special code separating these from our `IOException`s and considered this the least obtrusive variant
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3172
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3172: [FLINK-5559] let KvStateRequestSerializer#deserializeKeyA...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the issue:
https://github.com/apache/flink/pull/3172
I agree that it is more appropriate to throw the `IOException`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---