You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jason Brown (JIRA)" <ji...@apache.org> on 2018/07/20 04:40:00 UTC
[jira] [Comment Edited] (CASSANDRA-14574) Incomplete handling of
exceptions when decoding incoming messages
[ https://issues.apache.org/jira/browse/CASSANDRA-14574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550234#comment-16550234 ]
Jason Brown edited comment on CASSANDRA-14574 at 7/20/18 4:39 AM:
------------------------------------------------------------------
bq. longer important, since the source is now known, but I got a different instance of what I think is the same bug, triggered here
Yup, looks like the broken behavior of constantly evaluating the buffer even though we're in a bad (incorrect) spot in the stream.
bq. I'm wondering however if maybe this is a good time to improve handling of errors like this by skipping the remaining payload bytes, to leave the stream in a valid state, without closing down connections.
I agree, especially after seeing that we drop the connection when we get a message for a table we don't know about yet. (I'll have to spelunk the git log to uncover the original logic for that one). That's one example, of course. However, I'd like to separate that reevaluation effort from resolving this issue. That way we can unblock this faster.
was (Author: jasobrown):
bq. longer important, since the source is now known, but I got a different instance of what I think is the same bug, triggered here
Yup, looks like the broken behavior of constantly evaluating the buffer even though we're in a bad (incorrect) spot in the stream.
bq. I'm wondering however if maybe this is a good time to improve handling of errors like this by skipping the remaining payload bytes, to leave the stream in a valid state, without closing down connections.
I agree, especially after seeing that we drop the connection when we get a message for a table we don't know about yet. (I'll have to spelunk the git log to uncover the original logic for that one). That's one example, of course. That being said, however, I'd like to separate that effort from resolving this issue. That way we can unblock this faster.
> Incomplete handling of exceptions when decoding incoming messages
> ------------------------------------------------------------------
>
> Key: CASSANDRA-14574
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14574
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Reporter: Aleksey Yeschenko
> Assignee: Jason Brown
> Priority: Major
> Fix For: 4.0
>
>
> {{MessageInHandler.decode()}} occasionally reads the payload incorrectly, passing the full message to {{MessageIn.read()}} instead of just the payload bytes.
> You can see the stack trace in the logs from this [CI run|https://circleci.com/gh/iamaleksey/cassandra/437#tests/containers/38].
> {code}
> Caused by: java.lang.AssertionError: null
> at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:351)
> at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:371)
> at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:335)
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:158)
> at org.apache.cassandra.net.async.MessageInHandler.decode(MessageInHandler.java:132)
> {code}
> Reconstructed, truncated stream passed to {{MessageIn.read()}}:
> {{0000000b000743414c5f42414301002a01e1a5c9b089fd11e8b517436ee1243007040000005d10fc50ec}}
> You can clearly see parameters in there encoded before the payload:
> {{[43414c5f424143 - CAL_BAC] [01 - ONE_BYTE] [002a - 42, payload size] 01 e1 a5 c9 b0 89 fd 11 e8 b5 17 43 6e e1 24 30 07 04 00 00 00 1d 10 fc 50 ec}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org