[jira] [Created] (GEODE-8501) Thread waiting for a response hangs due to a deserialization error

["Bruce J Schuchardt (Jira)" <ji...@apache.org>]

Bruce J Schuchardt created GEODE-8501:
-----------------------------------------

             Summary: Thread waiting for a response hangs due to a deserialization error
                 Key: GEODE-8501
                 URL: https://issues.apache.org/jira/browse/GEODE-8501
             Project: Geode
          Issue Type: Improvement
          Components: messaging
    Affects Versions: 1.14.0
            Reporter: Bruce J Schuchardt


There was a failure that caused a hang due to a FetchEntryMessage reply failing to deserialize.  This reply message had already deserialized its reply processor ID, so there's no reason that the waiting thread couldn't have been notified of the error so that it could wake up and process the problem.

 
{noformat}
[fatal 2020/09/16 14:04:59.318 PDT <P2P message reader for rs-GEM-3059-QQ1128-1a0i32xlarge-hydra-client-36(peergemfire_2_3_host1_14238:14238)<ec><v37>:41011(version:VersionOrdinal[ordinal=125]) shared unordered uid=1 local port=50589 remote port=51132> tid=0x46] Error deserializing message[fatal 2020/09/16 14:04:59.318 PDT <P2P message reader for rs-GEM-3059-QQ1128-1a0i32xlarge-hydra-client-36(peergemfire_2_3_host1_14238:14238)<ec><v37>:41011(version:VersionOrdinal[ordinal=125]) shared unordered uid=1 local port=50589 remote port=51132> tid=0x46] Error deserializing messagejava.io.IOException: Could not create an instance of org.apache.geode.internal.cache.partitioned.FetchEntryMessage$FetchEntryReplyMessage . at org.apache.geode.internal.serialization.internal.DSFIDSerializerImpl.invokeFromData(DSFIDSerializerImpl.java:330) at org.apache.geode.internal.serialization.internal.DSFIDSerializerImpl.create(DSFIDSerializerImpl.java:368) at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:1024) at org.apache.geode.internal.InternalDataSerializer.readDSFID(InternalDataSerializer.java:2387) at org.apache.geode.internal.InternalDataSerializer.readDSFID(InternalDataSerializer.java:2401) at org.apache.geode.internal.tcp.Connection.readMessage(Connection.java:2930) at org.apache.geode.internal.tcp.Connection.processInputBuffer(Connection.java:2751) at org.apache.geode.internal.tcp.Connection.readMessages(Connection.java:1619) at org.apache.geode.internal.tcp.Connection.run(Connection.java:1456) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException: Unknown header byte 109 at org.apache.geode.internal.serialization.DscodeHelper.toDSCODE(DscodeHelper.java:40) at org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2494) at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2864) at org.apache.geode.DataSerializer.readLinkedList(DataSerializer.java:2096) at org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2574) at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2864) at org.apache.geode.internal.cache.NonLocalRegionEntry.fromData(NonLocalRegionEntry.java:159) at org.apache.geode.internal.cache.EntrySnapshot.fromData(EntrySnapshot.java:282) at org.apache.geode.internal.cache.EntrySnapshot.<init>(EntrySnapshot.java:253) at org.apache.geode.internal.cache.partitioned.FetchEntryMessage$FetchEntryReplyMessage.fromData(FetchEntryMessage.java:309) at org.apache.geode.internal.serialization.internal.DSFIDSerializerImpl.invokeFromData(DSFIDSerializerImpl.java:317) ... 11 more {noformat}
Deserialization code for cache operations in Connection.java already handles this sort of thing by using a thread local that is updated by the fromData method of each cache operation.  ReplyMessage could do the same thing with a new thread local and Connection.java could check that thread local in the event of a deserialization error.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)