You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Michael Oleske (JIRA)" <ji...@apache.org> on 2018/05/29 17:39:00 UTC
[jira] [Assigned] (GEODE-5253) PDX Object corrupted in remove(K,V)
or put(K,V,V) operations
[ https://issues.apache.org/jira/browse/GEODE-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Oleske reassigned GEODE-5253:
-------------------------------------
Assignee: Michael Oleske
> PDX Object corrupted in remove(K,V) or put(K,V,V) operations
> ------------------------------------------------------------
>
> Key: GEODE-5253
> URL: https://issues.apache.org/jira/browse/GEODE-5253
> Project: Geode
> Issue Type: Improvement
> Components: serialization
> Reporter: Bruce Schuchardt
> Assignee: Michael Oleske
> Priority: Major
>
> A regression test ran into corruption in the expectedValue argument of remove(K,V) and put(K,V,V) operations when readPdxSerialized was enabled in clients and servers. Here's an example:
> {noformat}
> bridgegemfire5_28694/system.log: [error 2018/05/24 11:55:13.360 PDT bridgegemfire5_trout_28694 <PartitionedRegion Message Processor3> tid=0x92] Caught Exception
> org.apache.geode.pdx.PdxSerializationException: Exception deserializing a PDX field
> at org.apache.geode.pdx.internal.PdxInputStream.readObject(PdxInputStream.java:250)
> at org.apache.geode.pdx.internal.PdxInputStream.readObject(PdxInputStream.java:93)
> at org.apache.geode.pdx.internal.PdxReaderImpl.readObject(PdxReaderImpl.java:333)
> at org.apache.geode.pdx.internal.PdxInstanceImpl.readObject(PdxInstanceImpl.java:560)
> at org.apache.geode.pdx.internal.PdxInstanceImpl.equals(PdxInstanceImpl.java:408)
> at org.apache.geode.internal.cache.entries.AbstractRegionEntry.checkPdxEquals(AbstractRegionEntry.java:1163)
> at org.apache.geode.internal.cache.entries.AbstractRegionEntry.checkEquals(AbstractRegionEntry.java:1030)
> at org.apache.geode.internal.cache.entries.AbstractRegionEntry.checkExpectedOldValue(AbstractRegionEntry.java:955)
> at org.apache.geode.internal.cache.entries.AbstractRegionEntry.destroy(AbstractRegionEntry.java:829)
> at org.apache.geode.internal.cache.map.RegionMapDestroy.destroyEntry(RegionMapDestroy.java:723)
> at org.apache.geode.internal.cache.map.RegionMapDestroy.destroyExistingEntry(RegionMapDestroy.java:387)
> at org.apache.geode.internal.cache.map.RegionMapDestroy.handleExistingRegionEntry(RegionMapDestroy.java:238)
> at org.apache.geode.internal.cache.map.RegionMapDestroy.destroy(RegionMapDestroy.java:149)
> at org.apache.geode.internal.cache.AbstractRegionMap.destroy(AbstractRegionMap.java:1035)
> at org.apache.geode.internal.cache.LocalRegion.mapDestroy(LocalRegion.java:6544)
> at org.apache.geode.internal.cache.LocalRegion.mapDestroy(LocalRegion.java:6518)
> at org.apache.geode.internal.cache.BucketRegion.basicDestroy(BucketRegion.java:1194)
> at org.apache.geode.internal.cache.PartitionedRegionDataStore.destroyLocally(PartitionedRegionDataStore.java:1330)
> at org.apache.geode.internal.cache.PartitionedRegionDataView.destroyOnRemote(PartitionedRegionDataView.java:107)
> at org.apache.geode.internal.cache.partitioned.DestroyMessage.operateOnPartitionedRegion(DestroyMessage.java:268)
> at org.apache.geode.internal.cache.partitioned.PartitionMessage.process(PartitionMessage.java:334)
> at org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:378)
> at org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:444)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at org.apache.geode.distributed.internal.ClusterDistributionManager.runUntilShutdown(ClusterDistributionManager.java:1121)
> at org.apache.geode.distributed.internal.ClusterDistributionManager.access$000(ClusterDistributionManager.java:109)
> at org.apache.geode.distributed.internal.ClusterDistributionManager$8$1.run(ClusterDistributionManager.java:945)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Unknown header byte: 116
> at org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:3100)
> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2978)
> at org.apache.geode.pdx.internal.PdxInputStream.readObject(PdxInputStream.java:248)
> ... 28 more
> {noformat}
> I was able to reproduce this and found that the bytes in the PdxInstance were corrupted. Sometimes the length of the underlying byte buffer (which holds the serialized state of the object) was wrong, sometimes other bytes had been copied over the state of the PdxInstance leading to "bad header byte" errors, BufferUnderflowExceptions, etc.
> I tracked this down to the PDX implementation having borrowed heavily from the TCPConduit buffer management classes. If an object is deserialized into PdxInstance form in a P2P reader thread, the object retains a reference to the area in the P2P buffer that holds its serialized state. If the object is then handed off to another thread, such as an executor thread, the object still points back to the P2P buffer. If the P2P reader thread then goes on to read another message it will overwrite the buffer and corrupt the state of the PdxInstance.
> This is more likely to happen with conserve-sockets=true since a thread-owned connection will typically only handle one message at a time and will directly execute the method in the P2P reader thread.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)