You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Barry Oglesby (JIRA)" <ji...@apache.org> on 2018/10/23 22:22:00 UTC
[jira] [Updated] (GEODE-5917) Gfsh query results show a mix of
PdxInstances and PreferBytesCachedDeserializables with read-serialized=true
[ https://issues.apache.org/jira/browse/GEODE-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Barry Oglesby updated GEODE-5917:
---------------------------------
Description:
Gfsh query results show a mix of PdxInstances and PreferBytesCachedDeserializables with read-serialized=true
A gfsh query on a partitioned region with pdx read-serialized=true shows results like:
{noformat}
shares | price | id | cusip | serializedValue | sizeInBytes | stringForm | valueSizeInBytes | DSFID | deserializedForReading | value | serialized
------ | ----------------- | ------ | ------ | ------------------ | ----------- | -------------------------------------------------------------------------- | ---------------- | ------ | --------------------------------------------- | ------------------ | ----------
70 | 590.923583984375 | 0 | MCD | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
77 | 740.6094970703125 | 3 | MGM | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=GGB, id=1, price=26.52454376220703, shares=49} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=STO, id=2, price=643.344482421875, shares=85} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=MGM, id=4, price=724.223388671875, shares=0} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
{noformat}
In this case, there are 2 servers and no redundant copies.
The DataCommandFunction.select query returns:
{noformat}
DataCommandFunction.select results=CumulativeNonDistinctResults::[
PDX[4456129,TradePdx]{cusip=MCD, id=0, price=590.923583984375, shares=70},
PDX[4456129,TradePdx]{cusip=MGM, id=3, price=740.6094970703125, shares=77},
PreferBytesCachedDeserializable@1599752189,
PreferBytesCachedDeserializable@1120782877,
PreferBytesCachedDeserializable@1023583807
]
{noformat}
The local query returns the 2 PdxInstances, and the remote query returns the 3 PreferBytesCachedDeserializables:
{noformat}
[info 2018/10/23 13:53:18.046 PDT <Function Execution Processor1> tid=0x4f] Trace Info for Query: SELECT * FROM /data limit 100
Local 192.168.2.6(76490)<v10>:1026 took 6.887ms and returned 2 results;
Remote 192.168.2.6(76479)<v9>:1025 took 45.164ms and returned 3 results; indexesUsed(0)
{noformat}
The 3 PreferBytesCachedDeserializables are not converted to PdxInstances before they are returned.
PartitionedRegionQueryEvaluator.addResultsToResultSet adds the results to the CumulativeNonDistinctResults result set.
The CumulativeCollectionIterator iterates the CumulativeNonDistinctResultsCollection and converts the objects to PDX here:
{noformat}
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1333)
at org.apache.geode.cache.query.internal.utils.PDXUtils.convertPDX(PDXUtils.java:83)
at org.apache.geode.cache.query.internal.CumulativeNonDistinctResults$CumulativeNonDistinctResultsCollection$CumulativeCollectionIterator.next(CumulativeNonDistinctResults.java:259)
at org.apache.geode.cache.query.internal.utils.LimitIterator.next(LimitIterator.java:49)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select_SelectResults(DataCommandFunction.java:271)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:226)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:177)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.execute(DataCommandFunction.java:125)
at org.apache.geode.internal.cache.MemberFunctionStreamingMessage.process(MemberFunctionStreamingMessage.java:186)
at org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:382)
at org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:448)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.geode.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:1099)
at org.apache.geode.distributed.internal.DistributionManager.access$000(DistributionManager.java:108)
at org.apache.geode.distributed.internal.DistributionManager$9$1.run(DistributionManager.java:970)
at java.lang.Thread.run(Thread.java:745)
{noformat}
PDXUtils.convertPDX does this:
{noformat}
if (getDeserializedObject && obj instanceof VMCachedDeserializable) {
obj = ((VMCachedDeserializable) obj).getDeserializedForReading();
{noformat}
Changing the instanceof check to check for CachedDeserializable instead fixes the issue:
{noformat}
if (getDeserializedObject && obj instanceof CachedDeserializable) {
obj = ((CachedDeserializable) obj).getDeserializedForReading();
{noformat}
was:
A gfsh query on a partitioned region with pdx read-serialized=true shows results like:
```
shares | price | id | cusip | serializedValue | sizeInBytes | stringForm | valueSizeInBytes | DSFID | deserializedForReading | value | serialized
------ | ----------------- | ------ | ------ | ------------------ | ----------- | -------------------------------------------------------------------------- | ---------------- | ------ | --------------------------------------------- | ------------------ | ----------
70 | 590.923583984375 | 0 | MCD | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
77 | 740.6094970703125 | 3 | MGM | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]\{cusip=GGB, id=1, price=26.52454376220703, shares=49} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]\{cusip=STO, id=2, price=643.344482421875, shares=85} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
<NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]\{cusip=MGM, id=4, price=724.223388671875, shares=0} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
```
In this case, there are 2 servers and no redundant copies.
The DataCommandFunction.select query returns:
```
DataCommandFunction.select results=CumulativeNonDistinctResults::[
PDX[4456129,TradePdx]\{cusip=MCD, id=0, price=590.923583984375, shares=70},
PDX[4456129,TradePdx]\{cusip=MGM, id=3, price=740.6094970703125, shares=77},
PreferBytesCachedDeserializable@1599752189,
PreferBytesCachedDeserializable@1120782877,
PreferBytesCachedDeserializable@1023583807
]
```
The local query returns the 2 PdxInstances, and the remote query returns the 3 PreferBytesCachedDeserializables:
```
[info 2018/10/23 13:53:18.046 PDT <Function Execution Processor1> tid=0x4f] Trace Info for Query: SELECT * FROM /data limit 100
Local 192.168.2.6(76490)<v10>:1026 took 6.887ms and returned 2 results;
Remote 192.168.2.6(76479)<v9>:1025 took 45.164ms and returned 3 results; indexesUsed(0)
```
The 3 PreferBytesCachedDeserializables are not converted to PdxInstances before they are returned.
PartitionedRegionQueryEvaluator.addResultsToResultSet adds the results to the CumulativeNonDistinctResults result set.
The CumulativeCollectionIterator iterates the CumulativeNonDistinctResultsCollection and converts the objects to PDX here:
```
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1333)
at org.apache.geode.cache.query.internal.utils.PDXUtils.convertPDX(PDXUtils.java:83)
at org.apache.geode.cache.query.internal.CumulativeNonDistinctResults$CumulativeNonDistinctResultsCollection$CumulativeCollectionIterator.next(CumulativeNonDistinctResults.java:259)
at org.apache.geode.cache.query.internal.utils.LimitIterator.next(LimitIterator.java:49)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select_SelectResults(DataCommandFunction.java:271)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:226)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:177)
at org.apache.geode.management.internal.cli.functions.DataCommandFunction.execute(DataCommandFunction.java:125)
at org.apache.geode.internal.cache.MemberFunctionStreamingMessage.process(MemberFunctionStreamingMessage.java:186)
at org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:382)
at org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:448)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.geode.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:1099)
at org.apache.geode.distributed.internal.DistributionManager.access$000(DistributionManager.java:108)
at org.apache.geode.distributed.internal.DistributionManager$9$1.run(DistributionManager.java:970)
at java.lang.Thread.run(Thread.java:745)
```
PDXUtils.convertPDX does this:
```
if (getDeserializedObject && obj instanceof VMCachedDeserializable) {
obj = ((VMCachedDeserializable) obj).getDeserializedForReading();
```
Changing the instanceof check to check for CachedDeserializable instead fixes the issue:
```
if (getDeserializedObject && obj instanceof CachedDeserializable) {
obj = ((CachedDeserializable) obj).getDeserializedForReading();
```
> Gfsh query results show a mix of PdxInstances and PreferBytesCachedDeserializables with read-serialized=true
> ------------------------------------------------------------------------------------------------------------
>
> Key: GEODE-5917
> URL: https://issues.apache.org/jira/browse/GEODE-5917
> Project: Geode
> Issue Type: Bug
> Components: querying
> Reporter: Barry Oglesby
> Priority: Major
>
> Gfsh query results show a mix of PdxInstances and PreferBytesCachedDeserializables with read-serialized=true
> A gfsh query on a partitioned region with pdx read-serialized=true shows results like:
> {noformat}
> shares | price | id | cusip | serializedValue | sizeInBytes | stringForm | valueSizeInBytes | DSFID | deserializedForReading | value | serialized
> ------ | ----------------- | ------ | ------ | ------------------ | ----------- | -------------------------------------------------------------------------- | ---------------- | ------ | --------------------------------------------- | ------------------ | ----------
> 70 | 590.923583984375 | 0 | MCD | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
> 77 | 740.6094970703125 | 3 | MGM | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL> | <NULL>
> <NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=GGB, id=1, price=26.52454376220703, shares=49} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
> <NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=STO, id=2, price=643.344482421875, shares=85} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
> <NULL> | <NULL> | <NULL> | <NULL> | org.json.JSONArray | 56 | PDX[4456129,TradePdx]{cusip=MGM, id=4, price=724.223388671875, shares=0} | 44 | -65 | org.apache.geode.pdx.internal.PdxInstanceImpl | org.json.JSONArray | true
> {noformat}
> In this case, there are 2 servers and no redundant copies.
> The DataCommandFunction.select query returns:
> {noformat}
> DataCommandFunction.select results=CumulativeNonDistinctResults::[
> PDX[4456129,TradePdx]{cusip=MCD, id=0, price=590.923583984375, shares=70},
> PDX[4456129,TradePdx]{cusip=MGM, id=3, price=740.6094970703125, shares=77},
> PreferBytesCachedDeserializable@1599752189,
> PreferBytesCachedDeserializable@1120782877,
> PreferBytesCachedDeserializable@1023583807
> ]
> {noformat}
> The local query returns the 2 PdxInstances, and the remote query returns the 3 PreferBytesCachedDeserializables:
> {noformat}
> [info 2018/10/23 13:53:18.046 PDT <Function Execution Processor1> tid=0x4f] Trace Info for Query: SELECT * FROM /data limit 100
> Local 192.168.2.6(76490)<v10>:1026 took 6.887ms and returned 2 results;
> Remote 192.168.2.6(76479)<v9>:1025 took 45.164ms and returned 3 results; indexesUsed(0)
> {noformat}
> The 3 PreferBytesCachedDeserializables are not converted to PdxInstances before they are returned.
> PartitionedRegionQueryEvaluator.addResultsToResultSet adds the results to the CumulativeNonDistinctResults result set.
> The CumulativeCollectionIterator iterates the CumulativeNonDistinctResultsCollection and converts the objects to PDX here:
> {noformat}
> java.lang.Exception: Stack trace
> at java.lang.Thread.dumpStack(Thread.java:1333)
> at org.apache.geode.cache.query.internal.utils.PDXUtils.convertPDX(PDXUtils.java:83)
> at org.apache.geode.cache.query.internal.CumulativeNonDistinctResults$CumulativeNonDistinctResultsCollection$CumulativeCollectionIterator.next(CumulativeNonDistinctResults.java:259)
> at org.apache.geode.cache.query.internal.utils.LimitIterator.next(LimitIterator.java:49)
> at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select_SelectResults(DataCommandFunction.java:271)
> at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:226)
> at org.apache.geode.management.internal.cli.functions.DataCommandFunction.select(DataCommandFunction.java:177)
> at org.apache.geode.management.internal.cli.functions.DataCommandFunction.execute(DataCommandFunction.java:125)
> at org.apache.geode.internal.cache.MemberFunctionStreamingMessage.process(MemberFunctionStreamingMessage.java:186)
> at org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:382)
> at org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:448)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at org.apache.geode.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:1099)
> at org.apache.geode.distributed.internal.DistributionManager.access$000(DistributionManager.java:108)
> at org.apache.geode.distributed.internal.DistributionManager$9$1.run(DistributionManager.java:970)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> PDXUtils.convertPDX does this:
> {noformat}
> if (getDeserializedObject && obj instanceof VMCachedDeserializable) {
> obj = ((VMCachedDeserializable) obj).getDeserializedForReading();
> {noformat}
> Changing the instanceof check to check for CachedDeserializable instead fixes the issue:
> {noformat}
> if (getDeserializedObject && obj instanceof CachedDeserializable) {
> obj = ((CachedDeserializable) obj).getDeserializedForReading();
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)