You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Stanislav Lukyanov <st...@gmail.com> on 2018/07/09 13:03:56 UTC
RE: OptimizedMarshaller instead of BinaryMarshaller is used
forScanQuery
Hi Calvin,
It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t even be used (except for JDK classes).
How did you check which marshaller is used in each case?
Can you share the code of your POJO? Or perhaps a runnable reproducer?
Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
Thanks,
Stan
From: Calvin KL Wong, CLSA
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Ok, I found that I can use binaryObject.deserialize() to get back the POJO. So, the only remaining question is:
When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle class versioning fine whereas ScanQuery cannot? Is that an expected behaviour?
Thanks,
Calvin
From: Calvin KL Wong, CLSA [mailto:calvin.kl.wong@clsa.com]
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery
Hi,
I have a cache of <Integer, POJO>.
I found that when I use a ScanQuery on that cache, result will be deserialized using OptimizedMarshaller; Ignite sends ‘ordered’ message using GridCacheQueryResonseEntry. Whereas if I use ContinousQuery, result will be deserialized using BinaryMarshaller.
My problem is that ScanQuery won’t be able to handle class versioning.
Question:
1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning related problem. Is that the recommended approach? Because it seems strange that “ScanQuery” and “ContinousQuery” have different behaviour.
2. What is the recommended way to convert BinaryObject into POJO? Ignite website talks about getting field of each object using ‘BinaryObject.field()’. However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally. How can I get a reference to that object?
Thanks,
Calvin
Calvin KL Wong
Sr. Lead Engineer, Execution Services
D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
clsa.com
Insights. Liquidity. Capital.
A CITIC Securities Company
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
RE: OptimizedMarshaller instead of BinaryMarshaller is
usedforScanQuery
Posted by "ilya.kasnacheev" <il...@gmail.com>.
Hello!
I have filed a ticket against Apache Ignite JIRA:
https://issues.apache.org/jira/browse/IGNITE-9054
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: OptimizedMarshaller instead of BinaryMarshaller is
usedforScanQuery
Posted by "Calvin KL Wong, CLSA" <ca...@clsa.com>.
Hi Stan,
Attached is a reproducer I created based on org.apache.ignite.examples.datagrid.CacheContinuousQueryExample. I am using a simple POJO to reproduce the behaviour I see.
Step:
1. Add those files to ignite-examples project, under package org.apache.ignite.examples.datagrid
2. Put a conditional breakpoint at OptimizedObjectInputStream.readObject0()
a. On the return line under “Case SERIALIZABLE”
b. The condition is:
i. curCls.getName().contains("Request")
c. [cid:image008.png@01D4190F.BD106880]
3. Run CacheContinuousQueryServerExample
4. Run CacheContinuousQueryClientPojoExample
a. You should hit the break point. This verifies that OptimizedMarshaller is used.
[cid:image009.png@01D4190F.BD106880]
5. Continue
6. Press enter in CacheContinuousQueryServerExample to add more objects into cache.
a. CacheContinuousQueryClientPojoExample will receive those updates.
b. You will notice that you won’t get stop at the breakpoint. This proves that OptimizedMarshaller isn’t being used in this case.
You may want to put some attention to the use of GridCacheQueryResponseEntry (which implements Externalizable) by Scan Query, but not by Continuous Query.
Thanks,
Calvin
From: Stanislav Lukyanov [mailto:stanlukyanov@gmail.com]
Sent: Tuesday, July 10, 2018 6:23 PM
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is usedforScanQuery
Hi Calvin,
It seems that the code is incomplete.
First, what are the Instant, DeltaMap types? What are the types stored in the Map fields?
Second, do you have some code processing tool generating some code for you? E.g. I see that version() implementation is missing in WorkingSetLedgerDelta – I assume it’s generated.
The reason why OptimizedMarshaller is used is most likely that some parts of your objects are Serializable with custom serialization (readObject()/writeObject() methods).
Also, OptimizedMarshaller may be used when processing JDK classes.
The logs should have a message about that though.
Are you checking Ignite’s stdout or the log files?
Ignite starts in a quiet mode by default, so not all messages are in the stdout.
There is a path to the full log file printed in the stdout.
I’m still not sure whether I grasp your use case in full though.
Any chance you could create a small reproducer project and share it on GitHub?
Thanks,
Stan
From: Calvin KL Wong, CLSA<ma...@clsa.com>
Sent: 10 июля 2018 г. 6:06
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is usedforScanQuery
Hi Stan,
Thanks for your response.
>> How did you check which marshaller is used in each case?
i) from stack trace
ii) step with a debugger
>> Can you share the code of your POJO?
Attached pojo.zip contains skeleton of the POJO. The object that was passed between ignite nodes is called WorkingSetLedgerDelta. It contains objects of other classes.
>> Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
I just checked them – did a grep of ‘-i marsh’, ‘-i warn’, ‘-i except’, ‘-i error’; no marshaller related problem found.
Background:
A newer version of NewOrderRequest extends AbstractRequest is deployed on the service node, but not on the client node. Client node sends a scan query to server node to get all WorkingSetLedgerDelta, which contains Request objects.
About the attached stack traces of the client node
1. exception.when.using.pojo.with.scan.query.log
Cache: IgniteCache<LedgerKey, LedgerDelta>
Caught exception when trying to deserialize WorkingSetLedgerDelta
1. forced.exception.when.using.binary.object.with.scan.query.with.keep.binary.log
Cache: IgniteCache<BinaryObject, BinaryObject>.withKeepBinary()
I forced an exception from eclipse when I saw that we successfully unmarshalled bytes into the BinaryObjectImpl of WorkingSetLedgerDelta.
The main difference between the two stack traces is at:
with POJO, the logic unmarshalls WorkingSetLedgerDelta using ‘readSerializable’, which cannot handle versioning.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:601) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:927) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
withKeepBinary, BinaryObjImpl object was received, the logic could unmarshall the BinaryObjImpl of WorkingSetLedgerDelta with no problem.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:558) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
Thanks,
Calvin
From: Stanislav Lukyanov [mailto:stanlukyanov@gmail.com]
Sent: Monday, July 09, 2018 9:04 PM
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Hi Calvin,
It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t even be used (except for JDK classes).
How did you check which marshaller is used in each case?
Can you share the code of your POJO? Or perhaps a runnable reproducer?
Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
Thanks,
Stan
From: Calvin KL Wong, CLSA<ma...@clsa.com>
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Ok, I found that I can use binaryObject.deserialize() to get back the POJO. So, the only remaining question is:
When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle class versioning fine whereas ScanQuery cannot? Is that an expected behaviour?
Thanks,
Calvin
From: Calvin KL Wong, CLSA [mailto:calvin.kl.wong@clsa.com]
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery
Hi,
I have a cache of <Integer, POJO>.
I found that when I use a ScanQuery on that cache, result will be deserialized using OptimizedMarshaller; Ignite sends ‘ordered’ message using GridCacheQueryResonseEntry. Whereas if I use ContinousQuery, result will be deserialized using BinaryMarshaller.
My problem is that ScanQuery won’t be able to handle class versioning.
Question:
1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning related problem. Is that the recommended approach? Because it seems strange that “ScanQuery” and “ContinousQuery” have different behaviour.
2. What is the recommended way to convert BinaryObject into POJO? Ignite website talks about getting field of each object using ‘BinaryObject.field()’. However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally. How can I get a reference to that object?
Thanks,
Calvin
Calvin KL Wong
Sr. Lead Engineer, Execution Services
D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
[:1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]<https://hk.linkedin.com/company/clsa>[:1. Social Media Icons:CLSA_Social Media Icons_twitter.png]<https://twitter.com/clsainsights?lang=en>[:1. Social Media Icons:CLSA_Social Media Icons_youtube.png]<https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>[:1. Social Media Icons:CLSA_Social Media Icons_facebook.png]<https://www.facebook.com/clsacommunity/>
clsa.com<https://www.clsa.com/>
Insights. Liquidity. Capital.
[CLSA_RGB]<https://www.clsa.com/member>
A CITIC Securities Company
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
RE: OptimizedMarshaller instead of BinaryMarshaller is
usedforScanQuery
Posted by Stanislav Lukyanov <st...@gmail.com>.
Hi Calvin,
It seems that the code is incomplete.
First, what are the Instant, DeltaMap types? What are the types stored in the Map fields?
Second, do you have some code processing tool generating some code for you? E.g. I see that version() implementation is missing in WorkingSetLedgerDelta – I assume it’s generated.
The reason why OptimizedMarshaller is used is most likely that some parts of your objects are Serializable with custom serialization (readObject()/writeObject() methods).
Also, OptimizedMarshaller may be used when processing JDK classes.
The logs should have a message about that though.
Are you checking Ignite’s stdout or the log files?
Ignite starts in a quiet mode by default, so not all messages are in the stdout.
There is a path to the full log file printed in the stdout.
I’m still not sure whether I grasp your use case in full though.
Any chance you could create a small reproducer project and share it on GitHub?
Thanks,
Stan
From: Calvin KL Wong, CLSA
Sent: 10 июля 2018 г. 6:06
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is usedforScanQuery
Hi Stan,
Thanks for your response.
>> How did you check which marshaller is used in each case?
i) from stack trace
ii) step with a debugger
>> Can you share the code of your POJO?
Attached pojo.zip contains skeleton of the POJO. The object that was passed between ignite nodes is called WorkingSetLedgerDelta. It contains objects of other classes.
>> Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
I just checked them – did a grep of ‘-i marsh’, ‘-i warn’, ‘-i except’, ‘-i error’; no marshaller related problem found.
Background:
A newer version of NewOrderRequest extends AbstractRequest is deployed on the service node, but not on the client node. Client node sends a scan query to server node to get all WorkingSetLedgerDelta, which contains Request objects.
About the attached stack traces of the client node
1. exception.when.using.pojo.with.scan.query.log
Cache: IgniteCache<LedgerKey, LedgerDelta>
Caught exception when trying to deserialize WorkingSetLedgerDelta
2. forced.exception.when.using.binary.object.with.scan.query.with.keep.binary.log
Cache: IgniteCache<BinaryObject, BinaryObject>.withKeepBinary()
I forced an exception from eclipse when I saw that we successfully unmarshalled bytes into the BinaryObjectImpl of WorkingSetLedgerDelta.
The main difference between the two stack traces is at:
with POJO, the logic unmarshalls WorkingSetLedgerDelta using ‘readSerializable’, which cannot handle versioning.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:601) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:927) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
withKeepBinary, BinaryObjImpl object was received, the logic could unmarshall the BinaryObjImpl of WorkingSetLedgerDelta with no problem.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:558) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
Thanks,
Calvin
From: Stanislav Lukyanov [mailto:stanlukyanov@gmail.com]
Sent: Monday, July 09, 2018 9:04 PM
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Hi Calvin,
It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t even be used (except for JDK classes).
How did you check which marshaller is used in each case?
Can you share the code of your POJO? Or perhaps a runnable reproducer?
Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
Thanks,
Stan
From: Calvin KL Wong, CLSA
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Ok, I found that I can use binaryObject.deserialize() to get back the POJO. So, the only remaining question is:
When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle class versioning fine whereas ScanQuery cannot? Is that an expected behaviour?
Thanks,
Calvin
From: Calvin KL Wong, CLSA [mailto:calvin.kl.wong@clsa.com]
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery
Hi,
I have a cache of <Integer, POJO>.
I found that when I use a ScanQuery on that cache, result will be deserialized using OptimizedMarshaller; Ignite sends ‘ordered’ message using GridCacheQueryResonseEntry. Whereas if I use ContinousQuery, result will be deserialized using BinaryMarshaller.
My problem is that ScanQuery won’t be able to handle class versioning.
Question:
1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning related problem. Is that the recommended approach? Because it seems strange that “ScanQuery” and “ContinousQuery” have different behaviour.
2. What is the recommended way to convert BinaryObject into POJO? Ignite website talks about getting field of each object using ‘BinaryObject.field()’. However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally. How can I get a reference to that object?
Thanks,
Calvin
Calvin KL Wong
Sr. Lead Engineer, Execution Services
D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
clsa.com
Insights. Liquidity. Capital.
A CITIC Securities Company
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
RE: OptimizedMarshaller instead of BinaryMarshaller is used
forScanQuery
Posted by "Calvin KL Wong, CLSA" <ca...@clsa.com>.
Hi Stan,
Thanks for your response.
>> How did you check which marshaller is used in each case?
i) from stack trace
ii) step with a debugger
>> Can you share the code of your POJO?
Attached pojo.zip contains skeleton of the POJO. The object that was passed between ignite nodes is called WorkingSetLedgerDelta. It contains objects of other classes.
>> Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
I just checked them – did a grep of ‘-i marsh’, ‘-i warn’, ‘-i except’, ‘-i error’; no marshaller related problem found.
Background:
A newer version of NewOrderRequest extends AbstractRequest is deployed on the service node, but not on the client node. Client node sends a scan query to server node to get all WorkingSetLedgerDelta, which contains Request objects.
About the attached stack traces of the client node
1. exception.when.using.pojo.with.scan.query.log
Cache: IgniteCache<LedgerKey, LedgerDelta>
Caught exception when trying to deserialize WorkingSetLedgerDelta
2. forced.exception.when.using.binary.object.with.scan.query.with.keep.binary.log
Cache: IgniteCache<BinaryObject, BinaryObject>.withKeepBinary()
I forced an exception from eclipse when I saw that we successfully unmarshalled bytes into the BinaryObjectImpl of WorkingSetLedgerDelta.
The main difference between the two stack traces is at:
with POJO, the logic unmarshalls WorkingSetLedgerDelta using ‘readSerializable’, which cannot handle versioning.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:601) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:927) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
withKeepBinary, BinaryObjImpl object was received, the logic could unmarshall the BinaryObjImpl of WorkingSetLedgerDelta with no problem.
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:558) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:199) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) ~[?:1.8.0_152]
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90) ~[ignite-core-2.3.0-clsa.20180130.59.jar:2.3.0-clsa.20180130.59]
Thanks,
Calvin
From: Stanislav Lukyanov [mailto:stanlukyanov@gmail.com]
Sent: Monday, July 09, 2018 9:04 PM
To: user@ignite.apache.org
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Hi Calvin,
It should work the same for all queries. Ideally OptimizedMarshaller shouldn’t even be used (except for JDK classes).
How did you check which marshaller is used in each case?
Can you share the code of your POJO? Or perhaps a runnable reproducer?
Can you also share the logs/check them for warnings? There might be a warning saying that there is a problem in using BinaryMarshaller.
Thanks,
Stan
From: Calvin KL Wong, CLSA<ma...@clsa.com>
Sent: 9 июля 2018 г. 13:40
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: RE: OptimizedMarshaller instead of BinaryMarshaller is used forScanQuery
Ok, I found that I can use binaryObject.deserialize() to get back the POJO. So, the only remaining question is:
When a cache does not have ‘withKeepBinary’ set, why ContinuousQuery can handle class versioning fine whereas ScanQuery cannot? Is that an expected behaviour?
Thanks,
Calvin
From: Calvin KL Wong, CLSA [mailto:calvin.kl.wong@clsa.com]
Sent: Monday, July 09, 2018 11:50 AM
To: user@ignite.apache.org
Subject: OptimizedMarshaller instead of BinaryMarshaller is used for ScanQuery
Hi,
I have a cache of <Integer, POJO>.
I found that when I use a ScanQuery on that cache, result will be deserialized using OptimizedMarshaller; Ignite sends ‘ordered’ message using GridCacheQueryResonseEntry. Whereas if I use ContinousQuery, result will be deserialized using BinaryMarshaller.
My problem is that ScanQuery won’t be able to handle class versioning.
Question:
1. I am using “keepBinary” and BinaryObject on the cache to avoid versioning related problem. Is that the recommended approach? Because it seems strange that “ScanQuery” and “ContinousQuery” have different behaviour.
2. What is the recommended way to convert BinaryObject into POJO? Ignite website talks about getting field of each object using ‘BinaryObject.field()’. However, Ignite uses ‘BinaryReaderExImpl implements BinaryReader’ internally. How can I get a reference to that object?
Thanks,
Calvin
Calvin KL Wong
Sr. Lead Engineer, Execution Services
D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
[:1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]<https://hk.linkedin.com/company/clsa>[:1. Social Media Icons:CLSA_Social Media Icons_twitter.png]<https://twitter.com/clsainsights?lang=en>[:1. Social Media Icons:CLSA_Social Media Icons_youtube.png]<https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>[:1. Social Media Icons:CLSA_Social Media Icons_facebook.png]<https://www.facebook.com/clsacommunity/>
clsa.com<https://www.clsa.com/>
Insights. Liquidity. Capital.
[CLSA_RGB]<https://www.clsa.com/member>
A CITIC Securities Company
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.