You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Akash Shinde <ak...@gmail.com> on 2019/01/07 16:11:54 UTC
After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Hi,
When execute affinity.partition(key), I am getting following exception on
Ignite 2.7.
Stacktrace:
2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
isCancelled=false, retry=null]
org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
optimized marshaller
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
unmarshal object with optimized marshaller
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
... 10 common frames omitted
Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
object with given class loader:
[clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
... 16 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
... 18 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 21 common frames omitted
Caused by: java.io.IOException: Failed to deserialize field
[name=assignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 27 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
... 30 common frames omitted
Caused by: java.io.IOException: Unexpected error occurred during
unmarshalling of an instance of the class:
org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
that all nodes are running the same version of Ignite and that all nodes
have GridOptimizedMarshaller configured with identical optimized classes
lists, if any (see setClassNames and setClassNamesPath methods). If your
serialized classes implement java.io.Externalizable interface, verify that
serialization logic is correct.
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 41 common frames omitted
2019-01-07 21:23:03,140 6699925 [main-SendThread(
qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Opening socket connection to server
qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
authenticate using SASL (unknown error)
2019-01-07 21:23:03,140 6699925 [main-SendThread(
qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Socket connection established to
qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
2019-01-07 21:23:03,140 6699925 [main-SendThread(
qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn -
Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
expired
2019-01-07 21:23:03,140 6699925 [main-SendThread(
qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
expired, closing socket connection
2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
org.apache.zookeeper.ZooKeeper - Initiating client connection,
connectString=qzkpr01.p13.eng.in03.com:50024,qzkpr02.p13.eng.in03.com:50024,
qzkpr03.p13.eng.in03.com:50024/qkafka sessionTimeout=10000
watcher=org.I0Itec.zkclient.ZkClient@3003d288
2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
0x3673a59f5dd01c7
2019-01-07 21:23:03,296 6700081 [main-SendThread(
qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Opening socket connection to server
qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
authenticate using SASL (unknown error)
2019-01-07 21:23:03,296 6700081 [main-SendThread(
qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Socket connection established to
qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
2019-01-07 21:23:03,296 6700081 [main-SendThread(
qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
Session establishment complete on server
qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
0x3673a59f5dd01ce, negotiated timeout = 10000
2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
isCancelled=false, retry=null]
org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
optimized marshaller
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
unmarshal object with optimized marshaller
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
... 10 common frames omitted
Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
object with given class loader:
[clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
... 16 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
... 18 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 21 common frames omitted
Caused by: java.io.IOException: Failed to deserialize field
[name=assignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 27 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
... 30 common frames omitted
Caused by: java.io.IOException: Unexpected error occurred during
unmarshalling of an instance of the class:
org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
that all nodes are running the same version of Ignite and that all nodes
have GridOptimizedMarshaller configured with identical optimized classes
lists, if any (see setClassNames and setClassNamesPath methods). If your
serialized classes implement java.io.Externalizable interface, verify that
serialization logic is correct.
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 41 common frames omitted
2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
isCancelled=false, retry=null]
org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
optimized marshaller
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
unmarshal object with optimized marshaller
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
... 10 common frames omitted
Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
object with given class loader:
[clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
... 16 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.util.lang.GridTuple3]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
... 18 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 21 common frames omitted
Caused by: java.io.IOException: Failed to deserialize field
[name=assignment]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 27 common frames omitted
Caused by: java.io.IOException: Failed to deserialize object
[typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
... 30 common frames omitted
Caused by: java.io.IOException: Unexpected error occurred during
unmarshalling of an instance of the class:
org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
that all nodes are running the same version of Ignite and that all nodes
have GridOptimizedMarshaller configured with identical optimized classes
lists, if any (see setClassNames and setClassNamesPath methods). If your
serialized classes implement java.io.Externalizable interface, verify that
serialization logic is correct.
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
at
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
... 41 common frames omitted
Thanks,
Akash
Re: [IGNITE-10925] After upgrading 2.7 getting Unexpected error
occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Hi,
I am able to create a reproducer for this issue. I have also created a JIRA
IGNITE-10925 <https://issues.apache.org/jira/browse/IGNITE-10925> for this
issue.
Reproducer: https://github.com/prasadbhalerao1983/IgniteIssueReproducer.git
Step to Reproduce:
1) First Run com.example.demo.Server class as a java program
2) Then run com.example.demo.Client as java program.
Thanks,
Prasad
On Sat, Jan 12, 2019 at 11:17 AM Prasad Bhalerao <
prasadbhalerao1983@gmail.com> wrote:
> No I am not using zookeeper discovery.
> Using TcpDiscoveryVmIpFinder.
>
> Can someone please explain on what event cacheMetrics in TcpDiscoveryNode
> gets populated. It is not getting populated in standalone program.
>
> If it gets populated then I might be able to reproduce this case.
>
> On Fri 11 Jan, 2019, 8:28 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> wrote:
>
>> Hello!
>>
>> Have you tried enabling Zookeeper in your reproducer? I have a hunch that
>> they are linked: this behavior is affected by zookeeper discovery.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 11 янв. 2019 г. в 17:44, Prasad Bhalerao <
>> prasadbhalerao1983@gmail.com>:
>>
>>> I tried to reproduce this in standalone program. But the thing is cache
>>> metrics map in TcpDiscoveryNode is empty even after setting
>>> statisticEnabled to true on all caches.
>>> So the flow does not enter into serializr/deserialize cacheMetrics block.
>>>
>>> Any idea how the cacheMetrics gets populated. On which event?
>>>
>>>
>>> Thanks,
>>> Prasad
>>>
>>> On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
>>> wrote:
>>>
>>>> Hello!
>>>>
>>>> I think the problem was introduced by
>>>> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
>>>> suspicious, however it is strange that it does not reproduce right away.
>>>>
>>>> I could try and devise a fix but I could not reproduce this behavior in
>>>> any
>>>> of tests. If you could do a reproducer project that would be awesome.
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
Re: [IGNITE-10925] After upgrading 2.7 getting Unexpected error
occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Hi,
I am able to create a reproducer for this issue. I have also created a JIRA
IGNITE-10925 <https://issues.apache.org/jira/browse/IGNITE-10925> for this
issue.
Reproducer: https://github.com/prasadbhalerao1983/IgniteIssueReproducer.git
Step to Reproduce:
1) First Run com.example.demo.Server class as a java program
2) Then run com.example.demo.Client as java program.
Thanks,
Prasad
On Sat, Jan 12, 2019 at 11:17 AM Prasad Bhalerao <
prasadbhalerao1983@gmail.com> wrote:
> No I am not using zookeeper discovery.
> Using TcpDiscoveryVmIpFinder.
>
> Can someone please explain on what event cacheMetrics in TcpDiscoveryNode
> gets populated. It is not getting populated in standalone program.
>
> If it gets populated then I might be able to reproduce this case.
>
> On Fri 11 Jan, 2019, 8:28 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> wrote:
>
>> Hello!
>>
>> Have you tried enabling Zookeeper in your reproducer? I have a hunch that
>> they are linked: this behavior is affected by zookeeper discovery.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 11 янв. 2019 г. в 17:44, Prasad Bhalerao <
>> prasadbhalerao1983@gmail.com>:
>>
>>> I tried to reproduce this in standalone program. But the thing is cache
>>> metrics map in TcpDiscoveryNode is empty even after setting
>>> statisticEnabled to true on all caches.
>>> So the flow does not enter into serializr/deserialize cacheMetrics block.
>>>
>>> Any idea how the cacheMetrics gets populated. On which event?
>>>
>>>
>>> Thanks,
>>> Prasad
>>>
>>> On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
>>> wrote:
>>>
>>>> Hello!
>>>>
>>>> I think the problem was introduced by
>>>> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
>>>> suspicious, however it is strange that it does not reproduce right away.
>>>>
>>>> I could try and devise a fix but I could not reproduce this behavior in
>>>> any
>>>> of tests. If you could do a reproducer project that would be awesome.
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
No I am not using zookeeper discovery.
Using TcpDiscoveryVmIpFinder.
Can someone please explain on what event cacheMetrics in TcpDiscoveryNode
gets populated. It is not getting populated in standalone program.
If it gets populated then I might be able to reproduce this case.
On Fri 11 Jan, 2019, 8:28 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
wrote:
> Hello!
>
> Have you tried enabling Zookeeper in your reproducer? I have a hunch that
> they are linked: this behavior is affected by zookeeper discovery.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 11 янв. 2019 г. в 17:44, Prasad Bhalerao <prasadbhalerao1983@gmail.com
> >:
>
>> I tried to reproduce this in standalone program. But the thing is cache
>> metrics map in TcpDiscoveryNode is empty even after setting
>> statisticEnabled to true on all caches.
>> So the flow does not enter into serializr/deserialize cacheMetrics block.
>>
>> Any idea how the cacheMetrics gets populated. On which event?
>>
>>
>> Thanks,
>> Prasad
>>
>> On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
>> wrote:
>>
>>> Hello!
>>>
>>> I think the problem was introduced by
>>> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
>>> suspicious, however it is strange that it does not reproduce right away.
>>>
>>> I could try and devise a fix but I could not reproduce this behavior in
>>> any
>>> of tests. If you could do a reproducer project that would be awesome.
>>>
>>> Regards,
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
No I am not using zookeeper discovery.
Using TcpDiscoveryVmIpFinder.
Can someone please explain on what event cacheMetrics in TcpDiscoveryNode
gets populated. It is not getting populated in standalone program.
If it gets populated then I might be able to reproduce this case.
On Fri 11 Jan, 2019, 8:28 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
wrote:
> Hello!
>
> Have you tried enabling Zookeeper in your reproducer? I have a hunch that
> they are linked: this behavior is affected by zookeeper discovery.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 11 янв. 2019 г. в 17:44, Prasad Bhalerao <prasadbhalerao1983@gmail.com
> >:
>
>> I tried to reproduce this in standalone program. But the thing is cache
>> metrics map in TcpDiscoveryNode is empty even after setting
>> statisticEnabled to true on all caches.
>> So the flow does not enter into serializr/deserialize cacheMetrics block.
>>
>> Any idea how the cacheMetrics gets populated. On which event?
>>
>>
>> Thanks,
>> Prasad
>>
>> On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
>> wrote:
>>
>>> Hello!
>>>
>>> I think the problem was introduced by
>>> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
>>> suspicious, however it is strange that it does not reproduce right away.
>>>
>>> I could try and devise a fix but I could not reproduce this behavior in
>>> any
>>> of tests. If you could do a reproducer project that would be awesome.
>>>
>>> Regards,
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Have you tried enabling Zookeeper in your reproducer? I have a hunch that
they are linked: this behavior is affected by zookeeper discovery.
Regards,
--
Ilya Kasnacheev
пт, 11 янв. 2019 г. в 17:44, Prasad Bhalerao <pr...@gmail.com>:
> I tried to reproduce this in standalone program. But the thing is cache
> metrics map in TcpDiscoveryNode is empty even after setting
> statisticEnabled to true on all caches.
> So the flow does not enter into serializr/deserialize cacheMetrics block.
>
> Any idea how the cacheMetrics gets populated. On which event?
>
>
> Thanks,
> Prasad
>
> On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
> wrote:
>
>> Hello!
>>
>> I think the problem was introduced by
>> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
>> suspicious, however it is strange that it does not reproduce right away.
>>
>> I could try and devise a fix but I could not reproduce this behavior in
>> any
>> of tests. If you could do a reproducer project that would be awesome.
>>
>> Regards,
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
I tried to reproduce this in standalone program. But the thing is cache
metrics map in TcpDiscoveryNode is empty even after setting
statisticEnabled to true on all caches.
So the flow does not enter into serializr/deserialize cacheMetrics block.
Any idea how the cacheMetrics gets populated. On which event?
Thanks,
Prasad
On Fri 11 Jan, 2019, 7:55 PM ilya.kasnacheev <ilya.kasnacheev@gmail.com
wrote:
> Hello!
>
> I think the problem was introduced by
> https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
> suspicious, however it is strange that it does not reproduce right away.
>
> I could try and devise a fix but I could not reproduce this behavior in any
> of tests. If you could do a reproducer project that would be awesome.
>
> Regards,
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: After upgrading 2.7 getting Unexpected error occurred during
unmarshalling
Posted by "ilya.kasnacheev" <il...@gmail.com>.
Hello!
I think the problem was introduced by
https://issues.apache.org/jira/browse/IGNITE-6846 which does look very
suspicious, however it is strange that it does not reproduce right away.
I could try and devise a fix but I could not reproduce this behavior in any
of tests. If you could do a reproducer project that would be awesome.
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
> Resending....
>
>
> I am not able reproduce this issue in small reproducer project but this is
> consistently happening in my project. So I debugged this issue and attached
> the screenshot in this mail.
>
>
>
> *NOTE: *This issue occurs if the statistics are enabled on cache
> configuration level [cacheCfg.setStatisticsEnabled(true)].
>
>
>
> As shown in screenshots there 22 cache metrics in cacheMetrics hashmap.
> All these cache metrics get serialized successfully. But at the time of
> deserialization on client node only first metrics get de-serialized
> successfully but all other metrics till iteration count 13 are
> de-serialized as null value and on iteration 14 “ref” byte value in
> “OptimizedObjectInputStream.readObject0()” method is read as 81 and code
> throws exception.
>
>
>
> I think this where it is going wrong. The object copy at the time of
> serialization and de-serialization should be same but that’s not happening
> in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
> 22 cacheMetrics are being de-serialized successfully.
>
>
>
> *AffinityJob result being serialized on server:*
>
>
>
>
>
> *[image: server.png][image: server.png]*
>
> *AffinityJob result being de-serialized on client:*
>
> [image: client.jpg]
>
>
> Thanks,
> Prasad
>
> On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Do you have a reproducer project to reliably confirm this issue?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>>
>>> Added dev@ignite.apache.org.
>>>
>>> Should I log Jira for this issue?
>>>
>>> Thanks,
>>> Akash
>>>
>>>
>>>
>>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > No both nodes, client and server are running on Ignite 2.7 version. I
>>> am
>>> > starting both server and client from Intellij IDE.
>>> >
>>> > Version printed in Server node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Version in client node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Thanks,
>>> > Akash
>>> >
>>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>>> wrote:
>>> >
>>> >> Hi!
>>> >>
>>> >> Any chance you might have one node running 2.6 or something like that
>>> ?
>>> >>
>>> >> It looks like it get a different object that does not match the one
>>> >> expected in 2.7
>>> >>
>>> >> Mikael
>>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>> >>
>>> >> Before submitting the affinity task ignite first gets the affinity
>>> cached
>>> >> function (AffinityInfo) by submitting the cluster wide task
>>> "AffinityJob".
>>> >> But while in the process of retrieving the output of this AffinityJob,
>>> >> ignite deserializes this output. I am getting exception while
>>> deserailizing
>>> >> this output.
>>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>>> >> CacheMetrics object from input stream on 14th iteration I am getting
>>> >> following exception. Complete stack trace is given in this mail chain.
>>> >>
>>> >> Caused by: java.io.IOException: Unexpected error occurred during
>>> >> unmarshalling of an instance of the class:
>>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> >>
>>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>> >>
>>> >> Is this a bug or am I doing something wrong?
>>> >>
>>> >> Can someone please help?
>>> >>
>>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> When execute affinity.partition(key), I am getting following
>>> exception
>>> >>> on Ignite 2.7.
>>> >>>
>>> >>> Stacktrace:
>>> >>>
>>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired, closing socket connection
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> >>> 0x3673a59f5dd01c7
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Session establishment complete on server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed
>>> (SyncConnected)
>>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>>
>>> >>> Thanks,
>>> >>> Akash
>>> >>>
>>> >>
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
> Resending....
>
>
> I am not able reproduce this issue in small reproducer project but this is
> consistently happening in my project. So I debugged this issue and attached
> the screenshot in this mail.
>
>
>
> *NOTE: *This issue occurs if the statistics are enabled on cache
> configuration level [cacheCfg.setStatisticsEnabled(true)].
>
>
>
> As shown in screenshots there 22 cache metrics in cacheMetrics hashmap.
> All these cache metrics get serialized successfully. But at the time of
> deserialization on client node only first metrics get de-serialized
> successfully but all other metrics till iteration count 13 are
> de-serialized as null value and on iteration 14 “ref” byte value in
> “OptimizedObjectInputStream.readObject0()” method is read as 81 and code
> throws exception.
>
>
>
> I think this where it is going wrong. The object copy at the time of
> serialization and de-serialization should be same but that’s not happening
> in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
> 22 cacheMetrics are being de-serialized successfully.
>
>
>
> *AffinityJob result being serialized on server:*
>
>
>
>
>
> *[image: server.png][image: server.png]*
>
> *AffinityJob result being de-serialized on client:*
>
> [image: client.jpg]
>
>
> Thanks,
> Prasad
>
> On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Do you have a reproducer project to reliably confirm this issue?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>>
>>> Added dev@ignite.apache.org.
>>>
>>> Should I log Jira for this issue?
>>>
>>> Thanks,
>>> Akash
>>>
>>>
>>>
>>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > No both nodes, client and server are running on Ignite 2.7 version. I
>>> am
>>> > starting both server and client from Intellij IDE.
>>> >
>>> > Version printed in Server node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Version in client node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Thanks,
>>> > Akash
>>> >
>>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>>> wrote:
>>> >
>>> >> Hi!
>>> >>
>>> >> Any chance you might have one node running 2.6 or something like that
>>> ?
>>> >>
>>> >> It looks like it get a different object that does not match the one
>>> >> expected in 2.7
>>> >>
>>> >> Mikael
>>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>> >>
>>> >> Before submitting the affinity task ignite first gets the affinity
>>> cached
>>> >> function (AffinityInfo) by submitting the cluster wide task
>>> "AffinityJob".
>>> >> But while in the process of retrieving the output of this AffinityJob,
>>> >> ignite deserializes this output. I am getting exception while
>>> deserailizing
>>> >> this output.
>>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>>> >> CacheMetrics object from input stream on 14th iteration I am getting
>>> >> following exception. Complete stack trace is given in this mail chain.
>>> >>
>>> >> Caused by: java.io.IOException: Unexpected error occurred during
>>> >> unmarshalling of an instance of the class:
>>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> >>
>>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>> >>
>>> >> Is this a bug or am I doing something wrong?
>>> >>
>>> >> Can someone please help?
>>> >>
>>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> When execute affinity.partition(key), I am getting following
>>> exception
>>> >>> on Ignite 2.7.
>>> >>>
>>> >>> Stacktrace:
>>> >>>
>>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired, closing socket connection
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> >>> 0x3673a59f5dd01c7
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Session establishment complete on server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed
>>> (SyncConnected)
>>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>>
>>> >>> Thanks,
>>> >>> Akash
>>> >>>
>>> >>
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Resending....
I am not able reproduce this issue in small reproducer project but this is
consistently happening in my project. So I debugged this issue and attached
the screenshot in this mail.
*NOTE: *This issue occurs if the statistics are enabled on cache
configuration level [cacheCfg.setStatisticsEnabled(true)].
As shown in screenshots there 22 cache metrics in cacheMetrics hashmap. All
these cache metrics get serialized successfully. But at the time of
deserialization on client node only first metrics get de-serialized
successfully but all other metrics till iteration count 13 are
de-serialized as null value and on iteration 14 “ref” byte value in
“OptimizedObjectInputStream.readObject0()” method is read as 81 and code
throws exception.
I think this where it is going wrong. The object copy at the time of
serialization and de-serialization should be same but that’s not happening
in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
22 cacheMetrics are being de-serialized successfully.
*AffinityJob result being serialized on server:*
*[image: server.png][image: server.png]*
*AffinityJob result being de-serialized on client:*
[image: client.jpg]
Thanks,
Prasad
On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Do you have a reproducer project to reliably confirm this issue?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>
>> Added dev@ignite.apache.org.
>>
>> Should I log Jira for this issue?
>>
>> Thanks,
>> Akash
>>
>>
>>
>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > No both nodes, client and server are running on Ignite 2.7 version. I am
>> > starting both server and client from Intellij IDE.
>> >
>> > Version printed in Server node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Version in client node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Thanks,
>> > Akash
>> >
>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>> wrote:
>> >
>> >> Hi!
>> >>
>> >> Any chance you might have one node running 2.6 or something like that ?
>> >>
>> >> It looks like it get a different object that does not match the one
>> >> expected in 2.7
>> >>
>> >> Mikael
>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>> >>
>> >> Before submitting the affinity task ignite first gets the affinity
>> cached
>> >> function (AffinityInfo) by submitting the cluster wide task
>> "AffinityJob".
>> >> But while in the process of retrieving the output of this AffinityJob,
>> >> ignite deserializes this output. I am getting exception while
>> deserailizing
>> >> this output.
>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> >> CacheMetrics object from input stream on 14th iteration I am getting
>> >> following exception. Complete stack trace is given in this mail chain.
>> >>
>> >> Caused by: java.io.IOException: Unexpected error occurred during
>> >> unmarshalling of an instance of the class:
>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> >>
>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>> >>
>> >> Is this a bug or am I doing something wrong?
>> >>
>> >> Can someone please help?
>> >>
>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> When execute affinity.partition(key), I am getting following exception
>> >>> on Ignite 2.7.
>> >>>
>> >>> Stacktrace:
>> >>>
>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired, closing socket connection
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>> >>> 0x3673a59f5dd01c7
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Session establishment complete on server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>>
>> >>> Thanks,
>> >>> Akash
>> >>>
>> >>
>>
>
Re: After upgrading 2.7 getting error during unmarshalling (Works
fine on 2.6)
Posted by Prasad Bhalerao <pr...@gmail.com>.
Hi Ilya,
I am not able reproduce this issue in small reproducer project but this is
consistently happening in my project. So I debugged this issue and attached
the screenshot in this mail.
*NOTE: *This issue occurs if the statistics are enabled on cache
configuration level [cacheCfg.setStatisticsEnabled(true)].
As shown in screenshots there 22 cache metrics in cacheMetrics hashmap. All
these cache metrics gets serialized successfully on server node. But at the
time of deserialization on client node only first metrics get de-serialized
successfully but all other metrics till iteration count 13 are
de-serialized as null value and on iteration 14 “ref” byte value in
“OptimizedObjectInputStream.readObject0()” method is read as 81 and code
throws exception.
I think this where it is going wrong. The object copy at the time of
serialization and de-serialization should be same but that’s not happening
in Ignite 2.7 version.
So I debugged this on Ignite 2.6 version. On 2.6 version all 22
cacheMetrics are being de-serialized successfully.
This Looks like a bug to me in serializer/deseriliazer code.
*AffinityJob result being serialized on server:*
*AffinityJob result being de-serialized on client:*
Thanks,
Prasad
On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Do you have a reproducer project to reliably confirm this issue?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>
>> Added dev@ignite.apache.org.
>>
>> Should I log Jira for this issue?
>>
>> Thanks,
>> Akash
>>
>>
>>
>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > No both nodes, client and server are running on Ignite 2.7 version. I am
>> > starting both server and client from Intellij IDE.
>> >
>> > Version printed in Server node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Version in client node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Thanks,
>> > Akash
>> >
>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>> wrote:
>> >
>> >> Hi!
>> >>
>> >> Any chance you might have one node running 2.6 or something like that ?
>> >>
>> >> It looks like it get a different object that does not match the one
>> >> expected in 2.7
>> >>
>> >> Mikael
>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>> >>
>> >> Before submitting the affinity task ignite first gets the affinity
>> cached
>> >> function (AffinityInfo) by submitting the cluster wide task
>> "AffinityJob".
>> >> But while in the process of retrieving the output of this AffinityJob,
>> >> ignite deserializes this output. I am getting exception while
>> deserailizing
>> >> this output.
>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> >> CacheMetrics object from input stream on 14th iteration I am getting
>> >> following exception. Complete stack trace is given in this mail chain.
>> >>
>> >> Caused by: java.io.IOException: Unexpected error occurred during
>> >> unmarshalling of an instance of the class:
>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> >>
>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>> >>
>> >> Is this a bug or am I doing something wrong?
>> >>
>> >> Can someone please help?
>> >>
>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> When execute affinity.partition(key), I am getting following exception
>> >>> on Ignite 2.7.
>> >>>
>> >>> Stacktrace:
>> >>>
>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired, closing socket connection
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>> >>> 0x3673a59f5dd01c7
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Session establishment complete on server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>>
>> >>> Thanks,
>> >>> Akash
>> >>>
>> >>
>>
>
Re: After upgrading 2.7 getting error during unmarshalling (Works
fine on 2.6)
Posted by Prasad Bhalerao <pr...@gmail.com>.
Hi Ilya,
I am not able reproduce this issue in small reproducer project but this is
consistently happening in my project. So I debugged this issue and attached
the screenshot in this mail.
*NOTE: *This issue occurs if the statistics are enabled on cache
configuration level [cacheCfg.setStatisticsEnabled(true)].
As shown in screenshots there 22 cache metrics in cacheMetrics hashmap. All
these cache metrics gets serialized successfully on server node. But at the
time of deserialization on client node only first metrics get de-serialized
successfully but all other metrics till iteration count 13 are
de-serialized as null value and on iteration 14 “ref” byte value in
“OptimizedObjectInputStream.readObject0()” method is read as 81 and code
throws exception.
I think this where it is going wrong. The object copy at the time of
serialization and de-serialization should be same but that’s not happening
in Ignite 2.7 version.
So I debugged this on Ignite 2.6 version. On 2.6 version all 22
cacheMetrics are being de-serialized successfully.
This Looks like a bug to me in serializer/deseriliazer code.
*AffinityJob result being serialized on server:*
*AffinityJob result being de-serialized on client:*
Thanks,
Prasad
On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Do you have a reproducer project to reliably confirm this issue?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>
>> Added dev@ignite.apache.org.
>>
>> Should I log Jira for this issue?
>>
>> Thanks,
>> Akash
>>
>>
>>
>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > No both nodes, client and server are running on Ignite 2.7 version. I am
>> > starting both server and client from Intellij IDE.
>> >
>> > Version printed in Server node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Version in client node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Thanks,
>> > Akash
>> >
>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>> wrote:
>> >
>> >> Hi!
>> >>
>> >> Any chance you might have one node running 2.6 or something like that ?
>> >>
>> >> It looks like it get a different object that does not match the one
>> >> expected in 2.7
>> >>
>> >> Mikael
>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>> >>
>> >> Before submitting the affinity task ignite first gets the affinity
>> cached
>> >> function (AffinityInfo) by submitting the cluster wide task
>> "AffinityJob".
>> >> But while in the process of retrieving the output of this AffinityJob,
>> >> ignite deserializes this output. I am getting exception while
>> deserailizing
>> >> this output.
>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> >> CacheMetrics object from input stream on 14th iteration I am getting
>> >> following exception. Complete stack trace is given in this mail chain.
>> >>
>> >> Caused by: java.io.IOException: Unexpected error occurred during
>> >> unmarshalling of an instance of the class:
>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> >>
>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>> >>
>> >> Is this a bug or am I doing something wrong?
>> >>
>> >> Can someone please help?
>> >>
>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> When execute affinity.partition(key), I am getting following exception
>> >>> on Ignite 2.7.
>> >>>
>> >>> Stacktrace:
>> >>>
>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired, closing socket connection
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>> >>> 0x3673a59f5dd01c7
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Session establishment complete on server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>>
>> >>> Thanks,
>> >>> Akash
>> >>>
>> >>
>>
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Resending....
I am not able reproduce this issue in small reproducer project but this is
consistently happening in my project. So I debugged this issue and attached
the screenshot in this mail.
*NOTE: *This issue occurs if the statistics are enabled on cache
configuration level [cacheCfg.setStatisticsEnabled(true)].
As shown in screenshots there 22 cache metrics in cacheMetrics hashmap. All
these cache metrics get serialized successfully. But at the time of
deserialization on client node only first metrics get de-serialized
successfully but all other metrics till iteration count 13 are
de-serialized as null value and on iteration 14 “ref” byte value in
“OptimizedObjectInputStream.readObject0()” method is read as 81 and code
throws exception.
I think this where it is going wrong. The object copy at the time of
serialization and de-serialization should be same but that’s not happening
in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
22 cacheMetrics are being de-serialized successfully.
*AffinityJob result being serialized on server:*
*[image: server.png][image: server.png]*
*AffinityJob result being de-serialized on client:*
[image: client.jpg]
Thanks,
Prasad
On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Do you have a reproducer project to reliably confirm this issue?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>
>> Added dev@ignite.apache.org.
>>
>> Should I log Jira for this issue?
>>
>> Thanks,
>> Akash
>>
>>
>>
>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > No both nodes, client and server are running on Ignite 2.7 version. I am
>> > starting both server and client from Intellij IDE.
>> >
>> > Version printed in Server node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Version in client node log:
>> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>> >
>> > Thanks,
>> > Akash
>> >
>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>> wrote:
>> >
>> >> Hi!
>> >>
>> >> Any chance you might have one node running 2.6 or something like that ?
>> >>
>> >> It looks like it get a different object that does not match the one
>> >> expected in 2.7
>> >>
>> >> Mikael
>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>> >>
>> >> Before submitting the affinity task ignite first gets the affinity
>> cached
>> >> function (AffinityInfo) by submitting the cluster wide task
>> "AffinityJob".
>> >> But while in the process of retrieving the output of this AffinityJob,
>> >> ignite deserializes this output. I am getting exception while
>> deserailizing
>> >> this output.
>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> >> CacheMetrics object from input stream on 14th iteration I am getting
>> >> following exception. Complete stack trace is given in this mail chain.
>> >>
>> >> Caused by: java.io.IOException: Unexpected error occurred during
>> >> unmarshalling of an instance of the class:
>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> >>
>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>> >>
>> >> Is this a bug or am I doing something wrong?
>> >>
>> >> Can someone please help?
>> >>
>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> When execute affinity.partition(key), I am getting following exception
>> >>> on Ignite 2.7.
>> >>>
>> >>> Stacktrace:
>> >>>
>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired
>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
>> has
>> >>> expired, closing socket connection
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>> >>> 0x3673a59f5dd01c7
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Opening socket connection to server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>> >>> authenticate using SASL (unknown error)
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Socket connection established to
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>> org.apache.zookeeper.ClientCnxn
>> >>> - Session establishment complete on server
>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> >>> isCancelled=false, retry=null]
>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>> >>> with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> >>> at
>> >>>
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> >>> at
>> >>>
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> >>> at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> >>> at java.lang.Thread.run(Thread.java:748)
>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> >>> unmarshal object with optimized marshaller
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> >>> ... 10 common frames omitted
>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> >>> deserialize object with given class loader:
>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> >>> deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> >>> at
>> >>>
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> >>> at
>> >>>
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> >>> ... 16 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> >>> ... 18 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 21 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize field
>> >>> [name=assignment]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 27 common frames omitted
>> >>> Caused by: java.io.IOException: Failed to deserialize object
>> >>>
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> >>> ... 30 common frames omitted
>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>> >>> unmarshalling of an instance of the class:
>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>> Check
>> >>> that all nodes are running the same version of Ignite and that all
>> nodes
>> >>> have GridOptimizedMarshaller configured with identical optimized
>> classes
>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>> your
>> >>> serialized classes implement java.io.Externalizable interface, verify
>> that
>> >>> serialization logic is correct.
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> >>> at
>> >>>
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> >>> at
>> >>>
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> >>> ... 41 common frames omitted
>> >>>
>> >>> Thanks,
>> >>> Akash
>> >>>
>> >>
>>
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Resending with screenshots.
> I am not able reproduce this issue in small reproducer project but this is
> consistently happening in my project. So I debugged this issue and attached
> the screenshot in this mail.
>
>
>
> *NOTE: *This issue occurs if the statistics are enabled on cache
> configuration level [cacheCfg.setStatisticsEnabled(true)].
>
>
>
> As shown in screenshots there 22 cache metrics in cacheMetrics hashmap.
> All these cache metrics get serialized successfully. But at the time of
> deserialization on client node only first metrics get de-serialized
> successfully but all other metrics till iteration count 13 are
> de-serialized as null value and on iteration 14 “ref” byte value in
> “OptimizedObjectInputStream.readObject0()” method is read as 81 and code
> throws exception.
>
>
>
> I think this where it is going wrong. The object copy at the time of
> serialization and de-serialization should be same but that’s not happening
> in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
> 22 cacheMetrics are being de-serialized successfully.
>
>
>
> *AffinityJob result being serialized on server:*
>
>
>
> [image: server.png]
>
>
>
>
>
> *AffinityJob result being de-serialized on client:*
>
>
>
> [image: client.jpg]
>
> Thanks,
> Prasad
>
>
> On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Do you have a reproducer project to reliably confirm this issue?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>>
>>> Added dev@ignite.apache.org.
>>>
>>> Should I log Jira for this issue?
>>>
>>> Thanks,
>>> Akash
>>>
>>>
>>>
>>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > No both nodes, client and server are running on Ignite 2.7 version. I
>>> am
>>> > starting both server and client from Intellij IDE.
>>> >
>>> > Version printed in Server node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Version in client node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Thanks,
>>> > Akash
>>> >
>>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>>> wrote:
>>> >
>>> >> Hi!
>>> >>
>>> >> Any chance you might have one node running 2.6 or something like that
>>> ?
>>> >>
>>> >> It looks like it get a different object that does not match the one
>>> >> expected in 2.7
>>> >>
>>> >> Mikael
>>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>> >>
>>> >> Before submitting the affinity task ignite first gets the affinity
>>> cached
>>> >> function (AffinityInfo) by submitting the cluster wide task
>>> "AffinityJob".
>>> >> But while in the process of retrieving the output of this AffinityJob,
>>> >> ignite deserializes this output. I am getting exception while
>>> deserailizing
>>> >> this output.
>>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>>> >> CacheMetrics object from input stream on 14th iteration I am getting
>>> >> following exception. Complete stack trace is given in this mail chain.
>>> >>
>>> >> Caused by: java.io.IOException: Unexpected error occurred during
>>> >> unmarshalling of an instance of the class:
>>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> >>
>>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>> >>
>>> >> Is this a bug or am I doing something wrong?
>>> >>
>>> >> Can someone please help?
>>> >>
>>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> When execute affinity.partition(key), I am getting following
>>> exception
>>> >>> on Ignite 2.7.
>>> >>>
>>> >>> Stacktrace:
>>> >>>
>>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired, closing socket connection
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> >>> 0x3673a59f5dd01c7
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Session establishment complete on server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed
>>> (SyncConnected)
>>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>>
>>> >>> Thanks,
>>> >>> Akash
>>> >>>
>>> >>
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Prasad Bhalerao <pr...@gmail.com>.
Resending with screenshots.
> I am not able reproduce this issue in small reproducer project but this is
> consistently happening in my project. So I debugged this issue and attached
> the screenshot in this mail.
>
>
>
> *NOTE: *This issue occurs if the statistics are enabled on cache
> configuration level [cacheCfg.setStatisticsEnabled(true)].
>
>
>
> As shown in screenshots there 22 cache metrics in cacheMetrics hashmap.
> All these cache metrics get serialized successfully. But at the time of
> deserialization on client node only first metrics get de-serialized
> successfully but all other metrics till iteration count 13 are
> de-serialized as null value and on iteration 14 “ref” byte value in
> “OptimizedObjectInputStream.readObject0()” method is read as 81 and code
> throws exception.
>
>
>
> I think this where it is going wrong. The object copy at the time of
> serialization and de-serialization should be same but that’s not happening
> in Ignite 2.7 version. So I debugged this on Ignite 2.6 version. On 2.6 all
> 22 cacheMetrics are being de-serialized successfully.
>
>
>
> *AffinityJob result being serialized on server:*
>
>
>
> [image: server.png]
>
>
>
>
>
> *AffinityJob result being de-serialized on client:*
>
>
>
> [image: client.jpg]
>
> Thanks,
> Prasad
>
>
> On Wed, Jan 9, 2019 at 6:41 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Do you have a reproducer project to reliably confirm this issue?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
>>
>>> Added dev@ignite.apache.org.
>>>
>>> Should I log Jira for this issue?
>>>
>>> Thanks,
>>> Akash
>>>
>>>
>>>
>>> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > No both nodes, client and server are running on Ignite 2.7 version. I
>>> am
>>> > starting both server and client from Intellij IDE.
>>> >
>>> > Version printed in Server node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Version in client node log:
>>> > Ignite ver.
>>> 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>>> >
>>> > Thanks,
>>> > Akash
>>> >
>>> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com>
>>> wrote:
>>> >
>>> >> Hi!
>>> >>
>>> >> Any chance you might have one node running 2.6 or something like that
>>> ?
>>> >>
>>> >> It looks like it get a different object that does not match the one
>>> >> expected in 2.7
>>> >>
>>> >> Mikael
>>> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>> >>
>>> >> Before submitting the affinity task ignite first gets the affinity
>>> cached
>>> >> function (AffinityInfo) by submitting the cluster wide task
>>> "AffinityJob".
>>> >> But while in the process of retrieving the output of this AffinityJob,
>>> >> ignite deserializes this output. I am getting exception while
>>> deserailizing
>>> >> this output.
>>> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
>>> >> CacheMetrics object from input stream on 14th iteration I am getting
>>> >> following exception. Complete stack trace is given in this mail chain.
>>> >>
>>> >> Caused by: java.io.IOException: Unexpected error occurred during
>>> >> unmarshalling of an instance of the class:
>>> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> >>
>>> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>> >>
>>> >> Is this a bug or am I doing something wrong?
>>> >>
>>> >> Can someone please help?
>>> >>
>>> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> When execute affinity.partition(key), I am getting following
>>> exception
>>> >>> on Ignite 2.7.
>>> >>>
>>> >>> Stacktrace:
>>> >>>
>>> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] WARN
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> >>> qzkpr01.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Unable to reconnect to ZooKeeper service, session
>>> 0x3673a59f5dd01c7 has
>>> >>> expired, closing socket connection
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> >>> 0x3673a59f5dd01c7
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Opening socket connection to server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> >>> authenticate using SASL (unknown error)
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Socket connection established to
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> >>> qzkpr03.p13.eng.in03.com:50024)] INFO
>>> org.apache.zookeeper.ClientCnxn
>>> >>> - Session establishment complete on server
>>> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed
>>> (SyncConnected)
>>> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> >>> isCancelled=false, retry=null]
>>> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> >>> with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> >>> at
>>> >>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> >>> at java.lang.Thread.run(Thread.java:748)
>>> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> >>> unmarshal object with optimized marshaller
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> >>> ... 10 common frames omitted
>>> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> >>> deserialize object with given class loader:
>>> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> >>> deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> >>> at
>>> >>>
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> >>> ... 16 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> >>> ... 18 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 21 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize field
>>> >>> [name=assignment]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 27 common frames omitted
>>> >>> Caused by: java.io.IOException: Failed to deserialize object
>>> >>>
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> >>> ... 30 common frames omitted
>>> >>> Caused by: java.io.IOException: Unexpected error occurred during
>>> >>> unmarshalling of an instance of the class:
>>> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>> Check
>>> >>> that all nodes are running the same version of Ignite and that all
>>> nodes
>>> >>> have GridOptimizedMarshaller configured with identical optimized
>>> classes
>>> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
>>> your
>>> >>> serialized classes implement java.io.Externalizable interface,
>>> verify that
>>> >>> serialization logic is correct.
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> >>> at
>>> >>>
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> >>> at
>>> >>>
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> >>> ... 41 common frames omitted
>>> >>>
>>> >>> Thanks,
>>> >>> Akash
>>> >>>
>>> >>
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Do you have a reproducer project to reliably confirm this issue?
Regards,
--
Ilya Kasnacheev
ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
> Added dev@ignite.apache.org.
>
> Should I log Jira for this issue?
>
> Thanks,
> Akash
>
>
>
> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com> wrote:
>
> > Hi,
> >
> > No both nodes, client and server are running on Ignite 2.7 version. I am
> > starting both server and client from Intellij IDE.
> >
> > Version printed in Server node log:
> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
> >
> > Version in client node log:
> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
> >
> > Thanks,
> > Akash
> >
> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com> wrote:
> >
> >> Hi!
> >>
> >> Any chance you might have one node running 2.6 or something like that ?
> >>
> >> It looks like it get a different object that does not match the one
> >> expected in 2.7
> >>
> >> Mikael
> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
> >>
> >> Before submitting the affinity task ignite first gets the affinity
> cached
> >> function (AffinityInfo) by submitting the cluster wide task
> "AffinityJob".
> >> But while in the process of retrieving the output of this AffinityJob,
> >> ignite deserializes this output. I am getting exception while
> deserailizing
> >> this output.
> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
> >> CacheMetrics object from input stream on 14th iteration I am getting
> >> following exception. Complete stack trace is given in this mail chain.
> >>
> >> Caused by: java.io.IOException: Unexpected error occurred during
> >> unmarshalling of an instance of the class:
> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
> >>
> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
> >>
> >> Is this a bug or am I doing something wrong?
> >>
> >> Can someone please help?
> >>
> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> When execute affinity.partition(key), I am getting following exception
> >>> on Ignite 2.7.
> >>>
> >>> Stacktrace:
> >>>
> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Opening socket connection to server
> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
> >>> authenticate using SASL (unknown error)
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Socket connection established to
> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn
> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
> has
> >>> expired
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
> has
> >>> expired, closing socket connection
> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
> >>> 0x3673a59f5dd01c7
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Opening socket connection to server
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
> >>> authenticate using SASL (unknown error)
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Socket connection established to
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Session establishment complete on server
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>>
> >>> Thanks,
> >>> Akash
> >>>
> >>
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Do you have a reproducer project to reliably confirm this issue?
Regards,
--
Ilya Kasnacheev
ср, 9 янв. 2019 г. в 12:39, Akash Shinde <ak...@gmail.com>:
> Added dev@ignite.apache.org.
>
> Should I log Jira for this issue?
>
> Thanks,
> Akash
>
>
>
> On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com> wrote:
>
> > Hi,
> >
> > No both nodes, client and server are running on Ignite 2.7 version. I am
> > starting both server and client from Intellij IDE.
> >
> > Version printed in Server node log:
> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
> >
> > Version in client node log:
> > Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
> >
> > Thanks,
> > Akash
> >
> > On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com> wrote:
> >
> >> Hi!
> >>
> >> Any chance you might have one node running 2.6 or something like that ?
> >>
> >> It looks like it get a different object that does not match the one
> >> expected in 2.7
> >>
> >> Mikael
> >> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
> >>
> >> Before submitting the affinity task ignite first gets the affinity
> cached
> >> function (AffinityInfo) by submitting the cluster wide task
> "AffinityJob".
> >> But while in the process of retrieving the output of this AffinityJob,
> >> ignite deserializes this output. I am getting exception while
> deserailizing
> >> this output.
> >> In TcpDiscoveryNode.readExternal() method while deserailizing the
> >> CacheMetrics object from input stream on 14th iteration I am getting
> >> following exception. Complete stack trace is given in this mail chain.
> >>
> >> Caused by: java.io.IOException: Unexpected error occurred during
> >> unmarshalling of an instance of the class:
> >> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
> >>
> >> This is working fine on Ignite 2.6 version but giving problem on 2.7.
> >>
> >> Is this a bug or am I doing something wrong?
> >>
> >> Can someone please help?
> >>
> >> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> When execute affinity.partition(key), I am getting following exception
> >>> on Ignite 2.7.
> >>>
> >>> Stacktrace:
> >>>
> >>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Opening socket connection to server
> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
> >>> authenticate using SASL (unknown error)
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Socket connection established to
> >>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn
> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
> has
> >>> expired
> >>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> >>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7
> has
> >>> expired, closing socket connection
> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
> >>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> >>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
> >>> connectString=qzkpr01.p13.eng.in03.com:50024,
> >>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
> >>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
> >>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
> >>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
> >>> 0x3673a59f5dd01c7
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Opening socket connection to server
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
> >>> authenticate using SASL (unknown error)
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Socket connection established to
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
> >>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> >>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
> >>> - Session establishment complete on server
> >>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
> >>> 0x3673a59f5dd01ce, negotiated timeout = 10000
> >>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
> >>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
> >>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
> >>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> >>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> >>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> >>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> >>> isCancelled=false, retry=null]
> >>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
> >>> with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> >>> at
> >>>
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> >>> at
> >>>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >>> at
> >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >>> at java.lang.Thread.run(Thread.java:748)
> >>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> >>> unmarshal object with optimized marshaller
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> >>> at
> >>>
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> >>> at
> >>>
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> >>> ... 10 common frames omitted
> >>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> >>> deserialize object with given class loader:
> >>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> >>> deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> >>> at
> >>>
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> >>> at
> >>>
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> >>> ... 16 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> >>> ... 18 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 21 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize field
> >>> [name=assignment]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 27 common frames omitted
> >>> Caused by: java.io.IOException: Failed to deserialize object
> >>>
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> >>> ... 30 common frames omitted
> >>> Caused by: java.io.IOException: Unexpected error occurred during
> >>> unmarshalling of an instance of the class:
> >>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> >>> that all nodes are running the same version of Ignite and that all
> nodes
> >>> have GridOptimizedMarshaller configured with identical optimized
> classes
> >>> lists, if any (see setClassNames and setClassNamesPath methods). If
> your
> >>> serialized classes implement java.io.Externalizable interface, verify
> that
> >>> serialization logic is correct.
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> >>> at
> >>>
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> >>> at
> >>>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> >>> ... 41 common frames omitted
> >>>
> >>> Thanks,
> >>> Akash
> >>>
> >>
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Akash Shinde <ak...@gmail.com>.
Added dev@ignite.apache.org.
Should I log Jira for this issue?
Thanks,
Akash
On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com> wrote:
> Hi,
>
> No both nodes, client and server are running on Ignite 2.7 version. I am
> starting both server and client from Intellij IDE.
>
> Version printed in Server node log:
> Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>
> Version in client node log:
> Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>
> Thanks,
> Akash
>
> On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com> wrote:
>
>> Hi!
>>
>> Any chance you might have one node running 2.6 or something like that ?
>>
>> It looks like it get a different object that does not match the one
>> expected in 2.7
>>
>> Mikael
>> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>
>> Before submitting the affinity task ignite first gets the affinity cached
>> function (AffinityInfo) by submitting the cluster wide task "AffinityJob".
>> But while in the process of retrieving the output of this AffinityJob,
>> ignite deserializes this output. I am getting exception while deserailizing
>> this output.
>> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> CacheMetrics object from input stream on 14th iteration I am getting
>> following exception. Complete stack trace is given in this mail chain.
>>
>> Caused by: java.io.IOException: Unexpected error occurred during
>> unmarshalling of an instance of the class:
>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>
>> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>
>> Is this a bug or am I doing something wrong?
>>
>> Can someone please help?
>>
>> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> When execute affinity.partition(key), I am getting following exception
>>> on Ignite 2.7.
>>>
>>> Stacktrace:
>>>
>>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Opening socket connection to server
>>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> authenticate using SASL (unknown error)
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Socket connection established to
>>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn
>>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>>> expired
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>>> expired, closing socket connection
>>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> 0x3673a59f5dd01c7
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Opening socket connection to server
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> authenticate using SASL (unknown error)
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Socket connection established to
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Session establishment complete on server
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>>
>>> Thanks,
>>> Akash
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Akash Shinde <ak...@gmail.com>.
Added dev@ignite.apache.org.
Should I log Jira for this issue?
Thanks,
Akash
On Tue, Jan 8, 2019 at 6:16 PM Akash Shinde <ak...@gmail.com> wrote:
> Hi,
>
> No both nodes, client and server are running on Ignite 2.7 version. I am
> starting both server and client from Intellij IDE.
>
> Version printed in Server node log:
> Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>
> Version in client node log:
> Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
>
> Thanks,
> Akash
>
> On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com> wrote:
>
>> Hi!
>>
>> Any chance you might have one node running 2.6 or something like that ?
>>
>> It looks like it get a different object that does not match the one
>> expected in 2.7
>>
>> Mikael
>> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>>
>> Before submitting the affinity task ignite first gets the affinity cached
>> function (AffinityInfo) by submitting the cluster wide task "AffinityJob".
>> But while in the process of retrieving the output of this AffinityJob,
>> ignite deserializes this output. I am getting exception while deserailizing
>> this output.
>> In TcpDiscoveryNode.readExternal() method while deserailizing the
>> CacheMetrics object from input stream on 14th iteration I am getting
>> following exception. Complete stack trace is given in this mail chain.
>>
>> Caused by: java.io.IOException: Unexpected error occurred during
>> unmarshalling of an instance of the class:
>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>>
>> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>>
>> Is this a bug or am I doing something wrong?
>>
>> Can someone please help?
>>
>> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> When execute affinity.partition(key), I am getting following exception
>>> on Ignite 2.7.
>>>
>>> Stacktrace:
>>>
>>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Opening socket connection to server
>>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>>> authenticate using SASL (unknown error)
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Socket connection established to
>>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn
>>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>>> expired
>>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>>> expired, closing socket connection
>>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>>> connectString=qzkpr01.p13.eng.in03.com:50024,
>>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>>> 0x3673a59f5dd01c7
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Opening socket connection to server
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>>> authenticate using SASL (unknown error)
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Socket connection established to
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn
>>> - Session establishment complete on server
>>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>>> isCancelled=false, retry=null]
>>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object
>>> with optimized marshaller
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>>> at
>>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>>> at
>>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>>> unmarshal object with optimized marshaller
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>>> at
>>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>>> at
>>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>>> at
>>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>>> ... 10 common frames omitted
>>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>>> deserialize object with given class loader:
>>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>>> deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>>> at
>>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>>> at
>>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>>> ... 16 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>>> ... 18 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 21 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize field
>>> [name=assignment]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 27 common frames omitted
>>> Caused by: java.io.IOException: Failed to deserialize object
>>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>>> ... 30 common frames omitted
>>> Caused by: java.io.IOException: Unexpected error occurred during
>>> unmarshalling of an instance of the class:
>>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>>> that all nodes are running the same version of Ignite and that all nodes
>>> have GridOptimizedMarshaller configured with identical optimized classes
>>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>>> serialized classes implement java.io.Externalizable interface, verify that
>>> serialization logic is correct.
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>>> at
>>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>>> at
>>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>>> ... 41 common frames omitted
>>>
>>> Thanks,
>>> Akash
>>>
>>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Akash Shinde <ak...@gmail.com>.
Hi,
No both nodes, client and server are running on Ignite 2.7 version. I am
starting both server and client from Intellij IDE.
Version printed in Server node log:
Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
Version in client node log:
Ignite ver. 2.7.0#20181201-sha1:256ae4012cb143b4855b598b740a6f3499ead4db
Thanks,
Akash
On Tue, Jan 8, 2019 at 5:18 PM Mikael <mi...@telia.com> wrote:
> Hi!
>
> Any chance you might have one node running 2.6 or something like that ?
>
> It looks like it get a different object that does not match the one
> expected in 2.7
>
> Mikael
> Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
>
> Before submitting the affinity task ignite first gets the affinity cached
> function (AffinityInfo) by submitting the cluster wide task "AffinityJob".
> But while in the process of retrieving the output of this AffinityJob,
> ignite deserializes this output. I am getting exception while deserailizing
> this output.
> In TcpDiscoveryNode.readExternal() method while deserailizing the
> CacheMetrics object from input stream on 14th iteration I am getting
> following exception. Complete stack trace is given in this mail chain.
>
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
>
> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>
> Is this a bug or am I doing something wrong?
>
> Can someone please help?
>
> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com> wrote:
>
>> Hi,
>>
>> When execute affinity.partition(key), I am getting following exception on
>> Ignite 2.7.
>>
>> Stacktrace:
>>
>> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> isCancelled=false, retry=null]
>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
>> optimized marshaller
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> unmarshal object with optimized marshaller
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> ... 10 common frames omitted
>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object with given class loader:
>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> ... 16 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> ... 18 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 21 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize field
>> [name=assignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 27 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> ... 30 common frames omitted
>> Caused by: java.io.IOException: Unexpected error occurred during
>> unmarshalling of an instance of the class:
>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>> that all nodes are running the same version of Ignite and that all nodes
>> have GridOptimizedMarshaller configured with identical optimized classes
>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>> serialized classes implement java.io.Externalizable interface, verify that
>> serialization logic is correct.
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 41 common frames omitted
>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Opening socket connection to server
>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
>> authenticate using SASL (unknown error)
>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Socket connection established to
>> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn -
>> Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>> expired
>> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
>> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
>> expired, closing socket connection
>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
>> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
>> org.apache.zookeeper.ZooKeeper - Initiating client connection,
>> connectString=qzkpr01.p13.eng.in03.com:50024,
>> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
>> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
>> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
>> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
>> 0x3673a59f5dd01c7
>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Opening socket connection to server
>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
>> authenticate using SASL (unknown error)
>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Socket connection established to
>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
>> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
>> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
>> Session establishment complete on server
>> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
>> 0x3673a59f5dd01ce, negotiated timeout = 10000
>> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
>> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
>> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> isCancelled=false, retry=null]
>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
>> optimized marshaller
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> unmarshal object with optimized marshaller
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> ... 10 common frames omitted
>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object with given class loader:
>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> ... 16 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> ... 18 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 21 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize field
>> [name=assignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 27 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> ... 30 common frames omitted
>> Caused by: java.io.IOException: Unexpected error occurred during
>> unmarshalling of an instance of the class:
>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>> that all nodes are running the same version of Ignite and that all nodes
>> have GridOptimizedMarshaller configured with identical optimized classes
>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>> serialized classes implement java.io.Externalizable interface, verify that
>> serialization logic is correct.
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 41 common frames omitted
>> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
>> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
>> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
>> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
>> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
>> isCancelled=false, retry=null]
>> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
>> optimized marshaller
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>> at
>> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>> at
>> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
>> unmarshal object with optimized marshaller
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>> at
>> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>> at
>> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>> at
>> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>> at
>> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
>> ... 10 common frames omitted
>> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
>> deserialize object with given class loader:
>> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
>> deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
>> at
>> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>> at
>> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
>> ... 16 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
>> ... 18 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 21 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize field
>> [name=assignment]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 27 common frames omitted
>> Caused by: java.io.IOException: Failed to deserialize object
>> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
>> ... 30 common frames omitted
>> Caused by: java.io.IOException: Unexpected error occurred during
>> unmarshalling of an instance of the class:
>> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
>> that all nodes are running the same version of Ignite and that all nodes
>> have GridOptimizedMarshaller configured with identical optimized classes
>> lists, if any (see setClassNames and setClassNamesPath methods). If your
>> serialized classes implement java.io.Externalizable interface, verify that
>> serialization logic is correct.
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
>> at
>> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
>> at
>> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>> ... 41 common frames omitted
>>
>> Thanks,
>> Akash
>>
>
Re: After upgrading 2.7 getting Unexpected error occurred during
unmarshalling
Posted by Mikael <mi...@telia.com>.
Hi!
Any chance you might have one node running 2.6 or something like that ?
It looks like it get a different object that does not match the one
expected in 2.7
Mikael
Den 2019-01-08 kl. 12:21, skrev Akash Shinde:
> Before submitting the affinity task ignite first gets the affinity
> cached function (AffinityInfo) by submitting the cluster wide task
> "AffinityJob". But while in the process of retrieving the output of
> this AffinityJob, ignite deserializes this output. I am getting
> exception while deserailizing this output.
> In TcpDiscoveryNode.readExternal() method while deserailizing the
> CacheMetrics object from input stream on 14th iteration I am getting
> following exception. Complete stack trace is given in this mail chain.
>
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. //
>
> This is working fine on Ignite 2.6 version but giving problem on 2.7.
>
> Is this a bug or am I doing something wrong?
>
> Can someone please help?
>
> On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <akashshinde@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi,
>
> When execute affinity.partition(key), I am getting following
> exception on Ignite 2.7.
>
> Stacktrace:
>
>
> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse
> [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> gridEx=null, isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal
> object with optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
> to unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
> Check that all nodes are running the same version of Ignite and
> that all nodes have GridOptimizedMarshaller configured with
> identical optimized classes lists, if any (see setClassNames and
> setClassNamesPath methods). If your serialized classes implement
> java.io.Externalizable interface, verify that serialization logic
> is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
> 2019-01-07 21:23:03,140 6699925
> [main-SendThread(qzkpr01.p13.eng.in03.com:50024
> <http://qzkpr01.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Opening socket connection to
> server qzkpr01.p13.eng.in03.com/10.114.113.201:50024
> <http://qzkpr01.p13.eng.in03.com/10.114.113.201:50024>. Will not
> attempt to authenticate using SASL (unknown error)
> 2019-01-07 21:23:03,140 6699925
> [main-SendThread(qzkpr01.p13.eng.in03.com:50024
> <http://qzkpr01.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Socket connection established to
> qzkpr01.p13.eng.in03.com/10.114.113.201:50024
> <http://qzkpr01.p13.eng.in03.com/10.114.113.201:50024>, initiating
> session
> 2019-01-07 21:23:03,140 6699925
> [main-SendThread(qzkpr01.p13.eng.in03.com:50024
> <http://qzkpr01.p13.eng.in03.com:50024>)] WARN
> org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper
> service, session 0x3673a59f5dd01c7 has expired
> 2019-01-07 21:23:03,140 6699925
> [main-SendThread(qzkpr01.p13.eng.in03.com:50024
> <http://qzkpr01.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper
> service, session 0x3673a59f5dd01c7 has expired, closing socket
> connection
> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> org.apache.zookeeper.ZooKeeper - Initiating client connection,
> connectString=qzkpr01.p13.eng.in03.com:50024
> <http://qzkpr01.p13.eng.in03.com:50024>,qzkpr02.p13.eng.in03.com:50024
> <http://qzkpr02.p13.eng.in03.com:50024>,qzkpr03.p13.eng.in03.com:50024/qkafka
> <http://qzkpr03.p13.eng.in03.com:50024/qkafka>
> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
> org.apache.zookeeper.ClientCnxn - EventThread shut down for
> session: 0x3673a59f5dd01c7
> 2019-01-07 21:23:03,296 6700081
> [main-SendThread(qzkpr03.p13.eng.in03.com:50024
> <http://qzkpr03.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Opening socket connection to
> server qzkpr03.p13.eng.in03.com/10.114.113.203:50024
> <http://qzkpr03.p13.eng.in03.com/10.114.113.203:50024>. Will not
> attempt to authenticate using SASL (unknown error)
> 2019-01-07 21:23:03,296 6700081
> [main-SendThread(qzkpr03.p13.eng.in03.com:50024
> <http://qzkpr03.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Socket connection established to
> qzkpr03.p13.eng.in03.com/10.114.113.203:50024
> <http://qzkpr03.p13.eng.in03.com/10.114.113.203:50024>, initiating
> session
> 2019-01-07 21:23:03,296 6700081
> [main-SendThread(qzkpr03.p13.eng.in03.com:50024
> <http://qzkpr03.p13.eng.in03.com:50024>)] INFO
> org.apache.zookeeper.ClientCnxn - Session establishment complete
> on server qzkpr03.p13.eng.in03.com/10.114.113.203:50024
> <http://qzkpr03.p13.eng.in03.com/10.114.113.203:50024>, sessionid
> = 0x3673a59f5dd01ce, negotiated timeout = 10000
> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse
> [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> gridEx=null, isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal
> object with optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
> to unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
> Check that all nodes are running the same version of Ignite and
> that all nodes have GridOptimizedMarshaller configured with
> identical optimized classes lists, if any (see setClassNames and
> setClassNamesPath methods). If your serialized classes implement
> java.io.Externalizable interface, verify that serialization logic
> is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse
> [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> gridEx=null, isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal
> object with optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed
> to unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
> Check that all nodes are running the same version of Ignite and
> that all nodes have GridOptimizedMarshaller configured with
> identical optimized classes lists, if any (see setClassNames and
> setClassNamesPath methods). If your serialized classes implement
> java.io.Externalizable interface, verify that serialization logic
> is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
>
> Thanks,
> Akash
>
Re: After upgrading 2.7 getting Unexpected error occurred during unmarshalling
Posted by Akash Shinde <ak...@gmail.com>.
Before submitting the affinity task ignite first gets the affinity cached
function (AffinityInfo) by submitting the cluster wide task "AffinityJob".
But while in the process of retrieving the output of this AffinityJob,
ignite deserializes this output. I am getting exception while deserailizing
this output.
In TcpDiscoveryNode.readExternal() method while deserailizing the
CacheMetrics object from input stream on 14th iteration I am getting
following exception. Complete stack trace is given in this mail chain.
Caused by: java.io.IOException: Unexpected error occurred during
unmarshalling of an instance of the class:
org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.
This is working fine on Ignite 2.6 version but giving problem on 2.7.
Is this a bug or am I doing something wrong?
Can someone please help?
On Mon, Jan 7, 2019 at 9:41 PM Akash Shinde <ak...@gmail.com> wrote:
> Hi,
>
> When execute affinity.partition(key), I am getting following exception on
> Ignite 2.7.
>
> Stacktrace:
>
> 2019-01-07 21:23:03,093 6699878 [mgmt-#67%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=4778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=5778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
> optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
> object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> that all nodes are running the same version of Ignite and that all nodes
> have GridOptimizedMarshaller configured with identical optimized classes
> lists, if any (see setClassNames and setClassNamesPath methods). If your
> serialized classes implement java.io.Externalizable interface, verify that
> serialization logic is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Opening socket connection to server
> qzkpr01.p13.eng.in03.com/10.114.113.201:50024. Will not attempt to
> authenticate using SASL (unknown error)
> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Socket connection established to
> qzkpr01.p13.eng.in03.com/10.114.113.201:50024, initiating session
> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> qzkpr01.p13.eng.in03.com:50024)] WARN org.apache.zookeeper.ClientCnxn -
> Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
> expired
> 2019-01-07 21:23:03,140 6699925 [main-SendThread(
> qzkpr01.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Unable to reconnect to ZooKeeper service, session 0x3673a59f5dd01c7 has
> expired, closing socket connection
> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
> 2019-01-07 21:23:03,140 6699925 [main-EventThread] INFO
> org.apache.zookeeper.ZooKeeper - Initiating client connection,
> connectString=qzkpr01.p13.eng.in03.com:50024,
> qzkpr02.p13.eng.in03.com:50024,qzkpr03.p13.eng.in03.com:50024/qkafka
> sessionTimeout=10000 watcher=org.I0Itec.zkclient.ZkClient@3003d288
> 2019-01-07 21:23:03,156 6699941 [main-EventThread] INFO
> org.apache.zookeeper.ClientCnxn - EventThread shut down for session:
> 0x3673a59f5dd01c7
> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Opening socket connection to server
> qzkpr03.p13.eng.in03.com/10.114.113.203:50024. Will not attempt to
> authenticate using SASL (unknown error)
> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Socket connection established to
> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, initiating session
> 2019-01-07 21:23:03,296 6700081 [main-SendThread(
> qzkpr03.p13.eng.in03.com:50024)] INFO org.apache.zookeeper.ClientCnxn -
> Session establishment complete on server
> qzkpr03.p13.eng.in03.com/10.114.113.203:50024, sessionid =
> 0x3673a59f5dd01ce, negotiated timeout = 10000
> 2019-01-07 21:23:03,296 6700081 [main-EventThread] INFO
> org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
> 2019-01-07 21:23:03,609 6700394 [mgmt-#68%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=d778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=e778e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
> optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
> object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> that all nodes are running the same version of Ignite and that all nodes
> have GridOptimizedMarshaller configured with identical optimized classes
> lists, if any (see setClassNames and setClassNamesPath methods). If your
> serialized classes implement java.io.Externalizable interface, verify that
> serialization logic is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
> 2019-01-07 21:23:04,140 6700925 [mgmt-#69%springDataNode%] ERROR
> o.a.i.i.p.task.GridTaskWorker - Error deserializing job response:
> GridJobExecuteResponse [nodeId=c0c832cb-33b0-4139-b11d-5cafab2fd046,
> sesId=f778e982861-31445139-523d-4d44-b071-9ca1eb2d73df,
> jobId=0878e982861-31445139-523d-4d44-b071-9ca1eb2d73df, gridEx=null,
> isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
> optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
> object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to
> deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
> ... 16 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.util.lang.GridTuple3]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
> ... 18 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.internal.processors.affinity.GridAffinityAssignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.util.lang.GridTuple3.readExternal(GridTuple3.java:197)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 21 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize field
> [name=assignment]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 27 common frames omitted
> Caused by: java.io.IOException: Failed to deserialize object
> [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readArrayList(OptimizedObjectInputStream.java:643)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:301)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
> ... 30 common frames omitted
> Caused by: java.io.IOException: Unexpected error occurred during
> unmarshalling of an instance of the class:
> org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check
> that all nodes are running the same version of Ignite and that all nodes
> have GridOptimizedMarshaller configured with identical optimized classes
> lists, if any (see setClassNames and setClassNamesPath methods). If your
> serialized classes implement java.io.Externalizable interface, verify that
> serialization logic is correct.
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
> at
> org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> ... 41 common frames omitted
>
> Thanks,
> Akash
>