You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Maxim Muzafarov (Jira)" <ji...@apache.org> on 2020/05/30 15:34:00 UTC
[jira] [Updated] (IGNITE-11235) Sending Continuous Query with
absent class on client knocks out server node
[ https://issues.apache.org/jira/browse/IGNITE-11235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maxim Muzafarov updated IGNITE-11235:
-------------------------------------
Labels: cq_todo discovery (was: discovery)
> Sending Continuous Query with absent class on client knocks out server node
> ---------------------------------------------------------------------------
>
> Key: IGNITE-11235
> URL: https://issues.apache.org/jira/browse/IGNITE-11235
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.7
> Reporter: Ilya Kasnacheev
> Priority: Critical
> Labels: cq_todo, discovery
>
> *In 2.7*, when you create a Continuous Query on client with class which is not present in server classpath, server node will be terminated immediately.
> {code}
> [2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false]]]]
> class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false]]
> {code}
> This means that one rogue client (e.g. with mismatched code base version) can knock whole cluster offline.
> I think we should disconnect client from topology instead when it sends incorrect discovery message.
> *In 2.6*, if you tried to create a Continuous Query on client with class which is not present in server classpath, you will get the following erorr on server:
> {code}
> [15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery custom message.
> class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader for unmarshalling (make sure same versions of all classes are available on all nodes or enable peer-class-loading) [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
> at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
> at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
> at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
> at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
> at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
> at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> ... 12 more
> [15:32:57,846][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery custom message.
> class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader for unmarshalling (make sure same versions of all classes are available on all nodes or enable peer-class-loading) [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
> at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
> at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
> at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5323)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
> at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
> at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
> at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
> at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
> at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
> ... 11 more
> {code}
> and client thread will wait forever:
> {code}
> "main" #1 prio=5 os_prio=0 tid=0x00007f403800d000 nid=0x1fa2 waiting on condition [0x00007f403e74a000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery0(CacheContinuousQueryManager.java:694)
> at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery(CacheContinuousQueryManager.java:517)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.queryContinuous(IgniteCacheProxyImpl.java:569)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:658)
> at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:344)
> {code}
> But otherwise both client and server are intact and no hanging PME occurs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)