You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Igor Belyakov (Jira)" <ji...@apache.org> on 2021/04/16 16:42:00 UTC

[jira] [Updated] (IGNITE-14583) Server node fails on remote filter deployment after client reconnect with new node id

     [ https://issues.apache.org/jira/browse/IGNITE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Igor Belyakov updated IGNITE-14583:
-----------------------------------
    Description: 
Cluster contains 1 server and 1 client node. PeerClassLoading turned on.

*1.* Client node deploys CQ with remote filter, server node doesn't have classes required for remote filter in classpath.

*2.* Remote filter successfully deployed to server node.

*3.* Client node disconnects and reconnects by using new node id:

[15:44:43] Client node was reconnected after it was already considered failed by the server topology (this could happen after all servers restarted or due to a long network outage between the client and servers). All continuous queries and remote event listeners created by this client will be unsubscribed, consider listening to EVT_CLIENT_NODE_RECONNECTED event to restore them.

*4.* On connect, client node tries to deploy remote filter to server node, but server node fails due to wrong ClassLoaderId, which is sent in GridDeploymentInfo (used ClassLoaderId for old client node):
{code:java}
[15:44:42] (err) Failed to notify listener: o.a.i.i.util.future.GridFutureChainListener@778d0fc6class org.apache.ignite.IgniteException: Failed to initialize a remote filter.
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.lambda$register$2bf956f5$1(CacheContinuousQueryHandler.java:353)
 at org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:77)
 at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:69)
 at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:29)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
 at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:477)
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1311)
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
 at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
 at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
 at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 at java.base/java.lang.Thread.run(Thread.java:834)
 Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
 ... 8 more
 авг. 07, 2020 3:44:42 PM org.apache.ignite.logger.java.JavaLogger error
 SEVERE: Failed to unmarshal continuous routine handler [routineId=8f0137dc-cb32-40ff-83f4-867f272d338e, srcNodeId=ebcf4d5b-a476-43d5-b057-00b5694104e6]
 class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
 at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
 at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
 at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
 at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
 at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 at java.base/java.lang.Thread.run(Thread.java:834){code}

  was:
Cluster contains 1 server and 1 client node. PeerClassLoading turned on.

*1.* Client node deploys CQ with remote filter, server node doesn't have classes required for remote filter in classpath.

*2.* Remote filter successfully deployed to server node.

*3.* Client node disconnects and reconnects by using new node id:

[15:44:43] Client node was reconnected after it was already considered failed by the server topology (this could happen after all servers restarted or due to a long network outage between the client and servers). All continuous queries and remote event listeners created by this client will be unsubscribed, consider listening to EVT_CLIENT_NODE_RECONNECTED event to restore them.

*4.* On connect, client node tries to deploy remote filter to server node, but server node fails due to wrong ClassLoaderId, which is sent in GridDeploymentInfo (used ClassLoaderId for old client node):

[15:44:42] (err) Failed to notify listener: o.a.i.i.util.future.GridFutureChainListener@778d0fc6class org.apache.ignite.IgniteException: Failed to initialize a remote filter.
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.lambda$register$2bf956f5$1(CacheContinuousQueryHandler.java:353)
at org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:77)
at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:69)
at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:29)
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:477)
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1311)
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
... 8 more
авг. 07, 2020 3:44:42 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to unmarshal continuous routine handler [routineId=8f0137dc-cb32-40ff-83f4-867f272d338e, srcNodeId=ebcf4d5b-a476-43d5-b057-00b5694104e6]
class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)


> Server node fails on remote filter deployment after client reconnect with new node id
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-14583
>                 URL: https://issues.apache.org/jira/browse/IGNITE-14583
>             Project: Ignite
>          Issue Type: Bug
>    Affects Versions: 2.10
>            Reporter: Igor Belyakov
>            Assignee: Igor Belyakov
>            Priority: Major
>
> Cluster contains 1 server and 1 client node. PeerClassLoading turned on.
> *1.* Client node deploys CQ with remote filter, server node doesn't have classes required for remote filter in classpath.
> *2.* Remote filter successfully deployed to server node.
> *3.* Client node disconnects and reconnects by using new node id:
> [15:44:43] Client node was reconnected after it was already considered failed by the server topology (this could happen after all servers restarted or due to a long network outage between the client and servers). All continuous queries and remote event listeners created by this client will be unsubscribed, consider listening to EVT_CLIENT_NODE_RECONNECTED event to restore them.
> *4.* On connect, client node tries to deploy remote filter to server node, but server node fails due to wrong ClassLoaderId, which is sent in GridDeploymentInfo (used ClassLoaderId for old client node):
> {code:java}
> [15:44:42] (err) Failed to notify listener: o.a.i.i.util.future.GridFutureChainListener@778d0fc6class org.apache.ignite.IgniteException: Failed to initialize a remote filter.
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.lambda$register$2bf956f5$1(CacheContinuousQueryHandler.java:353)
>  at org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:77)
>  at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:69)
>  at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:29)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:477)
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1311)
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
>  at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
>  at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
>  at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
>  at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
>  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
>  ... 8 more
>  авг. 07, 2020 3:44:42 PM org.apache.ignite.logger.java.JavaLogger error
>  SEVERE: Failed to unmarshal continuous routine handler [routineId=8f0137dc-cb32-40ff-83f4-867f272d338e, srcNodeId=ebcf4d5b-a476-43d5-b057-00b5694104e6]
>  class org.apache.ignite.internal.IgniteDeploymentCheckedException: Failed to obtain deployment for class: Client1$$Lambda$513/0x00000008003b8840
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryDeployableObject.unmarshal(CacheContinuousQueryDeployableObject.java:94)
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1308)
>  at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.p2pUnmarshal(CacheContinuousQueryHandler.java:1283)
>  at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.run(GridContinuousProcessor.java:693)
>  at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7162)
>  at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:826)
>  at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
>  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834){code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)