You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Gabriel Jimenez (BLOOMBERG/ 731 LEX)" <gj...@bloomberg.net> on 2019/02/23 00:55:12 UTC
Ignite 2.7 - New Issues on Node Termination
Hello,
I have been working with Apache Ignite for the past couple of months, mainly 2.6, and just recently upgraded our framework to use 2.7. However, I have encountered some concerning issues on grid node termination, and was wondering if anyone else had similar experiences or could reference available solutions.
While using 2.6, our grid always successfully terminated nodes, printing out
'Ignite ver. 2.6.0#(...) stopped OK'
However, since making the upgrade to 2.7.0 I have not had a successful node termination when the grid is under load (namely during grid rolling restart). At first I thought 2.7.0 might have improved error reporting, and these were errors present before that were not being caught. After looking into it more, it just seems as if each component appears to be throwing its own variant of 'Ignite___Exception: Node is stopping'. My confidence dropped even further when I noticed one of the 'errors':
"
[ERROR] [Thread-27] IgniteKernal - Failed to stop component (ignoring): GridProcessorAdapter []
java.lang.UnsupportedOperationException: null
at org.jsr166.ConcurrentLinkedHashMap.clear(ConcurrentLinkedHashMap.java:1551) ~[bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.processors.job.GridJobProcessor.stop(GridJobProcessor.java:264) ~[bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2356) [bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2228) [bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2612) [bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2575) [bdp-ignite-core-2.7.0.jar:2.7.0]
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$6.run(IgnitionEx.java:2100) [bdp-ignite-core-2.7.0.jar:2.7.0]
"
The relevant lines of code are:
GridJobProcessor.stop(): https://github.com/apache/ignite/blob/2.7.0/modules/core/src/main/java/org/apache/ignite/internal/processors/job/GridJobProcessor.java#L264
ConcurrentLinkedHashMap.clear(): https://github.com/apache/ignite/blob/2.7.0/modules/core/src/main/java/org/jsr166/ConcurrentLinkedHashMap.java#L1550
It seems ConcurrentLinkedHashMap was recently changed to make clear an unsupported operation, but that creates what would be arguably false errors on node termination. Has anyone else encountered issues on node termination in 2.7, or perhaps the mistake is on my end and I am missing something critical. Any insight on this matter would be greatly appreciated!
Sincerely,
Gabriel
Re: Ignite 2.7 - New Issues on Node Termination
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
This is a known issue: https://issues.apache.org/jira/browse/IGNITE-10860
Regards,
--
Ilya Kasnacheev
сб, 23 февр. 2019 г. в 03:55, Gabriel Jimenez (BLOOMBERG/ 731 LEX) <
gjimenez16@bloomberg.net>:
> Hello,
>
> I have been working with Apache Ignite for the past couple of months,
> mainly 2.6, and just recently upgraded our framework to use 2.7. However, I
> have encountered some concerning issues on grid node termination, and was
> wondering if anyone else had similar experiences or could reference
> available solutions.
>
> While using 2.6, our grid always successfully terminated nodes, printing
> out
>
> 'Ignite ver. 2.6.0#(...) stopped OK'
>
> However, since making the upgrade to 2.7.0 I have not had a successful
> node termination when the grid is under load (namely during grid rolling
> restart). At first I thought 2.7.0 might have improved error reporting, and
> these were errors present before that were not being caught. After looking
> into it more, it just seems as if each component appears to be throwing its
> own variant of 'Ignite___Exception: Node is stopping'. My confidence
> dropped even further when I noticed one of the 'errors':
>
> "
> [ERROR] [Thread-27] IgniteKernal - Failed to stop component (ignoring):
> GridProcessorAdapter []
> java.lang.UnsupportedOperationException: null
> at
> org.jsr166.ConcurrentLinkedHashMap.clear(ConcurrentLinkedHashMap.java:1551)
> ~[bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor.stop(GridJobProcessor.java:264)
> ~[bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2356)
> [bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2228)
> [bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2612)
> [bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2575)
> [bdp-ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$6.run(IgnitionEx.java:2100)
> [bdp-ignite-core-2.7.0.jar:2.7.0]
> "
>
> The relevant lines of code are:
>
> GridJobProcessor.stop():
> https://github.com/apache/ignite/blob/2.7.0/modules/core/src/main/java/org/apache/ignite/internal/processors/job/GridJobProcessor.java#L264
> ConcurrentLinkedHashMap.clear():
> https://github.com/apache/ignite/blob/2.7.0/modules/core/src/main/java/org/jsr166/ConcurrentLinkedHashMap.java#L1550
>
> It seems ConcurrentLinkedHashMap was recently changed to make clear an
> unsupported operation, but that creates what would be arguably false errors
> on node termination. Has anyone else encountered issues on node termination
> in 2.7, or perhaps the mistake is on my end and I am missing something
> critical. Any insight on this matter would be greatly appreciated!
>
> Sincerely,
> Gabriel