You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by arunkjn <ar...@gmail.com> on 2018/08/10 11:22:05 UTC
Re: Node with BaselineTopology cannot join mixed
clusterrunningincompatibility mode
Hi,
I am experiencing the same issue with a ignite 2.5 cluster deployed in
kubernetes.
Scenario:
5 server nodes hosting some caches. They are deployed multiple times without
issues.
I connected a visor client to this cluster (this is the first time i used
visor ever on this cluster)
On another rolling update, the nodes fail to restart with the message -
class org.apache.ignite.IgniteException: Node with BaselineTopology cannot
join mixed cluster running in compatibility mode
at
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onGridDataReceived(GridClusterStateProcessor.java:714)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:883)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:1939)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4354)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2744)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
I can also see this in the logs -
class org.apache.ignite.IgniteCheckedException: Failed to start SPI:
TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000,
marsh=JdkMarshaller
[clsFilter=org.apache.ignite.internal.IgniteKernal$5@3ed03652], reconCnt=10,
reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false,
clientReconnectDisabled=false, internalLsnr=null]
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:300)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:915)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1720)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1033)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1069)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:955)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
at org.apache.ignite.Ignition.start(Ignition.java:352)
at
com.mediaiq.caps.platform.choreography.data.DataNodeStartup.main(DataNodeStartup.java:43)
Caused by: class org.apache.ignite.spi.IgniteSpiException: Thread has been
interrupted.
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:938)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:373)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1948)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
... 13 more
I am not sure if visor was still connected while deploying, but the problem
seem to persist after disconnecting visor as well. Please advise on how to
fix this. Has this been fixed in 2.6?
Thanks,
Arun
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Node with BaselineTopology cannot join mixed
clusterrunningincompatibility mode
Posted by Stanislav Lukyanov <st...@gmail.com>.
FYI the issue https://issues.apache.org/jira/browse/IGNITE-8774 is now fixed,
the fix will be available in Ignite 2.7.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Node with BaselineTopology cannot join mixed clusterrunningincompatibility
mode
Posted by Alex Plehanov <pl...@gmail.com>.
You can restart the cluster. Compatibility mode flag is not stored to PDS,
it exists only in memory.
2018-08-13 9:06 GMT+03:00 arunkjn <ar...@gmail.com>:
> Thanks Alex,
>
> How can I fix this in the current cluster without resetting the persistence
> state of entire cluster?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Node with BaselineTopology cannot join mixed
clusterrunningincompatibility mode
Posted by arunkjn <ar...@gmail.com>.
Thanks Alex,
How can I fix this in the current cluster without resetting the persistence
state of entire cluster?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Node with BaselineTopology cannot join mixed clusterrunningincompatibility
mode
Posted by Alex Plehanov <pl...@gmail.com>.
Hi, Arun
When daemon node (visor) first time join the cluster it switch the cluster
to compatibility mode. In this mode node containing baseline topology
information will fail to join even if daemon node already leave the
cluster. This issue was fixed by ticket [1] and will be available in Ignite
2.7
[1] : https://issues.apache.org/jira/browse/IGNITE-8774
2018-08-10 14:22 GMT+03:00 arunkjn <ar...@gmail.com>:
> Hi,
>
> I am experiencing the same issue with a ignite 2.5 cluster deployed in
> kubernetes.
>
> Scenario:
>
> 5 server nodes hosting some caches. They are deployed multiple times
> without
> issues.
> I connected a visor client to this cluster (this is the first time i used
> visor ever on this cluster)
> On another rolling update, the nodes fail to restart with the message -
> class org.apache.ignite.IgniteException: Node with BaselineTopology cannot
> join mixed cluster running in compatibility mode
> at
> org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.
> onGridDataReceived(GridClusterStateProcessor.java:714)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.
> onExchange(GridDiscoveryManager.java:883)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.
> onExchange(TcpDiscoverySpi.java:1939)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processNodeAddedMessage(ServerImpl.java:4354)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2744)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.
> processMessage(ServerImpl.java:2536)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(
> ServerImpl.java:6775)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(
> ServerImpl.java:2621)
> at org.apache.ignite.spi.IgniteSpiThread.run(
> IgniteSpiThread.java:62)
>
>
> I can also see this in the logs -
>
> class org.apache.ignite.IgniteCheckedException: Failed to start SPI:
> TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000,
> marsh=JdkMarshaller
> [clsFilter=org.apache.ignite.internal.IgniteKernal$5@3ed03652],
> reconCnt=10,
> reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false,
> clientReconnectDisabled=false, internalLsnr=null]
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.
> startSpi(GridManagerAdapter.java:300)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(
> GridDiscoveryManager.java:915)
> at
> org.apache.ignite.internal.IgniteKernal.startManager(
> IgniteKernal.java:1720)
> at org.apache.ignite.internal.IgniteKernal.start(
> IgniteKernal.java:1033)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(
> IgnitionEx.java:2014)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(
> IgnitionEx.java:1723)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.
> java:1151)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(
> IgnitionEx.java:1069)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:955)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:854)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:724)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.
> java:693)
> at org.apache.ignite.Ignition.start(Ignition.java:352)
> at
> com.mediaiq.caps.platform.choreography.data.DataNodeStartup.main(
> DataNodeStartup.java:43)
> Caused by: class org.apache.ignite.spi.IgniteSpiException: Thread has been
> interrupted.
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.
> joinTopology(ServerImpl.java:938)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.
> spiStart(ServerImpl.java:373)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.
> spiStart(TcpDiscoverySpi.java:1948)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.
> startSpi(GridManagerAdapter.java:297)
> ... 13 more
>
>
> I am not sure if visor was still connected while deploying, but the problem
> seem to persist after disconnecting visor as well. Please advise on how to
> fix this. Has this been fixed in 2.6?
>
> Thanks,
> Arun
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>