You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by terryzhuo <te...@huawei.com> on 2019/01/14 16:46:40 UTC

Unable to activate an ignite cluster with multiple hosts.

I tried to activate my 3-node ignite cluster with "control.sh --activate",
but the activation keeps failing. The command just hang there for ever. The
same activate command works if the ignite cluster has only one node. But it
does not work if the cluster has multiple nodes. 

I am using apache-ignite-fabric-2.6.0-bin. The hosts are in one VPC in AWS. 

*Ignite Console Output:*
[16:29:47] Ignite node started OK (id=fd642d49)
[16:29:47] Topology snapshot [*ver=1, servers=1*, clients=0, CPUs=8,
offheap=12.0GB, heap=1.0GB]
[16:29:47]   ^-- Node [id=FD642D49-1E12-415C-85A9-BFFA028926F9,
clusterState=ACTIVE]
[16:29:47] Data Regions Configured:
[16:29:47]   ^-- default [initSize=256.0 MiB, maxSize=12.0 GiB,
persistenceEnabled=false]
[16:29:56] New version is available at ignite.apache.org: 2.7.0
[16:30:15] Topology snapshot [*ver=2, servers=2*, clients=0, CPUs=16,
offheap=24.0GB, heap=2.0GB]
[16:30:15]   ^-- Node [id=FD642D49-1E12-415C-85A9-BFFA028926F9,
clusterState=ACTIVE]
[16:30:15] Data Regions Configured:
[16:30:15]   ^-- default [initSize=256.0 MiB, maxSize=12.0 GiB,
persistenceEnabled=false]
[16:31:42] Topology snapshot [*ver=3, servers=3*, clients=0, CPUs=24,
offheap=36.0GB, heap=3.0GB]
[16:31:42]   ^-- Node [id=FD642D49-1E12-415C-85A9-BFFA028926F9,
clusterState=ACTIVE]
[16:31:42] Data Regions Configured:
[16:31:42]   ^-- default [initSize=256.0 MiB, maxSize=12.0 GiB,
persistenceEnabled=false]


*Ignite Log:*
[16:31:42,947][INFO][disco-event-worker-#41][GridDiscoveryManager] Added new
node to topology: TcpDiscoveryNode [id=ccd096f7-1daa-4a8f-9097-cb9e6ed0d07d,
addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.20.177],
sockAddrs=[/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500,
ip-172-31-20-177.us-west-1.compute.internal/172.31.20.177:47500],
discPort=47500, order=3, intOrder=3, lastExchangeTime=1547483492859,
loc=false, ver=2.6.0#20180710-sha1:669feacc, isClient=false]
[16:31:42,948][INFO][disco-event-worker-#41][GridDiscoveryManager] Topology
snapshot [*ver=3*, servers=3, clients=0, CPUs=24, offheap=36.0GB,
heap=3.0GB]
[16:31:42,948][INFO][disco-event-worker-#41][GridDiscoveryManager]   ^--
Node [id=FD642D49-1E12-415C-85A9-BFFA028926F9, clusterState=ACTIVE]
[16:31:42,948][INFO][disco-event-worker-#41][GridDiscoveryManager] Data
Regions Configured:
[16:31:42,948][INFO][disco-event-worker-#41][GridDiscoveryManager]   ^--
default [initSize=256.0 MiB, maxSize=12.0 GiB, persistenceEnabled=false]
[16:31:45,858][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:31:47,133][INFO][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=fd642d49, uptime=00:02:00.000]
    ^-- H/N/C [hosts=3, nodes=3, CPUs=24]
    ^-- CPU [cur=0.07%, avg=0.21%, GC=0%]
    ^-- PageMemory [pages=200]
    ^-- Heap [used=72MB, free=92.57%, comm=981MB]
    ^-- Non heap [used=45MB, free=93.83%, comm=46MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=6, qSize=0]
[16:31:55,859][*WARNING*][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
*Unable to await partitions release latch* within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:32:05,859][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:32:15,860][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:32:25,860][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:32:35,861][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]
[16:32:45,861][WARNING][exchange-worker-#42][GridDhtPartitionsExchangeFuture]
Unable to await partitions release latch within timeout: ServerLatch
[permits=1, pendingAcks=[35bb2402-58a3-4594-bed4-17e2e7eb939a],
super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=0]]]


*Ignite Config:*
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">

                <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            
                            <value>172.31.26.7:47500..47509</value>
                            <value>172.31.27.148:47500..47509</value>
                            <value>172.31.20.177:47500..47509</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

ubuntu@ip-172-31-27-148:~/apache-ignite-fabric-2.6.0-bin$ *netstat -nlpt*
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
PID/Program name
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN     
-
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
-
tcp6       0      0 :::47100                :::*                    LISTEN     
1091/java
tcp6       0      0 :::49154                :::*                    LISTEN     
1091/java
tcp6       0      0 :::33353                :::*                    LISTEN     
1091/java
tcp6       0      0 :::11211                :::*                    LISTEN     
1091/java
tcp6       0      0 :::47500                :::*                    LISTEN     
1091/java
tcp6       0      0 :::10800                :::*                    LISTEN     
1091/java
tcp6       0      0 :::80                   :::*                    LISTEN     
-
tcp6       0      0 :::45971                :::*                    LISTEN     
1091/java
tcp6       0      0 :::22                   :::*                    LISTEN     
-
tcp6       0      0 :::3001                 :::*                    LISTEN     
-

Thanks
Terry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unable to activate an ignite cluster with multiple hosts.

Posted by Mikhail <mi...@gmail.com>.
Hi,

as I can see you created a cluster without ignite native persistence, so it
already active.

Activation is required only when you created cluster with persistence:
https://apacheignite.readme.io/docs/distributed-persistent-store

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/