You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "javastuff.sam@gmail.com" <ja...@gmail.com> on 2016/12/01 22:00:05 UTC

Re: Cluster hung after a node killed

Val,

I have reproduced this with simple program - 
1. Node 1 - run example ExampleNodeStartup 
2. Node 2 - Run a program which create a transaction cache and add 100K
simple entries. <String, String> 
        cfg.setCacheMode(CacheMode.PARTITIONED); 
        cfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); 
        cfg.setMemoryMode(CacheMemoryMode.OFFHEAP_TIERED); 
        cfg.setSwapEnabled(false); 
        cfg.setBackups(0); 
3. Node 3 - Run a program which takes a lock (cache.lock(key)) 
4. Kill Node 3 before it can unlock.
5. Node 4 - Run a program which tries to get cached data. 

Node4 not able to join cluster, it is hung. In-fact complete cluster is hung
andy operation from Node2 also hung.

-Sam



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9337.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by vkulichenko <va...@gmail.com>.
Sam,

There is no exact date as for now. I would recommend to monitor the dev list
for activity around this.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9883.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by "javastuff.sam@gmail.com" <ja...@gmail.com>.
Thanks Val. Ticket has been fixed for 1.9, Any idea when 1.9 will be
available?

-Sam



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9880.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by vkulichenko <va...@gmail.com>.
Hi Sam,

I reproduced the issue using your code and created a ticket:
https://issues.apache.org/jira/browse/IGNITE-4450

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9616.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by "javastuff.sam@gmail.com" <ja...@gmail.com>.
Hi Val,

Did you had chance to look at attached sample program? Did it help figuring
out what is going on?

-Sam



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9465.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by "javastuff.sam@gmail.com" <ja...@gmail.com>.
Attaching 4 classes -
Node1.java - create empty cache node. 
Node2.java - seed cache with 100K <int,Str> dat.
Node3.java - take explicit lock on a key and wait for 15 seconds to unlock. 
Node4.java - fetch cached data.

Steps -
1. Run Node1, wait for empty node to boot up.
2. Run Node2, wait for completion message.
3. Run Node3, kill it when it prompts for "kill me...".
4. Run Node4 Topology snapshot on other node will show node joined, however
Node 4 will not able to complete with any fetch operations. Fetch from cache
hung.
5. Run another instance of Node2, it will not able to complete with any put
operation. Put to cache hung.

Node1.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n9375/Node1.java>  
Node2.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n9375/Node2.java>  
Node3.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n9375/Node3.java>  
Node4.java
<http://apache-ignite-users.70518.x6.nabble.com/file/n9375/Node4.java>  



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9375.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster hung after a node killed

Posted by vkulichenko <va...@gmail.com>.
Hi Sam,

Can you create a small project on GitHub with instructions how to run? I
just tried to repeat the same test, but everything works fine for me.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cluster-hung-after-a-node-killed-tp8965p9340.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.