You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Semen Boikov (JIRA)" <ji...@apache.org> on 2016/08/30 06:40:20 UTC

[jira] [Updated] (IGNITE-1183) IgniteAtomicLong closing and getting during rolling restart causes hangs

     [ https://issues.apache.org/jira/browse/IGNITE-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Semen Boikov updated IGNITE-1183:
---------------------------------
    Assignee:     (was: Semen Boikov)

> IgniteAtomicLong closing and getting during rolling restart causes hangs
> ------------------------------------------------------------------------
>
>                 Key: IGNITE-1183
>                 URL: https://issues.apache.org/jira/browse/IGNITE-1183
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Andrey Gura
>         Attachments: AtomicLongChangingTopologySelfTest.java, log.zip, thread_dump.zip
>
>
> During rolling restart of nodes some threads hang on closing or getting {{IgniteAtomicLong}}.
> Stack trace for thread that hangs on {{IgniteAtomicLong.close()}}:
> {noformat}
> "Test worker" #11 prio=5 os_prio=31 tid=0x00007fbbf4ae7800 nid=0x5b03 waiting on condition [0x0000000122f93000]
> java.lang.Thread.State: WAITING (parking)
>  at sun.misc.Unsafe.park(Native Method)
>  - parking to wait for <0x000000076db515a0> (a org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture)
>  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>  at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>  at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
>  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
>  at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:115)
>  at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4376)
>  at org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1448)
>  at org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.get(GridCacheProxyImpl.java:288)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$14.call(DataStructuresProcessor.java:1086)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$14.call(DataStructuresProcessor.java:1082)
>  at org.apache.ignite.internal.processors.cache.GridCacheUtils.outTx(GridCacheUtils.java:939)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeInternal(DataStructuresProcessor.java:1081)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.access$800(DataStructuresProcessor.java:51)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:453)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:448)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeDataStructure(DataStructuresProcessor.java:507)
>  at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeAtomicLong(DataStructuresProcessor.java:448)
>  at org.apache.ignite.internal.processors.datastructures.GridCacheAtomicLongImpl.close(GridCacheAtomicLongImpl.java:368)
> {noformat}
> Stack trace for thread that hangs on {{Ignite.atomicLong()}}:
> {noformat}
> "datastructures.AtomicLongChangingTopologySelfTest-1" prio=10 tid=0x00007f9cd800a800 nid=0x2bb3 waiting on condition [0x00007f9c49cfc000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x00000007f106e580> (a org.apache.ignite.internal.util.future.GridEmbeddedFuture)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
> 	at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:113)
> 	at org.apache.ignite.internal.processors.cache.GridCacheAdapter$24.op(GridCacheAdapter.java:2089)
> 	at org.apache.ignite.internal.processors.cache.GridCacheAdapter$24.op(GridCacheAdapter.java:2081)
> 	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:3990)
> 	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2081)
> 	at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:449)
> 	at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.atomicLong(DataStructuresProcessor.java:364)
> 	at org.apache.ignite.internal.IgniteKernal.atomicLong(IgniteKernal.java:2725)
> 	at org.apache.ignite.internal.processors.cache.datastructures.AtomicLongChangingTopologySelfTest$1.run(AtomicLongChangingTopologySelfTest.java:180)
> 	at org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:911)
> 	at org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86)
> {noformat}
> During testing I've noticed also that {{IgniteAtomicLong.getAndIncrement}} sometimes returns inconsistent values (sequence has gaps).
> See also full log ([^log.zip]) and full thread dump for second case ([^thread_dump.zip]).
> See attached test [^AtomicLongChangingTopologySelfTest.java] that reproduces this problem (not stable, thus can't be part of some test suite of Ignite). Note that {{CHECK_CONSISTENCY}} parameter controls whether test check consistency (in this case {{IgniteAtmocInstance}} will not be closed) or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)