You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Kirk Lund <kl...@apache.org> on 2020/05/26 17:14:23 UTC

DiskStoreImpl.waitForBackgroundTasks hang during Cache.close

We have a handful of Jira tickets filed against various off-heap tests that
are intermittently hanging in DiskStoreImpl.waitForBackgroundTasks during
Cache.close:

"Test worker" #25 prio=5 os_prio=0 tid=0x00007f1a7cb13800 nid=0x5e in
Object.wait() [0x00007f18ff6bd000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at org.apache.geode.internal.cache.DiskStoreImpl.waitForBackgroundTasks(DiskStoreImpl.java:2631)
        - locked <0x00000000fb920030> (a
java.util.concurrent.atomic.AtomicInteger)
        at org.apache.geode.internal.cache.DiskStoreImpl.close(DiskStoreImpl.java:2387)
        at org.apache.geode.internal.cache.DiskStoreImpl.close(DiskStoreImpl.java:2297)
        at org.apache.geode.internal.cache.GemFireCacheImpl.closeDiskStores(GemFireCacheImpl.java:2571)
        at org.apache.geode.internal.cache.GemFireCacheImpl.doClose(GemFireCacheImpl.java:2300)
        - locked <0x00000000d00bfd38> (a java.lang.Class for
org.apache.geode.internal.cache.GemFireCacheImpl)
        at org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:2119)
        at org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1987)
        at org.apache.geode.internal.cache.GemFireCacheImpl.close(GemFireCacheImpl.java:1977)
        at org.apache.geode.internal.offheap.OffHeapRegionBase.closeCache(OffHeapRegionBase.java:106)
        at org.apache.geode.internal.offheap.OffHeapRegionBase.testPersistentCompressorChange(OffHeapRegionBase.java:709)


Bruce did some analysis and found that the background tasks in
DiskStoreImpl is empty when the hang occurs.

1) GEODE-7524 CI failure: hang in
FreeListOffHeapRegionJUnitTest.testPersistentCompressorChange
<https://issues.apache.org/jira/browse/GEODE-7524>
2) GEODE-7876 OldFreeListOffHeapRegionJUnitTest
testPersistentChangeFromHeapToOffHeap
<https://issues.apache.org/jira/browse/GEODE-7876>
3) GEODE-8187
OldFreeListOffHeapRegionJUnitTest.testPersistentCompressorChange hangs in
DiskStoreImpl.waitForBackgroundTasks
<https://issues.apache.org/jira/browse/GEODE-8187>

Does anyone with deeper DiskStoreImpl knowledge want to take a look at this
hang?