[jira] [Created] (GEODE-6564) Clearing a replicated region with expiration causes a memory leak

["Barry Oglesby (JIRA)" <ji...@apache.org>]

Barry Oglesby created GEODE-6564:
------------------------------------

             Summary: Clearing a replicated region with expiration causes a memory leak
                 Key: GEODE-6564
                 URL: https://issues.apache.org/jira/browse/GEODE-6564
             Project: Geode
          Issue Type: Bug
          Components: regions
            Reporter: Barry Oglesby


Clearing a replicated region with expiration causes a memory leak

Both the RegionEntries and EntryExpiryTasks are still live after loading entries into the region and then clearing it.

Server Startup:
{noformat}
 num #instances #bytes class name
----------------------------------------------
 1: 29856 2797840 [C
 4: 2038 520600 [B
Total 187711 10089624
{noformat}
Load 100 entries with 600k payload (representing a session):
{noformat}
 num #instances #bytes class name
----------------------------------------------
 1: 2496 60666440 [B
 2: 30157 2828496 [C
 73: 100 7200 org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1
 93: 100 4800 org.apache.geode.internal.cache.EntryExpiryTask
Total 190737 70240472
{noformat}
Clear region:
{noformat}
 num #instances #bytes class name
----------------------------------------------
 1: 2398 60505944 [B
 2: 30448 2849456 [C
 74: 100 7200 org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1
 100: 100 4800 org.apache.geode.internal.cache.EntryExpiryTask
Total 192199 70373048
{noformat}
Load and clear another 100 entries:
{noformat}
 num #instances #bytes class name
----------------------------------------------
 1: 2503 120511688 [B
 2: 30506 2854384 [C
 46: 200 14400 org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1
 61: 200 9600 org.apache.geode.internal.cache.EntryExpiryTask
Total 193272 130421432
{noformat}
Load and clear another 100 entries:
{noformat}
 num #instances #bytes class name
----------------------------------------------
 1: 2600 180517240 [B
 2: 30562 2859584 [C
 33: 300 21600 org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1
 47: 300 14400 org.apache.geode.internal.cache.EntryExpiryTask
Total 194310 190468176
{noformat}
A heap dump shows the VersionedStatsRegionEntryHeapStringKey1 instances are referenced by the DistributedRegion entryExpiryTasks:
{noformat}
--> org.apache.geode.internal.cache.DistributedRegion@0x76adbbb88 (816 bytes) (field entryExpiryTasks:)
--> java.util.concurrent.ConcurrentHashMap@0x76adbc028 (100 bytes) (field table:)
--> [Ljava.util.concurrent.ConcurrentHashMap$Node;@0x76ee85358 (4112 bytes) (Element 276 of [Ljava.util.concurrent.ConcurrentHashMap$Node;@0x76ee85358:)
--> java.util.concurrent.ConcurrentHashMap$Node@0x76edc4e20 (44 bytes) (field next:)
--> java.util.concurrent.ConcurrentHashMap$Node@0x76edc32f0 (44 bytes) (field key:)
--> org.apache.geode.internal.cache.entries.VersionedStatsRegionEntryHeapStringKey1@0x76edc3210 (86 bytes) 
{noformat}
LocalRegion.cancelAllEntryExpiryTasks is called when the region is cleared:
{noformat}
java.lang.Exception: Stack trace
 at java.lang.Thread.dumpStack(Thread.java:1333)
 at org.apache.geode.internal.cache.LocalRegion.cancelAllEntryExpiryTasks(LocalRegion.java:8202)
 at org.apache.geode.internal.cache.LocalRegion.clearRegionLocally(LocalRegion.java:9094)
 at org.apache.geode.internal.cache.DistributedRegion.cmnClearRegion(DistributedRegion.java:1962)
 at org.apache.geode.internal.cache.LocalRegion.basicClear(LocalRegion.java:8998)
 at org.apache.geode.internal.cache.DistributedRegion.basicClear(DistributedRegion.java:1939)
 at org.apache.geode.internal.cache.LocalRegion.basicBridgeClear(LocalRegion.java:8988)
 at org.apache.geode.internal.cache.tier.sockets.command.ClearRegion.cmdExecute(ClearRegion.java:123)
{noformat}
But it doesn't clear the entryExpiryTasks map:
{noformat}
LocalRegion.clearRegionLocally before cancelAllEntryExpiryTasks entryExpiryTasks=100
LocalRegion.clearRegionLocally after cancelAllEntryExpiryTasks entryExpiryTasks=100
{noformat}
As a test, I added this call to the bottom of the cancelAllEntryExpiryTasks method:
{noformat}
this.entryExpiryTasks.clear();
{noformat}
This addressed the leak:
{noformat}
Server Startup: Total 182414 9855616
Load/Clear 1: Total 191049 10315832
Load/Clear 2: Total 191978 10329664
Load/Clear 3: Total 192638 10360360
{noformat}
As a work-around, a Function that clears the region by using removeAll on batches of keys also addresses the leak:
{noformat}
Server Startup: Total 182297 9849312
Load/Clear 1: Total 185932 10019248
Load/Clear 2: Total 191855 10278816
Load/Clear 3: Total 192511 10313168
Load/Clear 4: Total 193424 10352008
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)