[GitHub] [geode] jvarenina commented on a change in pull request #7145: GEODE-9854: Orphaned .drf file causing memory leak

[GitBox <gi...@apache.org>]

jvarenina commented on a change in pull request #7145:
URL: https://github.com/apache/geode/pull/7145#discussion_r760941193



##########
File path: geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
##########
@@ -5942,6 +5943,8 @@ void cleanupAfterCompaction(boolean compactFailed) {
     if (!compactFailed) {
       // all data has been copied forward to new oplog so no live entries remain
       getTotalLiveCount().set(0);
+      // No need for regionMap as there are no more live entries and .crf will be deleted
+      regionMap.set(new ConcurrentHashMap<>());

Review comment:
       @upthewaterspout thank you for the review!
   
   What you are proposing was actually our first solution to this problem and that we later changed to this current one.  When instantiated, the regionMap HashMap will be created with initial capacity of 16. This is really small value and it doesn't add a lot to the heap memory. The problem is that this HashMap can grow a lot from it's initial size during lifetime of the Oplog. It could contain hundreds or even more then thousand of elements. In order to consume that many elements the HashMap have to be rehashed to the bigger capacity. The problem is that new (bigger) capacity remain even after we clear the regionMap HashMap, and even it is empty, it is consuming not negligible amount of memory.
   
   We now know that it is possible for server to collect a lot of orphaned .drf files during the lifetime, and that difference between initial capacity of 16 and 1024 is huge in terms of memory consumption. So this is why we decided to use atomic reference and just reinitialize regionMap with new object of initial size 16. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org