[GitHub] [bookkeeper] wenbingshen commented on a diff in pull request #3417: Optimize concurrent collection's shrink logic

[GitBox <gi...@apache.org>]

wenbingshen commented on code in PR #3417:
URL: https://github.com/apache/bookkeeper/pull/3417#discussion_r929570822


##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/collections/ConcurrentLongHashMap.java:
##########
@@ -507,7 +507,11 @@ private V remove(long key, Object value, int keyHash) {
             } finally {
                 if (autoShrink && size < resizeThresholdBelow) {
                     try {
-                        int newCapacity = alignToPowerOfTwo((int) (capacity / shrinkFactor));
+                        // Shrinking must at least ensure initCapacity,
+                        // so as to avoid frequent shrinking and expansion near initCapacity,
+                        // frequent shrinking and expansion,
+                        // additionally opened arrays will consume more memory and affect GC
+                        int newCapacity = Math.max(alignToPowerOfTwo((int) (capacity / shrinkFactor)), initCapacity);

Review Comment:
   Assuming that `initCapacity` is 256,`mapFillFactor, mapIdleFactor, expandFactor, shrinkFactor ` keep the default values:
   
   | initCapacity | capacity | mapFillFactor | mapIdleFactor | resizeThresholdUp | resizeThresholdBelow | expandFactor | shrinkFactor |
   | -----------  | -------- | ------------- | ------------- | ----------------- | -------------------- | ------------ | ------------ |
   |    256       |    256   |     0.66f     |     0.15f     |         168       |           38         |      2       |        2     |
   |    256       |    512   |     0.66f     |     0.15f     |         337       |           76         |      2       |        2     |
   |    256       |    1024  |     0.66f     |     0.15f     |         675       |           153        |      2       |        2     |
   |    256       |    2048  |     0.66f     |     0.15f     |         1351      |           307        |      2       |        2     |
   
   When the `capacity` is expanded to 1024, the `size` needs to be less than 153 to shrink, and size=153 is less than initCapacity=256, causing at least 1024-153 of space wasted. This waste is worse when initCapacity is larger.
   
   Therefore, we cannot use `size` to judge that it will not shrink when it is smaller than `initCapacity`.
   
   @zymap PTAL



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org