You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "rootvector2 (via GitHub)" <gi...@apache.org> on 2026/07/07 09:37:00 UTC

[PR] clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

rootvector2 opened a new pull request, #705:
URL: https://github.com/apache/commons-collections/pull/705

   `add(Object, int)` in `AbstractMapBag` and `AbstractMapMultiSet` accumulates the running `size` and the per-element count into plain `int` fields with no upper bound, so a second `add` whose total crosses `Integer.MAX_VALUE` overflows both: `add(x, Integer.MAX_VALUE)` then `add(x, 1)` leaves `getCount(x) == Integer.MIN_VALUE` and `size()` negative. Clamp both to `Integer.MAX_VALUE` inside `add()`, matching the `Collection.size()` contract and the existing `BloomFilter`/`CompositeMap.size()` idiom. Found while auditing the bag/multiset count arithmetic; the regression tests in `HashBagTest` and `HashMultiSetTest` fail before the change (`-4` / `Integer.MIN_VALUE`) and pass after.
   
   - [x] Read the [contribution guidelines](CONTRIBUTING.md) for this project.
   - [ ] Read the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) if you use Artificial Intelligence (AI).
   - [ ] I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute?
   - [x] Run a successful build using the default [Maven](https://maven.apache.org/) goal with `mvn`; that's `mvn` on the command line by itself.
   - [x] Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible, but it is a best practice.
   - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
   - [x] Each commit in the pull request should have a meaningful subject line and body.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4930240447

   @rootvector2 
   Please see the copilot comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4935486144

   @rootvector2 
   Please review `org.apache.commons.collections4.multiset.AbstractMapMultiSet.add(E, int)` for the same problem.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "rootvector2 (via GitHub)" <gi...@apache.org>.
rootvector2 commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4935701566

   Went over the merged `AbstractMapMultiSet.add(E, int)` once more: negative `occurrences` throws, the per-element count saturates through the applied delta so `mut.value` can't wrap, and the exact total sits in the `long` `size` field and clamps in `size()`. The multiset regression test covers add saturation plus removal while the true size is still above `Integer.MAX_VALUE`, so I don't see anything left in that path.
   
   On the wider int/long pass: the same summation overflow exists in `CompositeCollection.size()`, `CompositeSet.size()` and `CompositeMap.size()`, which sum member sizes into an int. `CompositeMap`'s javadoc even promises `Integer.MAX_VALUE` but the sum wraps negative, and it's cheap to hit now by compositing two bags at max size. `AbstractMultiValuedMap.size()` and `AbstractMultiSet.size()` (for subclasses that don't override it) accumulate the same way. I'll put together a follow-up PR for those.
   
   Thanks for merging this one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "rootvector2 (via GitHub)" <gi...@apache.org>.
rootvector2 commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4934382255

   `remove()` already rejects negatives on both sides: `AbstractMapBag.remove(Object, int)` returns false for `nCopies <= 0` and `AbstractMapMultiSet.remove(Object, int)` throws `IllegalArgumentException`, so counts cannot grow through it.
   
   The copilot comments are right about the stored clamp though: once `size` saturated, removals dropped `size()` below `Integer.MAX_VALUE` while the true cardinality was still above it. Reworked: `size` is now a `long` kept exact and clamped on return in `size()`. Per-element counts still saturate at `Integer.MAX_VALUE` since they are stored as `int`, and `size` only accrues the applied delta so it always equals the sum of counts. Both tests now cover removal while the true size is still above `Integer.MAX_VALUE`. Full suite, checkstyle, pmd and japicmp are green.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4930172278

   Hello @rootvector2 
   Isn't `remove()` broken in the same kind of way? `add()` is guarded for negative `nCopies` but `remove()` isn't.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory merged PR #705:
URL: https://github.com/apache/commons-collections/pull/705


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4935944420

   @rootvector2 Great, thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #705:
URL: https://github.com/apache/commons-collections/pull/705#discussion_r3555245849


##########
src/test/java/org/apache/commons/collections4/multiset/HashMultiSetTest.java:
##########
@@ -57,6 +58,17 @@ void testDeserializeRejectsNonPositiveCount() throws Exception {
         }
     }
 
+    @Test
+    void testAddClampsCountAndSizeToIntegerMaxValue() {
+        final HashMultiSet<String> set = new HashMultiSet<>();
+        set.add("X", Integer.MAX_VALUE);
+        set.add("X", 1);
+        assertEquals(Integer.MAX_VALUE, set.getCount("X"));
+        assertEquals(Integer.MAX_VALUE, set.size());
+        set.add("Y", Integer.MAX_VALUE);
+        assertEquals(Integer.MAX_VALUE, set.size());
+    }

Review Comment:
   This test asserts `size()` clamps on additional adds, but it doesn’t validate the `Collection.size()` contract when removing elements from a multiset whose true cardinality remains > `Integer.MAX_VALUE`. An implementation that clamps the stored `size` field (instead of clamping on return) will typically fail this scenario.



##########
src/main/java/org/apache/commons/collections4/bag/AbstractMapBag.java:
##########
@@ -206,12 +206,12 @@ public boolean add(final E object, final int nCopies) {
         modCount++;
         if (nCopies > 0) {
             final MutableInteger mut = map.get(object);
-            size += nCopies;
+            size = (int) Math.min((long) size + nCopies, Integer.MAX_VALUE);

Review Comment:
   Clamping the stored `size` field to `Integer.MAX_VALUE` during `add()` makes `size()` diverge from the sum of per-element counts once multiple elements are added (e.g., adding two keys with `Integer.MAX_VALUE - 1` each). Subsequent `remove(...)` / iterator-remove paths decrement `size` (see `size -= ...` and `size--` elsewhere in this class), which can drop `size()` below `Integer.MAX_VALUE` even while the true cardinality is still > `Integer.MAX_VALUE`, violating the `Collection.size()` contract and potentially producing incorrect/negative sizes.



##########
src/main/java/org/apache/commons/collections4/multiset/AbstractMapMultiSet.java:
##########
@@ -313,11 +313,11 @@ public int add(final E object, final int occurrences) {
 
         if (occurrences > 0) {
             modCount++;
-            size += occurrences;
+            size = (int) Math.min((long) size + occurrences, Integer.MAX_VALUE);

Review Comment:
   Clamping the stored `size` field here loses information once the multiset’s true cardinality exceeds `Integer.MAX_VALUE` across multiple distinct elements. Since `remove(...)` and iterator removals later do `size -= occurrences` / `size -= count`, `size()` will drop below `Integer.MAX_VALUE` even when the multiset still contains > `Integer.MAX_VALUE` total elements, which contradicts the `Collection.size()` contract and can make `size` inconsistent with the backing map’s counts.



##########
src/test/java/org/apache/commons/collections4/bag/HashBagTest.java:
##########
@@ -57,6 +58,18 @@ void testDeserializeRejectsNonPositiveCount() throws Exception {
         }
     }
 
+    @Test
+    void testAddClampsCountAndSizeToIntegerMaxValue() {
+        final HashBag<String> bag = new HashBag<>();
+        bag.add("X", Integer.MAX_VALUE - 1);
+        bag.add("Y", Integer.MAX_VALUE - 1);
+        assertEquals(Integer.MAX_VALUE, bag.size());
+        bag.add("X", 10);
+        assertEquals(Integer.MAX_VALUE, bag.getCount("X"));
+        assertEquals(Integer.MAX_VALUE, bag.size());
+        assertEquals(2, bag.uniqueSet().size());
+    }

Review Comment:
   This test covers clamping on additional adds, but it doesn’t exercise the required `Collection.size()` behavior after removals when the true cardinality is still > `Integer.MAX_VALUE` (e.g., after adding two elements with very large counts). Without that, an implementation that clamps the stored `size` field can still be incorrect under `remove(...)`/iterator removal while this test passes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Clamp overflow in AbstractMapBag and AbstractMapMultiSet add [commons-collections]

Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #705:
URL: https://github.com/apache/commons-collections/pull/705#issuecomment-4935418363

   Hello @rootvector2 
   Re-running CI. Would you please check the rest of Collections to see for other int/long mismatches?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@commons.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org