[PR] KAFKA-20597: Fix prefixScan overflow for prefixes with no upper bound in in-memory state stores [kafka]

["edenfunf (via GitHub)" <gi...@apache.org>]

edenfunf opened a new pull request, #22789:
URL: https://github.com/apache/kafka/pull/22789

   ## Problem
   
   When a prefix has no lexicographically larger successor (e.g. it is all `0xFF` bytes), `prefixScan` computes the exclusive upper bound of its range by incrementing the prefix, which overflows. `RocksDBStore` already handles this by treating the overflow as an unbounded range, but `InMemoryKeyValueStore`, `MemoryNavigableLRUCache` and `CachingKeyValueStore` called `ByteUtils.increment()` directly and threw `IndexOutOfBoundsException`.
   
   ## Fix
   
   - Centralize the overflow-safe increment as `ByteUtils.incrementWithoutOverflow()`, which returns `null` on overflow. The RocksDB code paths that previously defined a private copy now reuse it.
   - The three in-memory stores treat a `null` upper bound as "scan to the end of the keyspace" (`tailMap`), matching the behaviour `RocksDBStore` already had.
   - `ByteUtils.increment()` is left unchanged: `RocksDBStore.deleteRange()` intentionally relies on it throwing, since RocksDB's `deleteRange()` has no null-upper-bound form.
   
   ## Testing
   
   - Added `prefixScanShouldReturnMatchesWhenPrefixHasNoUpperBound` to `AbstractKeyValueStoreTest`, so it runs against every `KeyValueStore` implementation. It uses `IntegerSerializer`'s encoding of `-1` (`0xFFFFFFFF`) to reproduce the overflow. Verified the test fails on the unpatched in-memory stores and passes with the fix, while the RocksDB-backed stores pass in both cases.
   - Added unit tests for `ByteUtils.incrementWithoutOverflow()`.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-20597: Fix prefixScan overflow for prefixes with no upper bound in in-memory state stores [kafka]

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] commented on PR #22789:
URL: https://github.com/apache/kafka/pull/22789#issuecomment-4998860440

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org