You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "iamkarasik (via GitHub)" <gi...@apache.org> on 2026/09/07 02:07:32 UTC

[PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

iamkarasik opened a new pull request, #23391:
URL: https://github.com/apache/kafka/pull/23391

   `SkimpyOffsetMap` supports configurable hash algorithms, but the tests only covered the default (`MD5`). 
   - Parameterize the tests to also run against `SHA-1`, `SHA-256`, and `SHA-512`.
   - Add tests to check entry size based on each algorithm (`bytesPerEntry` is public already)
   - Add test for negative case to fail on unknown-algorithm.
   
   No production changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on PR #23391:
URL: https://github.com/apache/kafka/pull/23391#issuecomment-5611827155

   Kafka doesn't seem to have a specific convention around avoiding `public` in tests, so I don't think this change brings much practical benefit. Since it's also unrelated to the main purpose of this patch, I'd prefer to leave this part unchanged for now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st merged PR #23391:
URL: https://github.com/apache/kafka/pull/23391


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "iamkarasik (via GitHub)" <gi...@apache.org>.
iamkarasik commented on PR #23391:
URL: https://github.com/apache/kafka/pull/23391#issuecomment-5587026929

   @m1a2st, thanks for the review!
   
   I reverted the visibility change and added the remaining hash algorithms.
   
   For context, JUnit 5 doesn't require test classes or methods to be public, so I reduced visibility according to the conventions. I have some other test-related cleanups in this file that I think should probably be done in a MINOR PR.
   
   Let me know your thoughts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #23391:
URL: https://github.com/apache/kafka/pull/23391#issuecomment-5569741146

   > could you approve the CI run when you get a chance?
   
   @m1a2st thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "iamkarasik (via GitHub)" <gi...@apache.org>.
iamkarasik commented on PR #23391:
URL: https://github.com/apache/kafka/pull/23391#issuecomment-5564373594

   @chia7712 could you approve the CI run when you get a chance?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20745: extend OffsetMapTest to verify different hash algorithms [kafka]

Posted by "m1a2st (via GitHub)" <gi...@apache.org>.
m1a2st commented on code in PR #23391:
URL: https://github.com/apache/kafka/pull/23391#discussion_r3956439668


##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -18,30 +18,45 @@
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
-import org.junit.jupiter.params.provider.ValueSource;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
 
 import java.nio.ByteBuffer;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
 import java.util.stream.IntStream;
+import java.util.stream.Stream;
 
 import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
 
-public class OffsetMapTest {
+class OffsetMapTest {

Review Comment:
   Why we need this change?



##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -18,30 +18,45 @@
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
-import org.junit.jupiter.params.provider.ValueSource;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
 
 import java.nio.ByteBuffer;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
 import java.util.stream.IntStream;
+import java.util.stream.Stream;
 
 import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
 
-public class OffsetMapTest {
+class OffsetMapTest {
 
     private static final int MEMORY_SIZE = 4096;
 
+    static Stream<String> hashAlgorithms() {
+        return Stream.of("MD5", "SHA-1", "SHA-256", "SHA-512");

Review Comment:
   `SkimpyOffsetMap` documents support for MD2, MD5, SHA-1, SHA-256, SHA-384, and SHA-512, but here only covers four of them. Should we add MD2 and SHA-384 as well?



##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -18,30 +18,45 @@
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
-import org.junit.jupiter.params.provider.ValueSource;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
 
 import java.nio.ByteBuffer;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
 import java.util.stream.IntStream;
+import java.util.stream.Stream;
 
 import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
 
-public class OffsetMapTest {
+class OffsetMapTest {
 
     private static final int MEMORY_SIZE = 4096;
 
+    static Stream<String> hashAlgorithms() {
+        return Stream.of("MD5", "SHA-1", "SHA-256", "SHA-512");
+    }
+
+    static Stream<Arguments> algorithmsAndItems() {
+        return hashAlgorithms().flatMap(algorithm ->
+            IntStream.of(10, 100, 1000, 5000).mapToObj(items -> Arguments.of(algorithm, items)));
+    }
+
     @ParameterizedTest
-    @ValueSource(ints = {10, 100, 1000, 5000})
-    public void testBasicValidation(int items) throws Exception {
-        SkimpyOffsetMap map = new SkimpyOffsetMap(items * 48);
+    @MethodSource("algorithmsAndItems")
+    void testBasicValidation(String algorithm, int items) throws Exception {
+        int bytesPerEntry = MessageDigest.getInstance(algorithm).getDigestLength() + 8;
+        SkimpyOffsetMap map = new SkimpyOffsetMap(items * bytesPerEntry * 2, algorithm);
         IntStream.range(0, items).forEach(i -> assertDoesNotThrow(() -> map.put(key(i), i)));
         for (int i = 0; i < items; i++) {
             assertEquals(i, map.get(key(i)));
         }
     }
 
     @Test
-    public void testClear() throws Exception {
+    void testClear() throws Exception {

Review Comment:
   ditto



##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -89,9 +105,9 @@ public void testLatestOffset() throws Exception {
         }
         assertEquals(i - 1, map.latestOffset());
     }
-    
+
     @Test
-    public void testUtilization() throws Exception {
+    void testUtilization() throws Exception {

Review Comment:
   ditto



##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -80,7 +96,7 @@ public void testUpdateLatestOffset() throws Exception {
     }
 
     @Test
-    public void testLatestOffset() throws Exception {
+    void testLatestOffset() throws Exception {

Review Comment:
   ditto



##########
storage/src/test/java/org/apache/kafka/storage/internals/log/OffsetMapTest.java:
##########
@@ -66,7 +82,7 @@ public void testGetWhenFull() throws Exception {
     }
 
     @Test
-    public void testUpdateLatestOffset() throws Exception {

Review Comment:
   ditto



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org