You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "thetumbled (via GitHub)" <gi...@apache.org> on 2023/09/02 01:33:07 UTC

[GitHub] [pulsar] thetumbled opened a new issue, #21109: [Bug] [broker] concurrent write and iterate with ConcurrentLongLongPairHashMap.

thetumbled opened a new issue, #21109:
URL: https://github.com/apache/pulsar/issues/21109

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Version
   
   all.
   
   ### Minimal reproduce step
   
   Concurrent write while iterate the map.
   ```
       @Test
       public void testConcurrentWriteAndIterate()  throws Throwable {
           ConcurrentLongLongPairHashMap map = ConcurrentLongLongPairHashMap.newBuilder()
                   .expectedItems(2)
                   .concurrencyLevel(1)
                   .autoShrink(true)
                   .mapIdleFactor(0.25f)
                   .build();
           assertEquals(map.capacity(), 4);
   
           ExecutorService executor = Executors.newCachedThreadPool();
           final int readThreads = 16;
           final int writeThreads = 1;
           final int n = 1_000;
           final int numKeys = 10;
           CyclicBarrier barrier = new CyclicBarrier(writeThreads + readThreads);
           Future<?> future = null;
           AtomicReference<Exception> ex = new AtomicReference<>();
   
           for (int i = 0; i < readThreads; i++) {
               executor.submit(() -> {
                   try {
                       barrier.await();
                   } catch (Exception e) {
                       throw new RuntimeException(e);
                   }
                   Set<Long> keys = new HashSet<>();
                   while (true) {
                       map.forEach((k1, k2, v1, v2) -> {
                           keys.add(k1);
                       });
                       for(long j = 1; j < numKeys; j++) {
                           if (keys.contains(j) && !keys.contains(j - 1)) {
                               ex.set(new Exception("Missing key: " + (j - 1) +" keys: " + keys));
                           }
                       }
                       keys.clear();
                   }
               });
           }
   
           future = executor.submit(() -> {
               try {
                   barrier.await();
               } catch (Exception e) {
                   throw new RuntimeException(e);
               }
   
               for (int i = 0; i < n; i++) {
                   // expand hashmap
                   for(int j = 0; j < numKeys; j++) {
                       assertTrue(map.put(j, j, j, j));
                   }
                   // shrink hashmap
                   map.clear();
                   assertEquals(map.capacity(), 4);
               }
           });
   
           future.get();
           System.out.println(ex.get());
           assertTrue(ex.get() == null);
           // shut down pool
           executor.shutdown();
       }
   ```
   
   ### What did you expect to see?
   
   I expect that, if i can see the pair `(n,n,n,n)` and `n>0`, i must can see the pair `(n-1,n-1,n-1,n-1)` because the pair `(n,n,n,n)` is inserted after the pair `(n-1,n-1,n-1,n-1)`.
   
   ### What did you see instead?
   
   I can't see the pair `(n-1,n-1,n-1,n-1)`.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on issue #21109: [Bug] [broker] concurrent write and iterate with ConcurrentLongLongPairHashMap.

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on issue #21109:
URL: https://github.com/apache/pulsar/issues/21109#issuecomment-1703635911

   This is the test code result:
   ```
   java.lang.Exception: Missing key: 4 keys: [0, 1, 2, 3, 5]
   
   java.lang.AssertionError
   	at org.junit.Assert.fail(Assert.java:86)
   	at org.junit.Assert.assertTrue(Assert.java:41)
   	at org.junit.Assert.assertTrue(Assert.java:52)
   	at org.apache.bookkeeper.util.collections.ConcurrentLongLongPairHashMapTest.testConcurrentWriteAndIterate(ConcurrentLongLongPairHashMapTest.java:244)
   ```
   You can recurring problems with the test code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] BewareMyPower closed issue #21109: [Bug] [broker] concurrent write and iterate with ConcurrentLongLongPairHashMap.

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower closed issue #21109: [Bug] [broker] concurrent write and iterate with ConcurrentLongLongPairHashMap.
URL: https://github.com/apache/pulsar/issues/21109


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org