Re: [I] [Bug] Client with shared subscription is blocked [pulsar]

["michalcukierman (via GitHub)" <gi...@apache.org>]

michalcukierman commented on issue #21104:
URL: https://github.com/apache/pulsar/issues/21104#issuecomment-1743609860

   It looks like I was able to reproduce the issue in the first run today.
   - I've created SmallRye project (one producer to produce 1 mln messages, one consumer to read it using shared subscription)
   - The cluster is created on GCP using Pulsar operator from SN (great tool!)
   
   The code is here:
   https://github.com/michalcukierman/pulsar-21104
   
   In general it's very much like in the bug description. Produce 1 mln messages of 30kb:
   ```java
     @Outgoing("requests-out")
     public Multi<String> produce() {
       return Multi.createBy().repeating()
           .uni(() -> Uni
               .createFrom()
               .item(() -> RandomStringUtils.randomAlphabetic(30_000))
               .onItem()
               .invoke(() -> System.out.println("+ Produced: " + outCount.incrementAndGet()))
           )
           .atMost(1_000_000);
     }
   ```
   
   Read it using shared client and write to another topic:
   ```java
   @ApplicationScoped
   public class Processor {
   
     private final AtomicLong inCount = new AtomicLong(0);
     @Incoming("requests-in")
     @Outgoing("dump-out")
     @Blocking
     PulsarOutgoingMessage<String> process(PulsarIncomingMessage<String> in) {
       System.out.println(" - Processed: " + inCount.incrementAndGet());
       return PulsarOutgoingMessage.from(in);
     }
   }
   ```
   
   The settings of the client are:
   ```
   pulsar.client.serviceUrl=pulsar://brokers-broker:6650
   
   mp.messaging.incoming.requests-in.subscriptionType=Shared
   mp.messaging.incoming.requests-in.numIoThreads=4
   mp.messaging.incoming.requests-in.subscriptionName=request-shared-subscription
   mp.messaging.incoming.requests-in.ackTimeoutMillis=5000
   mp.messaging.incoming.requests-in.subscriptionInitialPosition=Earliest
   mp.messaging.incoming.requests-in.receiverQueueSize=8
   mp.messaging.incoming.requests-in.topic=persistent://public/default/requests_4
   mp.messaging.incoming.requests-in.connector=smallrye-pulsar
   
   mp.messaging.outgoing.dump-out.topic=persistent://public/default/dump
   mp.messaging.outgoing.dump-out.connector=smallrye-pulsar
   mp.messaging.outgoing.dump-out.blockIfQueueFull=true
   mp.messaging.outgoing.dump-out.maxPendingMessages=8
   mp.messaging.outgoing.dump-out.maxPendingMessagesAcrossPartitions=12
   ```
   
   The retention of the topic `requests` is set using Pulsar Admin in Java to -1 -1.
   
   During two runs the consumer get stucked:
   <img width="1013" alt="Screenshot 2023-10-02 at 21 12 42" src="https://github.com/apache/pulsar/assets/4356553/b31bb39c-1491-4852-8eb6-a975155f31b2">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org