[GitHub] [pulsar] devinbost commented on issue #10813: Consumer stuck while reading from partitioned topic using Shared subscription

[GitBox <gi...@apache.org>]

devinbost commented on issue #10813:
URL: https://github.com/apache/pulsar/issues/10813#issuecomment-854130449


   @eolivelli @lhotari 
   
   Negative permits just means the dispatcher is waiting for permits from the consumer, which implies it's waiting for the consumer to complete processing of the messages. I'd like to know if the consumer is waiting for ack's from the broker. 
   If you get a thread dump of the consumer when it's frozen, is it waiting for the producer semaphor? Look for something like this in the consumer thread dump:
   
   ```
   "myTenant/myNamespace/function-filter-0" prio=5 tid=32 WAITING
       at sun.misc.Unsafe.park(Native Method)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
          local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#40
          local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#41
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
          local variable: java.util.concurrent.Semaphore$FairSync#1
       at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
       at org.apache.pulsar.client.impl.ProducerImpl.canEnqueueRequest(ProducerImpl.java:748)
       at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:391)
          local variable: org.apache.pulsar.client.impl.ProducerImpl#1
          local variable: org.apache.pulsar.client.impl.ProducerImpl$1#1
          local variable: io.netty.buffer.UnpooledHeapByteBuf#17
          local variable: org.apache.pulsar.common.api.proto.MessageMetadata#83
          local variable: org.apache.pulsar.client.impl.MessageImpl#82
       at org.apache.pulsar.client.impl.ProducerImpl.internalSendAsync(ProducerImpl.java:290)
          local variable: java.util.concurrent.CompletableFuture#33
       at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.sendAsync(TypedMessageBuilderImpl.java:103)
   ```
   
   Another thing to look for is in a consumer heap dump. I'd like to know if `ProducerImpl.pendingMessages` has messages in it (and how many.) If it's empty, I'd like to know if `ConsumerImpl.incomingMessages` has messages in it or not. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org