[GitHub] guillaumepitel commented on issue #2431: Unable to consume messages, need to re-subscribe client

[GitBox <gi...@apache.org>]

guillaumepitel commented on issue #2431: Unable to consume messages, need to re-subscribe client
URL: https://github.com/apache/incubator-pulsar/issues/2431#issuecomment-420235929
 
 
   Hi. I don't know if it's a similar issue, but here is what just happened on our Pulsar cluster (small three nodes clusters) : 
   
   One of the 3 broker suddenly stopped sending messages, its total MSG/S OUT was zero while MSG/I IN were still at ~100k/s. So it acted as if all the consumers on this particular broker were simultaneously disconnected. The odd thing is that **no exception were thrown, and no error or warning were issued**, either on the client side (the log level for pulsar is WARN) or on the brokers. The topics are persistent, failover non shared. All machines (clients and brokers) were under light load (0% wait, 20% CPU, low network BW).
   
   After a client restart, the consumers started receiving messages again.
   
   Is there anything we can do to detect such disconnection ? I thought pulsar clients & consumers were supposed to reconnect automatically, but maybe we need to listen to ConsumerEvents to detect disabled consumers ?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services