You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jeff Kim (Jira)" <ji...@apache.org> on 2023/02/08 22:32:00 UTC
[jira] [Created] (KAFKA-14694) RPCProducerIdManager should not wait for a new block
Jeff Kim created KAFKA-14694:
--------------------------------
Summary: RPCProducerIdManager should not wait for a new block
Key: KAFKA-14694
URL: https://issues.apache.org/jira/browse/KAFKA-14694
Project: Kafka
Issue Type: Bug
Reporter: Jeff Kim
Assignee: Jeff Kim
RPCProducerIdManager initiates an async request to the controller to grab a block of producer IDs and then blocks waiting for a response from the controller.
This is done in the request handler threads while holding a global lock. This means that if many producers are requesting producer IDs and the controller is slow to respond, many threads can get stuck waiting for the lock.
This may also be a deadlock concern under the following scenario:
if the controller has 1 request handler thread (1 chosen for simplicity) and receives an InitProducerId request, it may deadlock.
basically any time the controller has N InitProducerId requests where N >= # of request handler threads has the potential to deadlock.
consider this:
1. the request handler thread tries to handle an InitProducerId request to the controller by forwarding an AllocateProducerIds request.
2. the request handler thread then waits on the controller response (timed poll on nextProducerIdBlock)
3. the controller's request handler threads need to pick this request up, and handle it, but the controller's request handler threads are blocked waiting for the forwarded AllocateProducerIds response.
We should not block while waiting for a new block and instead return immediately to free the request handler threads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)