[jira] [Created] (KAFKA-20058) Fix race condition on backoffDeadlineMs on RPCProducerIdManager causing premature retries

["sanghyeok An (Jira)" <ji...@apache.org>]

sanghyeok An created KAFKA-20058:
------------------------------------

             Summary: Fix race condition on backoffDeadlineMs on RPCProducerIdManager causing premature retries
                 Key: KAFKA-20058
                 URL: https://issues.apache.org/jira/browse/KAFKA-20058
             Project: Kafka
          Issue Type: Bug
            Reporter: sanghyeok An
            Assignee: sanghyeok An


While investigating a flaky failure in ProducerIdManagerTest.testRetryBackoffOnNoResponse, I found a race in RPCProducerIdManager.maybeRequestNextBlock().

maybeRequestNextBlock() currently does:
 * sendRequest()
 * backoffDeadlineMs.set(NO_RETRY) (unconditional)


On the response path, handleUnsuccessfulResponse() does:
 * backoffDeadlineMs.set(now + RETRY_BACKOFF_MS)
 * requestInFlight.set(false)

 

Because sendRequest() is asynchronous, the unconditional backoffDeadlineMs reset can run after handleUnsuccessfulResponse(), overwriting the newly-set backoff deadline. If backoffDeadlineMs ends up as NO_RETRY, a subsequent generateProducerId() call can re-send immediately, which may prefill nextProducerIdBlock earlier than expected and lead to test flakiness (and potentially unnecessary controller traffic).

 

In production, this race is less likely because the request/response path typically has higher latency than in the unit test (which simulates the controller response on a local executor). However, the code still has a correctness window where a newly set backoff deadline can be clobbered by an unconditional reset. Using compareAndSet to conditionally reset backoff preserves the intended behavior, avoids overwriting newer backoff values, and should have negligible performance impact (CAS is only executed on the request path, and contention should be rare). This also eliminates the observed test flakiness.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)