[jira] [Comment Edited] (KAFKA-20804) Reduce lock contention in ProducerMetadata#add

["Xi Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097344#comment-18097344 ] 

Xi Wang edited comment on KAFKA-20804 at 7/19/26 1:37 AM:
----------------------------------------------------------

[~faheedy1230] Hi Fahid, sorry it's my first time creating the kafka Jira ticket and didn't make it clear. I've already created a PR [https://github.com/apache/kafka/pull/22810,] just added the link to this jira, can you help review?


was (Author: JIRAUSER313960):
[~faheedy1230] Hi Fahid, sorry it's my first time creating the kafka Jira ticket and didn't make it clear. I've already created a PR [https://github.com/apache/kafka/pull/22810,] can you help review?

> Reduce lock contention in ProducerMetadata#add
> ----------------------------------------------
>
>                 Key: KAFKA-20804
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20804
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients
>            Reporter: Xi Wang
>            Assignee: Fahid Abdullah
>            Priority: Critical
>
> add() is called on every send() via KafkaProducer.waitOnMetadata, and the synchronized method acquiring the instance lock causing synchronized contention which impacts the performance.
> Switch the topics map to a ConcurrentHashMap and refresh known topics via a lock-free replace(), which only writes if the topic is still present. For a topic seen for the first time (or one concurrently evicted by retainTopic()), fall back to a synchronized block so the map insert and newTopics bookkeeping happen atomically with retainTopic() - otherwise retainTopic() could expire-and-remove the topic in the gap between the two, leaving it recorded in newTopics without the corresponding map entry, or silently re-inserting an evicted topic without the newTopics bookkeeping needed to trigger an immediate refresh.
> retainTopic() switches to a conditional remove(topic, expireMs) so a concurrent refresh of an existing topic in add() can't be undone by an eviction decision based on the expiry value read just before the refresh landed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)