[jira] [Commented] (KAFKA-15178) Poor performance of ConsumerCoordinator with many TopicPartitions

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-15178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742151#comment-17742151 ] 

A. Sophie Blee-Goldman commented on KAFKA-15178:
------------------------------------------------

Nice catch!

> Poor performance of ConsumerCoordinator with many TopicPartitions
> -----------------------------------------------------------------
>
>                 Key: KAFKA-15178
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15178
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 3.5.0
>            Reporter: Nicholas Telford
>            Assignee: Nicholas Telford
>            Priority: Minor
>              Labels: easyfix, patch-available
>         Attachments: pollPhase.png
>
>
> Doing some profiling of my Kafka Streams application, I noticed that the {{pollPhase}} suffers from a minor performance issue.
> See the pink tree on the left of the flame graph below.  !pollPhase.png|width=1028,height=308!
> {{ConsumerCoordinator.poll}} calls {{{}rejoinNeededOrPending{}}}, which checks the current {{metadataSnapshot}} against the {{{}assignmentSnapshot{}}}. This comparison is a deep-equality check, and if there's a large number of topic-partitions being consumed by the application, then this comparison can perform poorly.
> I suspect this can be trivially addressed with a {{boolean}} flag that indicates when the {{metadataSnapshot}} has been updated (or is "dirty"), and actually needs to be checked, since most of the time it should be identical to {{{}assignmentSnapshot{}}}.
> I plan to raise a PR with this optimization to address this issue.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)