You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Sean Quah (Jira)" <ji...@apache.org> on 2025/04/16 11:55:00 UTC
[jira] [Created] (KAFKA-19160) Fetching stable offsets is very slow with a large number of partition and open transactions
Sean Quah created KAFKA-19160:
---------------------------------
Summary: Fetching stable offsets is very slow with a large number of partition and open transactions
Key: KAFKA-19160
URL: https://issues.apache.org/jira/browse/KAFKA-19160
Project: Kafka
Issue Type: Bug
Components: group-coordinator
Reporter: Sean Quah
Assignee: Sean Quah
When fetching stable offsets in the group coordinator, we iterate over all requested partitions and for each partition, check if any ongoing transactions have updated the group's offset for that partition. This is O(#partitions * #open_transactions).
For a large topic with 4,000 partitions and a correspondingly large group, we might have up to 4,000 open transactions. Fetching stable offsets for all partitions in this case can take around 350 ms.
It looks like the old coordinator also has this issue but it's being removed soon anyway.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)