You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "aliehsaeedii (via GitHub)" <gi...@apache.org> on 2026/10/09 12:03:40 UTC
[PR] KAFKA-21251: Label global stores with their real sub-topology id in --describe --topology [kafka]
aliehsaeedii opened a new pull request, #23762:
URL: https://github.com/apache/kafka/pull/23762
`TopologyDescriptionFormatter` labelled global stores starting at `subtopologies().size()`, assuming regular subtopologies are numbered `0..N-1`. Kafka Streams assigns subtopology and global store ids from a single contiguous counter (`InternalTopologyBuilder#makeNodeGroups`), so when a global store is defined first it gets id `0` and the regular subtopologies get `1..N`, and `kafka-streams-groups.sh --describe --topology` printed a global store label that collided with a real subtopology id.
`TopologyDescriptionConverter` sends both lists sorted by id, so the global store ids are exactly the ids not taken by a subtopology, in ascending order. Each global store is now labelled with the lowest unused id. This matches stock `TopologyDescription#toString()` output without any protocol change, and can never collide with a subtopology id. Non-numeric subtopology ids are ignored, since they cannot collide with a numeric label.
Testing: added `TopologyDescriptionFormatterTest` cases for a global store defined before the subtopologies, global stores interleaved with subtopologies, and non-numeric subtopology ids.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-21251: Label global stores with their real sub-topology id in --describe --topology [kafka]
Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky merged PR #23762:
URL: https://github.com/apache/kafka/pull/23762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-21251: Label global stores with their real sub-topology id in --describe --topology [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #23762:
URL: https://github.com/apache/kafka/pull/23762#discussion_r4313392564
##########
tools/src/main/java/org/apache/kafka/tools/streams/TopologyDescriptionFormatter.java:
##########
@@ -44,14 +46,32 @@ public static String format(final StreamsGroupTopologyDescription topology) {
sb.append(" ");
appendSubtopology(sb, subtopology);
}
- int globalStoreId = topology.subtopologies().size();
+ // Kafka Streams numbers subtopologies and global stores from a single counter, and both lists are sent
+ // sorted by id, so the global store ids are exactly the ids not taken by a subtopology, in ascending order.
Review Comment:
This depends on `TopologyDescriptionConverter.toWire` sorting both lists and on the topology description plugin keeping that order. Neither the protocol nor `StreamsGroupTopologyDescriptionPlugin` guarantees it. Could we say it is an assumption and point to where the sort happens?
Longer term it would be more robust to carry the global store id in `TopologyDescriptionGlobalStore`, but since these RPCs shipped in 4.4.0 that needs new versions and a KIP, so not for this PR.
##########
tools/src/main/java/org/apache/kafka/tools/streams/TopologyDescriptionFormatter.java:
##########
@@ -44,14 +46,32 @@ public static String format(final StreamsGroupTopologyDescription topology) {
sb.append(" ");
appendSubtopology(sb, subtopology);
}
- int globalStoreId = topology.subtopologies().size();
+ // Kafka Streams numbers subtopologies and global stores from a single counter, and both lists are sent
+ // sorted by id, so the global store ids are exactly the ids not taken by a subtopology, in ascending order.
+ final Set<Integer> subtopologyIds = numericIds(topology.subtopologies());
+ int globalStoreId = 0;
for (final GlobalStore globalStore : topology.globalStores()) {
Review Comment:
Now that the labels are right, the order still differs from `Topology#describe()`, which interleaves subtopologies and global stores by id (see `InternalTopologyBuilder.TopologyDescription#toString`). With a global store at 0 and subtopologies 1, 2, the app logs show the global store first, while here it comes last. It would be nice to match since the class javadoc says we mirror `describe()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org