You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "krishvishal (via GitHub)" <gi...@apache.org> on 2026/10/26 07:20:13 UTC
[PR] feat(gateways): report Kafka group members' client id and host [iggy]
krishvishal opened a new pull request, #4651:
URL: https://github.com/apache/iggy/pull/4651
## Which issue does this PR address?
Closes #4511
## Rationale
`kafka-consumer-groups.sh --describe --members` showed an empty CLIENT-ID and HOST for every member of a gateway group. Operators use these two columns to find the process behind a consumer. Kafka fills both from the `JoinGroup` request: the client id from the request header, and the host from the address of the connection. The gateway handlers saw neither.
## What changed?
Each connection now keeps the first client id that a request header names, cut to 1,024 bytes. It also keeps the peer IP address in Kafka's `/10.0.0.7` form. A member takes both from the connection that it last joined through, and DescribeGroups reports them. A new member id now starts with the client id, as in Kafka, where it used to start with `member`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(gateways): report Kafka group members' client id and host [iggy]
Posted by "krishvishal (via GitHub)" <gi...@apache.org>.
krishvishal merged PR #4651:
URL: https://github.com/apache/iggy/pull/4651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(gateways): report Kafka group members' client id and host [iggy]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #4651:
URL: https://github.com/apache/iggy/pull/4651#issuecomment-6411079896
## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/4651?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 81.01%. Comparing base ([`6ce2f92`](https://app.codecov.io/gh/apache/iggy/commit/6ce2f92a9f9650242caf09997e7ec6c2b4c763df?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`3d2d7ae`](https://app.codecov.io/gh/apache/iggy/commit/3d2d7ae33ad1358e77e05cf10123460b92d7b0b1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (6ce2f92) and HEAD (3d2d7ae). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (6ce2f92) | HEAD (3d2d7ae) |
>|------|------|------|
>|rust|4|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #4651 +/- ##
============================================
- Coverage 90.77% 81.01% -9.77%
Complexity 2989 2989
============================================
Files 1459 1459
Lines 348165 348162 -3
Branches 292747 292744 -3
============================================
- Hits 316042 282047 -33995
- Misses 25485 59457 +33972
- Partials 6638 6658 +20
```
| [Components](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `80.28% <ø> (-11.81%)` | :arrow_down: |
| [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.80% <ø> (ø)` | |
| [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.74% <ø> (ø)` | |
| [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.69% <ø> (ø)` | |
| [PHP SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.67% <ø> (ø)` | |
| [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `96.44% <ø> (ø)` | |
| [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.97% <ø> (ø)` | |
| [C++ SDK](https://app.codecov.io/gh/apache/iggy/pull/4651/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.51% <ø> (ø)` | |
[see 477 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/4651/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(gateways): report Kafka group members' client id and host [iggy]
Posted by "krishvishal (via GitHub)" <gi...@apache.org>.
krishvishal commented on PR #4651:
URL: https://github.com/apache/iggy/pull/4651#issuecomment-6411068039
/skill team-review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(gateways): report Kafka group members' client id and host [iggy]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #4651:
URL: https://github.com/apache/iggy/pull/4651#discussion_r4375136285
##########
gateways/kafka/tests/kafka_client_e2e_tests.rs:
##########
@@ -1553,6 +1553,80 @@ fn consume_as_static_member(gateway: SocketAddr, count: usize) -> String {
.expect_ran("java console consumer as a static member")
}
+/// `kafka-consumer-groups --describe --members` shows the client id and host of each member, as
+/// Kafka does (#4511). A static member stays in its group for its whole session after its
+/// consumer exits, so the tool still finds it.
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn given_a_java_consumer_when_describing_the_members_should_show_its_client_id_and_host() {
+ let Some((_server, gateway)) = bridged_stack().await else {
+ return;
+ };
+ produce_with_kcat(gateway, &RECORDS);
+ let consumed = run_client(
Review Comment:
simplification: The new test repeats the console-consumer argument list from `consume_as_static_member()` (line 1525), with only the group, instance id and client id changed. Simpler: pass those three to the helper and call it here. Saves: about 20 lines.
##########
gateways/kafka/src/group/state.rs:
##########
@@ -2032,12 +2054,15 @@ fn park_or_respond(group: &mut GroupState, member_id: &StrBytes, now: Instant) -
}
}
-/// Kafka shapes a member id as `clientId-UUID`. Handlers never see the header's `client_id`, so
-/// the static-membership id stands in for it when present. The UUID matters on its own: a
-/// counter would repeat across a gateway restart and turn a stale client's `UNKNOWN_MEMBER_ID`
-/// into a silent identity theft.
-fn generate_member_id(group_instance_id: Option<&StrBytes>) -> StrBytes {
- let prefix = group_instance_id.map_or(DEFAULT_MEMBER_PREFIX, StrBytes::as_str);
+/// Kafka shapes a member id as `groupInstanceId-UUID` for a static member, and as `clientId-UUID`
+/// for any other. The UUID matters on its own: a counter would repeat across a gateway restart and
+/// turn a stale client's `UNKNOWN_MEMBER_ID` into a silent identity theft.
+fn generate_member_id(request: &JoinRequest) -> StrBytes {
+ let prefix = match &request.group_instance_id {
+ Some(instance_id) => instance_id.as_str(),
+ None if !request.client_id.is_empty() => request.client_id.as_str(),
Review Comment:
warning: `generate_member_id()` copies `request.client_id` into the member id without the `MAX_CLIENT_ID_BYTES` cut, which only `ConnectionState::note_client_id()` applies on the gateway path. A caller of `GroupCoordinator::join()` can then store a client id or member id that the legacy string encoder refuses to encode, and the connection closes. Reject a `client_id` above `MAX_CLIENT_ID_BYTES` in `join_request_error()`.
##########
gateways/kafka/tests/kafka_client_e2e_tests.rs:
##########
@@ -1553,6 +1553,80 @@ fn consume_as_static_member(gateway: SocketAddr, count: usize) -> String {
.expect_ran("java console consumer as a static member")
}
+/// `kafka-consumer-groups --describe --members` shows the client id and host of each member, as
+/// Kafka does (#4511). A static member stays in its group for its whole session after its
Review Comment:
nit: This comment cites `#4511`, but `docs/CONSUMER_GROUPS.md` attributes that issue to `DeleteGroups` and `OffsetDelete`. Cite the issue that this feature tracks, or drop the number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org