[I] feat(server): read consumer group offsets over the HTTP API [iggy]

["gmr (via GitHub)" <gi...@apache.org>]

gmr opened a new issue, #4279:
URL: https://github.com/apache/iggy/issues/4279

   ### Description
   
   The HTTP API cannot read a consumer group's stored offset. `GET /streams/{stream_id}/topics/{topic_id}/consumer-offsets` always resolves a regular consumer, because `Consumer.kind` is `#[serde(skip)]` (`core/common/src/types/consumer/consumer_kind.rs:34`). A `kind` query parameter is ignored, and `consumer_offset_wire_request` always sends kind `Consumer` (`core/server/src/http/wire.rs:128`, and see the comment at `:146`).
   
   The server already supports this read. `resolve_consumer_offset_request` (`core/server/src/dispatch/partition.rs:1103`) handles `KIND_CONSUMER_GROUP`, and any client can read a group offset without being a member. Only the TCP path can reach it (for example, `iggy consumer-offset get <group> <stream> <topic> 0 --kind consumer-group`).
   
   `GET .../consumer-groups/{group_id}` returns members and their partitions, but no offsets.
   
   This matters for monitoring. To compute consumer lag (partition `current_offset` minus the group's stored offset) from an HTTP-only tool such as a dashboard, a health check, or an admin UI, you must add a TCP client only for this read. In #4273, a stuck group showed healthy connector status for days. Group lag over HTTP would have shown the problem at once.
   
   ### Affected area / component
   
   Server (HTTP API); common (`GetConsumerOffset` / `Consumer` HTTP types); SDK HTTP client.
   
   ### Proposed solution
   
   Either of these, or both:
   
   1. Accept a `kind` query parameter on `GET .../consumer-offsets` (`consumer` by default, or `consumer_group`) and pass it through to the wire request. For example: `GET /streams/s/topics/t/consumer-offsets?consumer_id=my-group&kind=consumer_group&partition_id=0`.
   2. Include a per-partition `stored_offset` in the `GET .../consumer-groups/{group_id}` response, so that one call gives the members, the assignments and the group's progress.
   
   Option 1 keeps the existing default, so current HTTP clients see no change.
   
   ### Alternatives considered
   
   - Use the TCP protocol or the CLI for this one read. This works, but an HTTP-only client must then keep a second transport and credentials.
   - Compute lag from connector runtime metrics (`iggy_connector_messages_consumed_total`). This covers only connector consumers, and it cannot show a group's committed position.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] feat(server): read consumer group offsets over the HTTP API [iggy]

["hubcio (via GitHub)" <gi...@apache.org>]

hubcio commented on issue #4279:
URL: https://github.com/apache/iggy/issues/4279#issuecomment-5802120172

   sure, take it @haubur 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] feat(server): read consumer group offsets over the HTTP API [iggy]

["hubcio (via GitHub)" <gi...@apache.org>]

hubcio closed issue #4279: feat(server): read consumer group offsets over the HTTP API
URL: https://github.com/apache/iggy/issues/4279


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] feat(server): read consumer group offsets over the HTTP API [iggy]

["haubur (via GitHub)" <gi...@apache.org>]

haubur commented on issue #4279:
URL: https://github.com/apache/iggy/issues/4279#issuecomment-5800574283

   @hubcio I can take this as I am anyways looking at the HTTP API in another issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org