[jira] [Commented] (FLINK-38287) Kafka 4.1.0 Queue Semantics support in Flink Connector Kafka

["Shekhar Prasad Rajak (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-38287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18071083#comment-18071083 ] 

Shekhar Prasad Rajak commented on FLINK-38287:
----------------------------------------------

I have updated the FLIP https://docs.google.com/document/d/1_wf3GwqWV3yiD-0oG2d6jwN8bfdnbk-Kz13btva_QAc/edit?pli=1&tab=t.0 
Also please assign this ticket to me, I am not able to assign to myself. 

> Kafka 4.1.0 Queue Semantics support in Flink Connector Kafka
> ------------------------------------------------------------
>
>                 Key: FLINK-38287
>                 URL: https://issues.apache.org/jira/browse/FLINK-38287
>             Project: Flink
>          Issue Type: New Feature
>          Components: API / Core
>    Affects Versions: kafka-4.1.0
>            Reporter: Shekhar Prasad Rajak
>            Priority: Major
>         Attachments: Screenshot 2025-09-02 at 10.59.05 AM.png, Screenshot 2025-09-02 at 10.59.55 AM.png
>
>
> KIP-932 introduces share groups as a new consumption model that provides queue semantics.
> This directly addresses *use cases* where:
> 1. Multiple consumers need to process items efficiently in parallel from a single/multiple topic(s).
> 2. Messages  need explicit acknowledgment/release (to avoid reprocessing or allow retries).
> h4.   *Operational Benefits*
>   - {*}Higher Throughput{*}: ShareGroupHeartbeat helps in Queue-like workloads, maximum throughput scenarios. Share groups distribute messages at the record level, not partition level, so multiple readers can consume from the same topic with Kafka coordinating message distribution.
>   - {*}Better Availability and Flexible Scaling{*}: consumers assignment logic is simpler in server side and rebalancing frequency is minimised.
>   - *Reduce Operational overhead at consumer side:* Broker manages partition state via SharePartitionManager rather than client-side coordination. Eliminates the need for the complex rebalancing protocol that consumer groups required and reduced latency for consumer membership changes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)