[jira] [Created] (FLINK-38287) Kafka 4.1.0 Queue Semantics support in Flink Connector Kafka

["Shekhar Prasad Rajak (Jira)" <ji...@apache.org>]

Shekhar Prasad Rajak created FLINK-38287:
--------------------------------------------

             Summary: Kafka 4.1.0 Queue Semantics support in Flink Connector Kafka
                 Key: FLINK-38287
                 URL: https://issues.apache.org/jira/browse/FLINK-38287
             Project: Flink
          Issue Type: New Feature
          Components: API / Core
    Affects Versions: kafka-4.1.0, flink-2.0.0
            Reporter: Shekhar Prasad Rajak


KIP-932 introduces share groups as a new consumption model that provides queue semantics.

This directly addresses *use cases* where:

1. Multiple consumers need to process items efficiently in parallel from a single/multiple topic(s).

2. Messages  need explicit acknowledgment/release (to avoid reprocessing or allow retries).
h4.   *Operational Benefits*

  - {*}Higher Throughput{*}: ShareGroupHeartbeat helps in Queue-like workloads, maximum throughput scenarios. Share groups distribute messages at the record level, not partition level, so multiple readers can consume from the same topic with Kafka coordinating message distribution.
  - {*}Better Availability and Flexible Scaling{*}: consumers assignment logic is simpler in server side and rebalancing frequency is minimised.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)