[jira] [Commented] (FLINK-33224) Flink Batch Job does not complete when using Kafka topics with empty partitions

["Sylwester Lachiewicz (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18113948#comment-18113948 ] 

Sylwester Lachiewicz commented on FLINK-33224:
----------------------------------------------

Triage finding (2026-09-10): the exception in the original 1.14.6 report no longer exists on any maintained connector line; the bounded path in {{KafkaPartitionSplitReader}} removes empty splits up front ({{removeEmptySplits}}) and {{testAssignEmptySplit}} is green on main. The remaining symptom, the job staying up because the final MAX_WATERMARK is never emitted for a source that only saw empty partitions, is the runtime issue [FLINK-40499|https://issues.apache.org/jira/browse/FLINK-40499] that Martijn linked, with the fix pending in [apache/flink PR #29037|https://github.com/apache/flink/pull/29037].

Related reports checked at the same time: [FLINK-33001|https://issues.apache.org/jira/browse/FLINK-33001] was fixed by [06789ec7|https://github.com/apache/flink-connector-kafka/commit/06789ec7b86a3b2edae1dc7f9f29d672ebd3610b] in 3.1.0, and [FLINK-35565|https://issues.apache.org/jira/browse/FLINK-35565] by [FLINK-34470|https://issues.apache.org/jira/browse/FLINK-34470] in 3.3.0; a new unit test for an out-of-range starting offset reset onto the stopping offset stays green on main.

Recommendation: nothing to change in the connector. Close this once FLINK-40499 ships, or move it to the runtime component now.

_This comment was created with AI assistance._

> Flink  Batch Job does not complete when using Kafka topics with empty partitions
> --------------------------------------------------------------------------------
>
>                 Key: FLINK-33224
>                 URL: https://issues.apache.org/jira/browse/FLINK-33224
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kafka
>    Affects Versions: 1.14.6
>            Reporter: Kanthi Vaidya
>            Priority: Major
>
> Kafka Source Connector no longer throws an exception when running a flink job in batch mode when using Kafka topics with empty partitions. However, flink does not emit the final watermark of Long.MAX_VALUE even if all the messages have been read. Note that we are using bounded sources, since that is a requirement for batch mode. It appears that the Kafka consumer continues to run even though the source is bounded.
> Same is the case with an empty topic. We do have optional sources, where data need not be present all the time. However if we mark the source as bounded, the Kafka connector should recognize empty partitions and mark it as completed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)