[jira] [Created] (FLINK-40011) CommonExecSinkITCase.testStreamRecordTimestampInserterSinkRuntimeProvider is flaky: test source can finish before NoMoreSplitsEvent is delivered

["Martijn Visser (Jira)" <ji...@apache.org>]

Martijn Visser created FLINK-40011:
--------------------------------------

             Summary: CommonExecSinkITCase.testStreamRecordTimestampInserterSinkRuntimeProvider is flaky: test source can finish before NoMoreSplitsEvent is delivered
                 Key: FLINK-40011
                 URL: https://issues.apache.org/jira/browse/FLINK-40011
             Project: Flink
          Issue Type: Bug
          Components: Table SQL / Planner, Tests
    Affects Versions: 2.4.0
            Reporter: Martijn Visser
            Assignee: Martijn Visser


testStreamRecordTimestampInserterSinkRuntimeProvider fails intermittently when the
job fails instead of finishing.

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=76431&view=results (leg: test_ci table)

{code}
org.apache.flink.table.api.TableException: Failed to wait job finish
  at CommonExecSinkITCase.testStreamRecordTimestampInserterSinkRuntimeProvider(CommonExecSinkITCase.java:127)
Caused by: JobExecutionException: Job execution failed.
Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
Caused by: org.apache.flink.util.FlinkException: An OperatorEvent from an OperatorCoordinator to a task was lost. Triggering task failover to ensure consistency. Event: '[NoMoreSplitEvent]', targetTask: Source: T1[154] -> WatermarkAssigner[155] -> StreamRecordTimestampInserter[156] -> T1[156]: Writer (1/4) - execution #0
Caused by: org.apache.flink.runtime.operators.coordination.TaskNotRunningException: Task is not running, but in state FINISHED
{code}

Root cause: the in-test TestSource reader returned END_OF_INPUT as soon as it had
emitted its rows. The SingleSplitEnumerator assigns the split and then delivers a
NoMoreSplitsEvent as a separate event via the coordinator thread. If the split-holding
subtask (here, (1/4)) finishes before that event is delivered, the coordinator sends
it to an already-FINISHED task; the framework treats the undeliverable event as a lost
OperatorEvent and triggers a failover, which the job's NoRestartBackoffTimeStrategy
turns into a hard failure. No data is lost (all rows are emitted) — this is a benign
test-config race, not a coordinator bug.

Fix: defer END_OF_INPUT until notifyNoMoreSplits() has been received, so the reader
cannot finish before the event lands (mirroring SourceReaderBase.finishedOrAvailableLater()).
First observed in build 76431; the fix is justified on the structural race, not frequency.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)