[jira] [Commented] (FLINK-40077) CommonExecSinkITCase.testNullEnforcer fails intermittently: lost NoMoreSplitsEvent failover masks the NOT NULL enforcer cause

["Martijn Visser (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-40077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129626#comment-18129626 ] 

Martijn Visser commented on FLINK-40077:
----------------------------------------

Another one on master, nightly 2026-10-21 (AdaptiveScheduler profile, module table):
https://github.com/apache/flink/actions/runs/39531011651/job/118998630186

{code}
[ERROR] org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecSinkITCase.testNullEnforcer -- Time elapsed: 0.124 s <<< FAILURE!
[Any cause contains message 'Column 'b' is NOT NULL, however, a null value is being written into it. ...']
    org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
    org.apache.flink.util.FlinkException: An OperatorEvent from an OperatorCoordinator to a task was lost. Triggering task failover to ensure consistency. Event: '[NoMoreSplitEvent]', targetTask: Source: T1[7264] -> ConstraintEnforcer[7265] -> T1[7265]: Writer (2/4) - execution #0
    org.apache.flink.runtime.operators.coordination.TaskNotRunningException: Task is not running, but in state FAILED
{code}

> CommonExecSinkITCase.testNullEnforcer fails intermittently: lost NoMoreSplitsEvent failover masks the NOT NULL enforcer cause
> -----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-40077
>                 URL: https://issues.apache.org/jira/browse/FLINK-40077
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner
>    Affects Versions: 2.4.0
>            Reporter: Martijn Visser
>            Priority: Major
>              Labels: pull-request-available
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=76751&view=results (leg: test_cron_adaptive_scheduler table)
> {code}
> [Any cause contains message 'Column 'b' is NOT NULL, however, a null value is being written into it. ...']
> Expecting any element of:
>   ...
>     org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
>     org.apache.flink.util.FlinkException: An OperatorEvent from an OperatorCoordinator to a task was lost. Triggering task failover to ensure consistency. Event: '[NoMoreSplitEvent]', targetTask: Source: T1 -> ConstraintEnforcer -> T1: Writer (3/4) - execution #0
>     org.apache.flink.runtime.operators.coordination.TaskNotRunningException: Task is not running, but in state FAILED
> {code}
> Commit 3196a998cf0. The test asserts the job failure cause chain contains the NOT NULL enforcer message, but the reported chain ends in the lost-OperatorEvent failover instead; the enforcer error is absent.
> Root cause: the Source V2 test source (introduced by FLINK-32695) emits all rows on the first {{pollNext}}, before the coordinator delivers {{NoMoreSplitsEvent}}. The NOT NULL enforcer then fails the chained Source -> ConstraintEnforcer -> Writer task while that event is still in flight. Delivery to the already-FAILED task throws {{TaskNotRunningException}}; because {{NoMoreSplitsEvent}} is not loss-tolerant, {{SubtaskGatewayImpl}} escalates it to a lost-OperatorEvent failover, which under {{NoRestartBackoffTimeStrategy}} becomes the reported terminal cause and masks the enforcer error. Surfaces under CI load; does not reproduce on an idle machine.
> Relation to FLINK-40011: that ticket fixed the FINISHED-task variant in the same class (readers returning END_OF_INPUT before the event arrived) by gating completion on {{noMoreSplits}}. This is the FAILED-task variant, which the completion-only gate cannot cover.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)