[PR] [FLINK-40077][table-planner] Fix lost NoMoreSplitsEvent masking failure cause in CommonExecSinkITCase [flink]

["MartijnVisser (via GitHub)" <gi...@apache.org>]

MartijnVisser opened a new pull request, #28646:
URL: https://github.com/apache/flink/pull/28646

   ## What is the purpose of the change
   
   `CommonExecSinkITCase.testNullEnforcer` is flaky on CI (observed on the AdaptiveScheduler nightly leg, Azure build [76751](https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=76751&view=results)). The test asserts that the job failure cause chain contains the NOT NULL enforcer message, but intermittently the reported terminal cause is instead `FlinkException: An OperatorEvent ... was lost ... Event: '[NoMoreSplitEvent]'`.
   
   Root cause: the Source V2 test source emits all rows on the first `pollNext`, before the coordinator's `NoMoreSplitsEvent` is delivered. The NOT NULL enforcer then fails the chained `Source -> ConstraintEnforcer -> Writer` task; the still-in-flight `NoMoreSplitsEvent` reaches an already-FAILED task and, since it is not loss-tolerant, escalates to a lost-OperatorEvent failover. Under `NoRestartBackoffTimeStrategy` that failover wins as the reported cause and masks the enforcer error. This race is distinct from FLINK-40011 (which addressed the reader reaching FINISHED before the event arrived); it occurs on the FAILED-task path, which gating `END_OF_INPUT` alone cannot cover.
   
   The underlying runtime behavior (a lost, non-loss-tolerant coordinator event to an already-failing task superseding the real application failure cause) is a pre-existing general rough edge that deserves its own runtime ticket; this PR only removes the race from the test scaffolding.
   
   ## Brief change log
   
   - Gate record emission in the test source's `pollNext` on `noMoreSplits`, so both coordinator events are delivered before any record is emitted (and thus before the enforcer can fail the task), leaving no coordinator event in flight at failure time. This extends the `noMoreSplits` gating added in FLINK-40011 from completion to emission.
   
   ## Verifying this change
   
   This change is a test-stability fix, covered by the existing `CommonExecSinkITCase` tests.
   
   - Ran `CommonExecSinkITCase#testNullEnforcer` 30 times under `-Penable-adaptive-scheduler` (the failing profile): all green.
   - The other tests in the class that reuse the same source (`testStreamRecordTimestampInserter*`, `testUnifiedSinksAreUsableWithDataStreamSinkProvider`, `testCharLengthEnforcer`, `testBinaryLengthEnforcer`) still pass, since the enumerator always signals no-more-splits and the rows are still emitted.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code (Opus 4.8)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-40077][table-planner] Fix lost NoMoreSplitsEvent masking failure cause in CommonExecSinkITCase [flink]

["flinkbot (via GitHub)" <gi...@apache.org>]

flinkbot commented on PR #28646:
URL: https://github.com/apache/flink/pull/28646#issuecomment-4890355618

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "77df62a01454f000b3d20eec7d0db7bcd489007e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "77df62a01454f000b3d20eec7d0db7bcd489007e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 77df62a01454f000b3d20eec7d0db7bcd489007e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org