[jira] [Resolved] (FLINK-40571) ProcessTableFunctionSemanticTests[process-stateful-multi-input-with-timeout] is flaky on master

["Martijn Visser (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martijn Visser resolved FLINK-40571.
------------------------------------
    Resolution: Fixed

> ProcessTableFunctionSemanticTests[process-stateful-multi-input-with-timeout] is flaky on master
> -----------------------------------------------------------------------------------------------
>
>                 Key: FLINK-40571
>                 URL: https://issues.apache.org/jira/browse/FLINK-40571
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner
>    Affects Versions: 2.2.2, 2.3.1, 2.4.0
>            Reporter: Purushottam Sinha
>            Assignee: Purushottam Sinha
>            Priority: Critical
>              Labels: pull-request-available, test-stability
>             Fix For: 2.2.2, 2.3.1, 2.4.0
>
>
> h3. Problem
> {{ProcessTableFunctionSemanticTests}} is flaky on master for the {{process-stateful-multi-input-with-timeout}} program. It runs the two-input {{TimedJoinFunction}} with {{on_time => DESCRIPTOR(ts)}} but no {{ORDER BY}}, and asserts an exact append changelog. Per the {{ProcessTableFunction}} contract, {{on_time}} "does NOT affect the processing order of rows", so the two inputs may reach {{eval()}} in any interleaving. The pinned expectation only matches the common interleaving.
> h3. Evidence
> * master run [https://github.com/apache/flink/actions/runs/34019180875/job/101449611519] (module: table, commit {{ca6e927}}) — {{runSteps(TableTestProgram)[63]}} {{[process-stateful-multi-input-with-timeout]}}.
> * Failing assertion:
> {noformat}
> missing:  +I[Bob, Bob, 1 score in city London, ...T00:00:00Z],
>           +I[Bob, Bob, 2 score in city London, ...00.002Z],
>           +I[Bob, Bob, 3 score in city London, ...00.003Z]
> extra:    +I[Bob, Bob, 3 score in city London, ...T00:00:00Z]
> {noformat}
> * Root cause: when {{city@0}} is processed after {{score1@0/score2@2/score3@3}}, the single-value {{score}} state keeps only score 3; the join then emits at the city row's timestamp ({{@0}}). Scores 1/2 are lost. This is a valid outcome for that (unordered) interleaving.
> h3. Notes
> * Sibling of FLINK-40568, but the fix there (assert materialized data) does NOT apply here: the output is append-only with no convergent view, so materializing changes nothing.
> * Fixing this needs a redesign so the assertion no longer depends on cross-input arrival order (e.g., assert only the order-independent timeout path, or restructure the data). Best decided by a PTF-test owner.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)