[jira] [Commented] (FLINK-40898) TRY_RESOLVE wrongly rejects non-deterministic filters inside ChangelogNormalize

["Sergey Paryshev (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-40898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129557#comment-18129557 ] 

Sergey Paryshev commented on FLINK-40898:
-----------------------------------------

I offer a bugfix, [~martijnvisser] [~snuyanzin] please check

> TRY_RESOLVE wrongly rejects non-deterministic filters inside ChangelogNormalize
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-40898
>                 URL: https://issues.apache.org/jira/browse/FLINK-40898
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner
>            Reporter: Martijn Visser
>            Priority: Major
>              Labels: pull-request-available
>
> Since FLINK-39313 the NDU check rejects a non-deterministic condition that {{PushCalcPastChangelogNormalizeRule}} pushed into ChangelogNormalize. The operator applies that condition to inserts and updates only and emits retractions from state ({{DeduplicateFunctionHelper#processLastRowOnChangelogWithFilter}}), so it cannot drop a retraction and the rejection is a false positive. A probe with a UDF that is true on its first call only ends with an empty sink here, while every Calc placement keeps the stale row.
> We should drop the check in {{visitChangelogNormalize}}, or downgrade it to a note that the condition is evaluated once per key, and keep the Calc checks from FLINK-28737. The upsert-kafka shape from FLINK-35341 has been handled by this placement since FLINK-37390.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)