You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "godfrey he (Jira)" <ji...@apache.org> on 2022/05/31 07:23:00 UTC
[jira] [Assigned] (FLINK-27849) Harden correctness for non-deterministic updates present in the changelog pipeline
[ https://issues.apache.org/jira/browse/FLINK-27849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
godfrey he reassigned FLINK-27849:
----------------------------------
Assignee: lincoln lee
> Harden correctness for non-deterministic updates present in the changelog pipeline
> ----------------------------------------------------------------------------------
>
> Key: FLINK-27849
> URL: https://issues.apache.org/jira/browse/FLINK-27849
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Reporter: lincoln lee
> Assignee: lincoln lee
> Priority: Major
> Fix For: 1.16.0
>
>
> There commonly exists updates(which means not only RowKind.INSERT messages) in a streaming pipeline, then wrong results or error may occurs when use some non-deterministic functions or operations.
> It is a long lived issue since the first day that flink sql was available in streaming, but it still not totally be eliminated though some efforts have been taken.
> We should detect all the non-deterministic operations in the changelog pipelines, raise an error to tell users the risk and also add an mechanism that can process such a issue if a user is willing to pay some cost(probably introduce the state).
> All non-deterministic operations include builtin temporal functions(now, current_timestamp...), UUID, RAND...
> or user defined non-deterministic functions (override isDeterministic return false)
> or a lookup join on a lookup source which data may change over time
--
This message was sent by Atlassian Jira
(v8.20.7#820007)