You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Ingo Bürk (Jira)" <ji...@apache.org> on 2021/08/27 07:00:06 UTC

[jira] [Created] (FLINK-24014) Filters pushed through ChangelogNormalize cause incorrect results

Ingo Bürk created FLINK-24014:
---------------------------------

             Summary: Filters pushed through ChangelogNormalize cause incorrect results
                 Key: FLINK-24014
                 URL: https://issues.apache.org/jira/browse/FLINK-24014
             Project: Flink
          Issue Type: Bug
          Components: Table SQL / Planner
    Affects Versions: 1.13.2
            Reporter: Ingo Bürk
            Assignee: Ingo Bürk


If a source implements SupportsFilterPushDown, all filters of a query get pushed through to the source, including past a ChangelogNormalize.

However, this can cause incorrect results as pushing non-PK filters is incorrect. For example, consider a filter a < 10 and records +I[key1, a=9], -D[key1, a=10].

A strategy to fix it could be a physical rule which deoptimizes the filter pushdown, however in this case we should investigate more how strict we need to be. For now we could also simply document this in the SupportsFilterPushDown interface and warn implementations of this possibility.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)