[jira] [Updated] (FLINK-16860) TableException: Failed to push filter into table source! when upgrading flink to 1.9.2

["Aljoscha Krettek (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-16860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aljoscha Krettek updated FLINK-16860:
-------------------------------------
    Component/s: Table SQL / API

> TableException: Failed to push filter into table source! when upgrading flink to 1.9.2
> --------------------------------------------------------------------------------------
>
>                 Key: FLINK-16860
>                 URL: https://issues.apache.org/jira/browse/FLINK-16860
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / ORC, Table SQL / API
>    Affects Versions: 1.9.2, 1.10.0, 1.10.1, 1.11.0, 1.10.2
>         Environment: flink 1.8.2
> flink 1.9.2
>            Reporter: Nikola
>            Priority: Major
>         Attachments: flink-1.8.2.txt, flink-1.9.2.txt
>
>
> We have a batch job which we currently have on a flink cluster running 1.8.2
>  The job runs fine. We wanted to upgrade to flink 1.10, but that yielded errors, so we started downgrading until we found that the issue is in flink 1.9.2
> The job on 1.9.2 fails with:
> {code:java}
> Caused by: org.apache.flink.table.api.TableException: Failed to push filter into table source! table source with pushdown capability must override and change explainSource() API to explain the pushdown applied!{code}
> Which is not happening on flink 1.8.2. You can check the logs for the exactly same job, just running on different cluster versions: [^flink-1.8.2.txt] [^flink-1.9.2.txt]
>  
> I tried to narrow it down and it seems that this exception has been added in FLINK-12399 and there was a small discussion regarding the exception: [https://github.com/apache/flink/pull/8468#discussion_r329876088]
> Our code looks something like this:
>  
> {code:java}
> String tempTableName = "tempTable";
> String sql = SqlBuilder.buildSql(tempTableName);
> BatchTableEnvironment tableEnv = BatchTableEnvironment.create(env);
> OrcTableSource orcTableSource = OrcTableSource.builder()
>  .path(hdfsFolder, true)
>  .forOrcSchema(ORC.getSchema())
>  .withConfiguration(config)
>  .build();
> tableEnv.registerTableSource(tempTableName, orcTableSource);
> Table tempTable = tableEnv.sqlQuery(sql);
> return tableEnv.toDataSet(tempTable, Row.class); 
> {code}
> Where the sql build is nothing more than
> {code:java}
> SELECT * FROM table WHERE id IN (1,2,3) AND mid IN(4,5,6){code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)