You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "godfrey he (JIRA)" <ji...@apache.org> on 2017/04/19 09:50:41 UTC
[jira] [Updated] (FLINK-6326) add ProjectMergeRule at logical optimization stage

     [ https://issues.apache.org/jira/browse/FLINK-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

godfrey he updated FLINK-6326:
------------------------------
    Description: 
add ProjectMergeRule to merge projections. Some SQLs can not push projection into scan without this rule. 
e.g.

table1: id: int, name: string
table2: id: string, score: double, first: string, last: string

{code}
SELECT a.id, b.score FROM 
 (SELECT id FROM table1 WHERE id > 10) a  
LEFT OUTER JOIN 
 (SELECT * FROM table2) b 
ON CAST(a.id AS VARCHAR) = b.id
{code}

{code}
== Optimized Logical Plan without ProjectMergeRule ==
DataSetCalc(select=[id, score])
  DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], joinType=[LeftOuterJoin])
    DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
      BatchTableSourceScan(table=[[table1]], fields=[id])
    BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
{code}

{code}
== Optimized Logical Plan with ProjectMergeRule ==
DataSetCalc(select=[id, score])
  DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], joinType=[LeftOuterJoin])
    DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
      BatchTableSourceScan(table=[[table1]], fields=[id])
    BatchTableSourceScan(table=[[table2]], fields=[id, score])
{code}

  was:
add ProjectMergeRule to merge projections. without this rule, some SQLs can not push projection into scan. 
e.g.

table1: id: int, name: string
table2: id: string, score: double, first: string, last: string

{code}
SELECT a.id, b.score FROM 
 (SELECT id FROM table1 WHERE id > 10) a  
LEFT OUTER JOIN 
 (SELECT * FROM table2) b 
ON CAST(a.id AS VARCHAR) = b.id
{code}

{code}
== Optimized Logical Plan without ProjectMergeRule ==
DataSetCalc(select=[id, score])
  DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], joinType=[LeftOuterJoin])
    DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
      BatchTableSourceScan(table=[[table1]], fields=[id])
    BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
{code}

{code}
== Optimized Logical Plan with ProjectMergeRule ==
DataSetCalc(select=[id, score])
  DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], joinType=[LeftOuterJoin])
    DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
      BatchTableSourceScan(table=[[table1]], fields=[id])
    BatchTableSourceScan(table=[[table2]], fields=[id, score])
{code}


> add ProjectMergeRule at logical optimization stage
> --------------------------------------------------
>
>                 Key: FLINK-6326
>                 URL: https://issues.apache.org/jira/browse/FLINK-6326
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: godfrey he
>            Assignee: godfrey he
>
> add ProjectMergeRule to merge projections. Some SQLs can not push projection into scan without this rule. 
> e.g.
> table1: id: int, name: string
> table2: id: string, score: double, first: string, last: string
> {code}
> SELECT a.id, b.score FROM 
>  (SELECT id FROM table1 WHERE id > 10) a  
> LEFT OUTER JOIN 
>  (SELECT * FROM table2) b 
> ON CAST(a.id AS VARCHAR) = b.id
> {code}
> {code}
> == Optimized Logical Plan without ProjectMergeRule ==
> DataSetCalc(select=[id, score])
>   DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score, first, last], joinType=[LeftOuterJoin])
>     DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
>       BatchTableSourceScan(table=[[table1]], fields=[id])
>     BatchTableSourceScan(table=[[table2]], fields=[id, score, first, last])
> {code}
> {code}
> == Optimized Logical Plan with ProjectMergeRule ==
> DataSetCalc(select=[id, score])
>   DataSetJoin(where=[=(id0, id1)], join=[id, id0, id1, score], joinType=[LeftOuterJoin])
>     DataSetCalc(select=[id, CAST(id) AS id0], where=[>(id, 10)])
>       BatchTableSourceScan(table=[[table1]], fields=[id])
>     BatchTableSourceScan(table=[[table2]], fields=[id, score])
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)