You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Homer <lo...@gmail.com> on 2019/02/12 20:51:27 UTC

Question on ProjectMergeRule

Hi

The ProjectMergeRule 
<https://calcite.apache.org/apidocs/org/apache/calcite/rel/rules/ProjectMergeRule.html> 
specifies the following

ProjectMergeRule merges a |Project| 
<https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html> 
into another |Project| 
<https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html>, 
provided the projects aren't projecting identical sets of input references.

I would like to get rid of identical projection in a plan, but this rule 
specifically doesn't remove identical projections

i am wondering if there is a standard rule that will remove identical 
projections.

Also curious why we dont want to merge identical rules in the 
ProjectMergeRule case?

thanks

Re: Question on ProjectMergeRule

Posted by Homer <lo...@gmail.com>.
Julian,

I am going to speculate it is due to the ProjectRemoveRule.

Since the Project is on the Join and only projects its inputs.

Thanks

On 2019-02-12 1:36 p.m., Homer wrote:
>
> Julian,
>
> Thank for your reply, I also found ProjectRemoveRule and it 
> successfully did what I needed.  Much appreciated.
>
> One interesting thing
>
> I am using these 4 rules:
>
> FilterProjectTransposeRule.INSTANCE,
> ProjectRemoveRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> FilterMergeRule.INSTANCE
>
> I am not quite sure why I am losing the project in the After case
>
>
> 2019-02-12 13:21:44 ERROR Parser:ToSqlNode:270 - before
> LogicalProject(approx_value=[$0], exact_value=[$1])
>    LogicalJoin(condition=[true], joinType=[inner])
>      LogicalAggregate(group=[{}], approx_value=[COUNT_DISTINCT($0)])
>        LogicalProject(x=[$0])
>          EnumerableTableScan(table=[[schema, tab1]])
>      LogicalAggregate(group=[{}], exact_value=[COUNT(DISTINCT $0)])
>        LogicalProject(y=[$1])
>          EnumerableTableScan(table=[[schema, tab1]])
>
> 2019-02-12 13:21:44 ERROR Parser:ToSqlNode:278 - after
> LogicalJoin(condition=[true], joinType=[inner])
>    LogicalAggregate(group=[{}], approx_value=[COUNT_DISTINCT($0)])
>      LogicalProject(x=[$0])
>        EnumerableTableScan(table=[[schema, tab1]])
>    LogicalAggregate(group=[{}], exact_value=[COUNT(DISTINCT $0)])
>      LogicalProject(y=[$1])
>        EnumerableTableScan(table=[[schema, tab1]])
>
> Thanks
>
> On 2019-02-12 1:05 p.m., Julian Hyde wrote:
>> Sorry the documentation isn’t clear, but read the code.
>>
>> It doesn’t remove identical projections so that ProjectRemoveRule can do it instead.
>>
>> ProjectMergeRule has a boolean “force" flag that will cause it to remove even if projections are identical.
>>
>> If you use RelBuilder to build the tree, a lot of these optimizations will happen automatically as you build the tree, so you may not even need this rule if your use case is just “building” and not “optimizing”.
>>
>> Julian
>>
>>
>>> On Feb 12, 2019, at 12:51 PM, Homer<lo...@gmail.com>  wrote:
>>>
>>> Hi
>>>
>>> The ProjectMergeRule<https://calcite.apache.org/apidocs/org/apache/calcite/rel/rules/ProjectMergeRule.html>  specifies the following
>>>
>>> ProjectMergeRule merges a |Project|<https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html>  into another |Project|<https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html>, provided the projects aren't projecting identical sets of input references.
>>>
>>> I would like to get rid of identical projection in a plan, but this rule specifically doesn't remove identical projections
>>>
>>> i am wondering if there is a standard rule that will remove identical projections.
>>>
>>> Also curious why we dont want to merge identical rules in the ProjectMergeRule case?
>>>
>>> thanks

Re: Question on ProjectMergeRule

Posted by Homer <lo...@gmail.com>.
Julian,

Thank for your reply, I also found ProjectRemoveRule and it successfully 
did what I needed.  Much appreciated.

One interesting thing

I am using these 4 rules:

FilterProjectTransposeRule.INSTANCE,
ProjectRemoveRule.INSTANCE,
ProjectMergeRule.INSTANCE,
FilterMergeRule.INSTANCE

I am not quite sure why I am losing the project in the After case


2019-02-12 13:21:44 ERROR Parser:ToSqlNode:270 - before
LogicalProject(approx_value=[$0], exact_value=[$1])
   LogicalJoin(condition=[true], joinType=[inner])
     LogicalAggregate(group=[{}], approx_value=[COUNT_DISTINCT($0)])
       LogicalProject(x=[$0])
         EnumerableTableScan(table=[[schema, tab1]])
     LogicalAggregate(group=[{}], exact_value=[COUNT(DISTINCT $0)])
       LogicalProject(y=[$1])
         EnumerableTableScan(table=[[schema, tab1]])

2019-02-12 13:21:44 ERROR Parser:ToSqlNode:278 - after
LogicalJoin(condition=[true], joinType=[inner])
   LogicalAggregate(group=[{}], approx_value=[COUNT_DISTINCT($0)])
     LogicalProject(x=[$0])
       EnumerableTableScan(table=[[schema, tab1]])
   LogicalAggregate(group=[{}], exact_value=[COUNT(DISTINCT $0)])
     LogicalProject(y=[$1])
       EnumerableTableScan(table=[[schema, tab1]])

Thanks

On 2019-02-12 1:05 p.m., Julian Hyde wrote:
> Sorry the documentation isn’t clear, but read the code.
>
> It doesn’t remove identical projections so that ProjectRemoveRule can do it instead.
>
> ProjectMergeRule has a boolean “force" flag that will cause it to remove even if projections are identical.
>
> If you use RelBuilder to build the tree, a lot of these optimizations will happen automatically as you build the tree, so you may not even need this rule if your use case is just “building” and not “optimizing”.
>
> Julian
>
>
>> On Feb 12, 2019, at 12:51 PM, Homer <lo...@gmail.com> wrote:
>>
>> Hi
>>
>> The ProjectMergeRule <https://calcite.apache.org/apidocs/org/apache/calcite/rel/rules/ProjectMergeRule.html> specifies the following
>>
>> ProjectMergeRule merges a |Project| <https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html> into another |Project| <https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html>, provided the projects aren't projecting identical sets of input references.
>>
>> I would like to get rid of identical projection in a plan, but this rule specifically doesn't remove identical projections
>>
>> i am wondering if there is a standard rule that will remove identical projections.
>>
>> Also curious why we dont want to merge identical rules in the ProjectMergeRule case?
>>
>> thanks

Re: Question on ProjectMergeRule

Posted by Julian Hyde <jh...@apache.org>.
Sorry the documentation isn’t clear, but read the code.

It doesn’t remove identical projections so that ProjectRemoveRule can do it instead.

ProjectMergeRule has a boolean “force" flag that will cause it to remove even if projections are identical.

If you use RelBuilder to build the tree, a lot of these optimizations will happen automatically as you build the tree, so you may not even need this rule if your use case is just “building” and not “optimizing”.

Julian


> On Feb 12, 2019, at 12:51 PM, Homer <lo...@gmail.com> wrote:
> 
> Hi
> 
> The ProjectMergeRule <https://calcite.apache.org/apidocs/org/apache/calcite/rel/rules/ProjectMergeRule.html> specifies the following
> 
> ProjectMergeRule merges a |Project| <https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html> into another |Project| <https://calcite.apache.org/apidocs/org/apache/calcite/rel/core/Project.html>, provided the projects aren't projecting identical sets of input references.
> 
> I would like to get rid of identical projection in a plan, but this rule specifically doesn't remove identical projections
> 
> i am wondering if there is a standard rule that will remove identical projections.
> 
> Also curious why we dont want to merge identical rules in the ProjectMergeRule case?
> 
> thanks