[jira] [Commented] (CALCITE-2223) ProjectMergeRule is infinitely matched when is applied after ProjectReduceExpressionsRule

["Volodymyr Vysotskyi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421750#comment-16421750 ] 

Volodymyr Vysotskyi commented on CALCITE-2223:
----------------------------------------------

[~zhenw], thanks for looking into this issue.

I don't know details of CALCITE-1862, but regarding this issue, I don't think that forcing zero importance for matching the rule to {{RelNode}} is the right solution. It will prevent applying {{ProjectMergeRule}} rule more times for the same {{RelNode}}, but we will hide a symptom, not a problem which causes this bug.

I spent some time debugging it and noticed that at some stage after applying the {{ProjectReduceExpressionsRule}} new {{LogicalProject}} instance is added to the wrong {{RelSubset}} and it causes merging of {{RelSet}} s which shouldn't be merged, because rel nodes from one {{RelSet}} are inputs of rel nodes from another one. It causes infinite matching of {{ProjectMergeRule}} for existing and new one {{LogicalProject}} s.

I believe that the problem in {{VolcanoPlanner.registerImpl()}} method. This method has a check for {{RelNode}} s digest, and if {{mapDigestToRel}} map already contains {{RelNode}} with the same digest, new {{RelNode}} will be added into the same {{RelSubset}} where the old one. So I think that the solution is to make {{RelNode}} s digest more unique to reflect information about its position in the plan.

> ProjectMergeRule is infinitely matched when is applied after ProjectReduceExpressionsRule
> -----------------------------------------------------------------------------------------
>
>                 Key: CALCITE-2223
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2223
>             Project: Calcite
>          Issue Type: Bug
>            Reporter: Volodymyr Vysotskyi
>            Assignee: Julian Hyde
>            Priority: Critical
>
> For queries like this:
> {code:sql}
> select t1.f from (select cast(f as int) f, f from (select cast(f as int) f from (values('1')) t(f))) as t1
> {code}
> OOM is thrown when {{ProjectMergeRule}} is applied before applying {{ProjectReduceExpressionsRule}} in VolcanoPlanner.
>  A simple test to reproduce this issue (in {{RelOptRulesTest}}):
> {code:java}
>   @Test public void testOomProjectMergeRule() {
>     RelBuilder relBuilder = RelBuilder.create(RelBuilderTest.config().build());
>     RelNode relNode = relBuilder
>         .values(new String[]{"f"}, "1")
>         .project(
>             relBuilder.alias(
>                 relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
>                 "f"))
>         .project(
>             relBuilder.alias(
>                 relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
>                 "f0"),
>             relBuilder.alias(relBuilder.field(0), "f"))
>         .project(
>             relBuilder.alias(relBuilder.field(0), "f"))
>         .build();
>     RelOptPlanner planner = relNode.getCluster().getPlanner();
>     RuleSet ruleSet =
>         RuleSets.ofList(
>             ReduceExpressionsRule.PROJECT_INSTANCE,
>             new ProjectMergeRuleWithLongerName(),
>             EnumerableRules.ENUMERABLE_PROJECT_RULE,
>             EnumerableRules.ENUMERABLE_VALUES_RULE);
>     Program program = Programs.of(ruleSet);
>     RelTraitSet toTraits =
>         relNode.getCluster().traitSet()
>             .replace(0, EnumerableConvention.INSTANCE);
>     RelNode output = program.run(planner, relNode, toTraits,
>         ImmutableList.<RelOptMaterialization>of(), ImmutableList.<RelOptLattice>of());
>     // check for output
>   }
>   /**
>    * ProjectMergeRule inheritor which has
>    * class name greater than ProjectReduceExpressionsRule class name (String.compareTo()).
>    *
>    * It is needed for RuleQueue.popMatch() method
>    * to apply this rule before ProjectReduceExpressionsRule.
>    */
>   private static class ProjectMergeRuleWithLongerName extends ProjectMergeRule {
>     public ProjectMergeRuleWithLongerName() {
>       super(true, RelFactories.LOGICAL_BUILDER);
>     }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)