You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Danny Chan (JIRA)" <ji...@apache.org> on 2019/06/05 01:48:00 UTC
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output
map for LogicalAggregate decorrelate
[ https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856237#comment-16856237 ]
Danny Chan commented on CALCITE-2744:
-------------------------------------
I digged into the code and found that it is not because the agg call use multiple parameters, it is because the final output map for the aggregate is totally wrong. Say, We have a "combinedMap" as the mapping between the old input and the new project:
{code:java}
Aggregate
| new project - - - -
| new_input combinedMap
| old_input - - - -
{code}
This "combinedMap" can not be used as map of the old aggregate and the new generated aggregate.
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> ---------------------------------------------------------------------
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.20.0
> Reporter: Wei Zhong
> Assignee: Danny Chan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest, we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 32 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:646)
> ... 37 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 39 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalFilter)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:646)
> ... 44 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 46 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalCorrelate)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1008)
> ... 51 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 53 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalAggregate)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:1059)
> ... 58 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 60 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:466)
> ... 65 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 67 more
> Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelDecorrelator$Frame org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalAggregate)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:646)
> ... 72 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524)
> ... 74 more
> Caused by: java.lang.AssertionError: out of range; value: 1, limit: 1
> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> at org.apache.calcite.sql2rel.RelDecorrelator.allLessThan(RelDecorrelator.java:1473)
> at org.apache.calcite.sql2rel.RelDecorrelator$Frame.<init>(RelDecorrelator.java:2781)
> at org.apache.calcite.sql2rel.RelDecorrelator.register(RelDecorrelator.java:1464)
> at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(RelDecorrelator.java:609)
> ... 79 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)