[jira] [Commented] (CALCITE-3087) Mappings.apply2 doesn't keep order

["Feng Zhu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854227#comment-16854227 ] 

Feng Zhu commented on CALCITE-3087:
-----------------------------------

Hi, [~hyuan]. I think we need to check whether the mapping keeps order in *{{AggregateOnProjectToAggregateUnifyRule}}*. According to the rule's description:

"

{color:#808080}The rule is necessary when we unify query=Aggregate(x) with{color}{color:#808080} target=Aggregate(x, y). Query will tend to have an extra Project(x) on its {color}{color:#808080}input, which this rule knows {color:#FF0000}is safe to ignore{color}.{color}

"

However, when the Project mapping changes the field's order, it cannot be ignored.  Even we sort groupsets in such case, the rowtype of new created result (MultableAggregate) is different to the query.
{code:java}
Result: RecordType(JavaType(class java.lang.Integer) empid, JavaType(class java.lang.Integer) deptno, BIGINT $f2)
Query:  RecordType(JavaType(class java.lang.Integer) deptno, JavaType(class java.lang.Integer) empid, BIGINT $f2)
{code}
Moreover, I noticed that some assertions need to be added to ensure ordering in *MultableAggregate*, as that in *Aggregate:*
{code:java}
assert ImmutableBitSet.ORDERING.isStrictlyOrdered(groupSets) : groupSets;
for (ImmutableBitSet set : groupSets) {
  assert groupSet.contains(set);
}
assert groupSet.length() <= child.getRowType().getFieldCount();
{code}

> Mappings.apply2 doesn't keep order
> ----------------------------------
>
>                 Key: CALCITE-3087
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3087
>             Project: Calcite
>          Issue Type: Bug
>            Reporter: Haisheng Yuan
>            Priority: Major
>
> Add the following test to MaterializationTest:
> {code:java}
> @Test public void testAggregateGroupSetsRollUp0() {
>     checkMaterialize(
>         "select \"empid\", \"deptno\", count(*) as c, sum(\"salary\") as s from \"emps\" "
>             + "group by \"empid\", \"deptno\"",
>         "select count(*) + 1 as c,  \"deptno\" from \"emps\" group by cube(\"deptno\", \"empid\")",
>         HR_FKUK_MODEL,
>         CalciteAssert.checkResultContains(
>             "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], "
>                 + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n"
>                 + "  EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {1}, {}]], agg#0=[$SUM0($2)])\n"
>                 + "    EnumerableTableScan(table=[[hr, m0]])"));
>   }
> {code}
> We got error:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: must be sorted: [{0, 1}, {1}, {0}, {}]
> 	at org.apache.calcite.rel.core.Aggregate$Group.induce(Aggregate.java:472)
> 	at org.apache.calcite.rel.mutable.MutableAggregate.getGroupType(MutableAggregate.java:83)
> 	at org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1281)
> 	at org.apache.calcite.plan.SubstitutionVisitor$AggregateOnProjectToAggregateUnifyRule.apply(SubstitutionVisitor.java:1345)
> 	at org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:531)
> 	at org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:466)
> 	at org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56)
> 	at org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200)
> 	at org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72)
> 	at org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:348)
> 	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:584)
> {code}
> The reason is that Mappings.apply2 doesn't sort.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)