[jira] [Commented] (CALCITE-569) ArrayIndexOutOfBoundsException when deducing collation

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293819#comment-14293819 ] 

Julian Hyde commented on CALCITE-569:
-------------------------------------

Your code change to RexProgram doesn't look quite right. If the input has collations (a, b), (x, p), (x, y, z) and we're looking for (x, y) then we shouldn't stop at (x, p), should we?

There was a specific reason that I introduced PRESERVE (ordering by non-projected columns). I couldn't tell from the patch -- is that case still handled?

I am working on CALCITE-526 in branch https://github.com/julianhyde/incubator-calcite/tree/calcite-526. A lot of the work is to do with traits and collation; for instance, I allow a RelNode (but not a RelSubset) to have multiple traits of the same type, if the traitDef supports it. I think we need to solve both issues at the same time.

So I propose that we split the patch in two -- the column renames can be checked in first, and I'll fold the collation work into my branch. What do you think?

I'm uncomfortable with the statement "LogicalProject is always created with empty collation". The LogicalXxx nodes are of logical convention but I'm not sure we should disallow them from having other traits. I'm going to put the logic to deduce the collations for core types (project, filter, sort, aggregate, join, union) into a new class RelMdCollation. By default, each RelNode subclass will have the traits you'd expect - e.g. LogicalProject(x, y) on LogicalSort(y) will indeed be sorted on y - but any code that creates a RelNode subclass can override.

I am well aware that not every implementation of Aggregate(x, y, sum(z)) produces output sorted on (x, y) but to ban collations on logical RelNodes would be going to far the other direction. I don't want to have to wait til we get into the physical domain (e.g. EnumerableXxx) before collation comes into play. That will make it more difficult to share rules.

> ArrayIndexOutOfBoundsException when deducing collation
> ------------------------------------------------------
>
>                 Key: CALCITE-569
>                 URL: https://issues.apache.org/jira/browse/CALCITE-569
>             Project: Calcite
>          Issue Type: Bug
>    Affects Versions: 1.0.0-incubating
>            Reporter: Aman Sinha
>            Assignee: Julian Hyde
>         Attachments: 0001-CALCITE-569-Create-a-Project-with-empty-collation-if.patch, 0001-Properly-track-collation-trait-for-select-a-from-.-o.patch, 0001-Properly-track-collation-trait-for-select-a-from-.-o.patch, PlanTest.java.diff
>
>
> If a subquery has an ORDER BY on a column that is not in the SELECT list and the outer query does another ORDER BY,  Calcite encounters an ArrayIndexOutOfBoundException when deducing collation. 
> In PlannerTest, I created a simple test by first adding the following traits: 
>  {code}
> 	    List<RelTraitDef> traitDefs = new ArrayList<RelTraitDef>();
> 	    traitDefs.add(ConventionTraitDef.INSTANCE);
> 	    traitDefs.add(RelCollationTraitDef.INSTANCE);
> {code}
> And ran the following query: 
> {code}
> select t.psPartkey from (select ps.psPartkey from `tpch`.`partsupp` ps order by ps.psSupplyCost) t order by t.psPartkey"
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: -1
> 	at org.apache.calcite.rex.RexProgram.deduceCollations(RexProgram.java:589)
> 	at org.apache.calcite.rex.RexProgram.getCollations(RexProgram.java:558)
> 	at org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2685)
> 	at org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2623)
> 	at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3571)
> 	at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:613)
> 	at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:568)
> 	at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2929)
> 	at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:526)
> 	at org.apache.calcite.prepare.PlannerImpl.convert(PlannerImpl.java:189)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)