[jira] [Commented] (DRILL-5359) ClassCastException when push down filter on the output of flatten into parquet scan

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929297#comment-15929297 ] 

ASF GitHub Bot commented on DRILL-5359:
---------------------------------------

GitHub user jinfengni opened a pull request:

    https://github.com/apache/drill/pull/786

    DRILL-5359: Fix ClassCastException when Drill pushes down filter on t…

    …he output of flatten operator.
    
    - Move findItemOrFlatten as a static method in DrillRelOptUtil.
    - Exclude filter conditions if they contain item/flatten operator.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jinfengni/incubator-drill DRILL-5359

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/drill/pull/786.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #786
    
----
commit 73d4859afae0818f2f907fcf33d2ae9430efe728
Author: Jinfeng Ni <jn...@apache.org>
Date:   2017-03-16T21:44:35Z

    DRILL-5359: Fix ClassCastException when Drill pushes down filter on the output of flatten operator.
    
    - Move findItemOrFlatten as a static method in DrillRelOptUtil.
    - Exclude filter conditions if they contain item/flatten operator.

----


> ClassCastException when push down filter on the output of flatten into parquet scan
> -----------------------------------------------------------------------------------
>
>                 Key: DRILL-5359
>                 URL: https://issues.apache.org/jira/browse/DRILL-5359
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.9.0
>            Reporter: Jinfeng Ni
>            Assignee: Jinfeng Ni
>             Fix For: 1.11.0
>
>
> The following simplified query would hit ClassCastException.
> {code}
> select n_regionkey
> from (select n_regionkey, 
>             flatten(nation.cities) as cities 
>       from cp.`tpch/nation.parquet` nation) as flattenedCities 
> where flattenedCities.cities.`zip` = '12345';
> {code}
> Here is the stacktrace for the Exception : 
> {code}
> caused by: java.lang.ClassCastException: org.apache.drill.common.expression.FunctionCall cannot be cast to org.apache.drill.common.expression.SchemaPath
> 	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:170) ~[classes/:na]
> 	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80) ~[classes/:na]
> 	at org.apache.calcite.rex.RexCall.accept(RexCall.java:107) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> 	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.doFunction(DrillOptiq.java:205) ~[classes/:na]
> 	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:105) ~[classes/:na]
> 	at org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80) ~[classes/:na]
> 	at org.apache.calcite.rex.RexCall.accept(RexCall.java:107) ~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
> 	at org.apache.drill.exec.planner.logical.DrillOptiq.toDrill(DrillOptiq.java:77) ~[classes/:na]
> 	at org.apache.drill.exec.store.parquet.ParquetPushDownFilter.doOnMatch(ParquetPushDownFilter.java:141) ~[classes/:na]
> 	at org.apache.drill.exec.store.parquet.ParquetPushDownFilter$1.onMatch(ParquetPushDownFilter.java:68) ~[classes/:na]
> {code}
> The cause of this problem: Parquet filter pushdown rule tries to push a filter expression containing item/flatten operators into parquet scan. However, the method DrillOptiq.toDrill() does not allow such expression (since "flatten" is not a scalar function).  
> The solution is to disable pushing such filter expression.  Even the rule allows, the underneath parquet metadata would not have the corresponding statistics; there is no point to considering push such filter expression. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)