[jira] [Updated] (DRILL-1549) Support partition pruning for simple IN filters

["Aman Sinha (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aman Sinha updated DRILL-1549:
------------------------------
    Attachment: 0001-DRILL-1549-Partition-pruning-for-simple-IN-filters.patch

> Support partition pruning for simple IN filters
> -----------------------------------------------
>
>                 Key: DRILL-1549
>                 URL: https://issues.apache.org/jira/browse/DRILL-1549
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 0.6.0
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
>             Fix For: 0.7.0
>
>         Attachments: 0001-DRILL-1549-Partition-pruning-for-simple-IN-filters.patch
>
>
> Queries containing directory filters of the form 'dir0 IN (1, 2)' currently don't get considered for partition pruning due to a bug inside the visitor of DirPathBuilder.  This JIRA is to address the issue for basic IN filters, which actually is converted to an OR condition by the optimizer (dir0 = 1 OR dir1 = 2).  Note that this JIRA does not cover the more general IN filters which are combined with other ANDs etc.
> To illustrate, here's the restricted case that we want to fix here: (the explain shows that the scan is reading all directories, not just 1995, 1996): 
> explain plan for select * from dfs.`/Users/asinha/data/multilevel/parquet` where dir0 in (1995, 1996);
> ...
>  "graph" : [ {
>     "pop" : "parquet-scan",
>     "@id" : 4,
>     "entries" : [ {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet"
>     }, {
>       "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet"
>     } ],



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)