You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Aman Sinha (JIRA)" <ji...@apache.org> on 2014/10/21 04:16:33 UTC
[jira] [Resolved] (DRILL-1549) Support partition pruning for simple
IN filters
[ https://issues.apache.org/jira/browse/DRILL-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aman Sinha resolved DRILL-1549.
-------------------------------
Resolution: Fixed
Target Version/s: 0.7.0
Fixed in commit #: 8dedd7182
> Support partition pruning for simple IN filters
> -----------------------------------------------
>
> Key: DRILL-1549
> URL: https://issues.apache.org/jira/browse/DRILL-1549
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 0.6.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Fix For: 0.7.0
>
> Attachments: 0001-DRILL-1549-Partition-pruning-for-simple-IN-filters.patch
>
>
> Queries containing directory filters of the form 'dir0 IN (1, 2)' currently don't get considered for partition pruning due to a bug inside the visitor of DirPathBuilder. This JIRA is to address the issue for basic IN filters, which actually is converted to an OR condition by the optimizer (dir0 = 1 OR dir1 = 2). Note that this JIRA does not cover the more general IN filters which are combined with other ANDs etc.
> To illustrate, here's the restricted case that we want to fix here: (the explain shows that the scan is reading all directories, not just 1995, 1996):
> explain plan for select * from dfs.`/Users/asinha/data/multilevel/parquet` where dir0 in (1995, 1996);
> ...
> "graph" : [ {
> "pop" : "parquet-scan",
> "@id" : 4,
> "entries" : [ {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet"
> }, {
> "path" : "file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet"
> } ],
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)