You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/07/25 17:14:20 UTC
[jira] [Commented] (DRILL-4786) Improve metadata cache performance
for queries with multiple partitions
[ https://issues.apache.org/jira/browse/DRILL-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392312#comment-15392312 ]
ASF GitHub Bot commented on DRILL-4786:
---------------------------------------
GitHub user amansinha100 opened a pull request:
https://github.com/apache/drill/pull/553
DRILL-4786: Read the metadata cache file from the least common ancest…
…or directory when multiple partitions are selected.
Handle wildcards appropriately when metadata cache is present.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amansinha100/incubator-drill DRILL-4786-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/553.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #553
----
commit 002898ac21738cdcd9f367969d91fa7b7c5b2a3c
Author: Aman Sinha <as...@maprtech.com>
Date: 2016-07-22T23:42:09Z
DRILL-4786: Read the metadata cache file from the least common ancestor directory when multiple partitions are selected.
Handle wildcards appropriately when metadata cache is present.
----
> Improve metadata cache performance for queries with multiple partitions
> -----------------------------------------------------------------------
>
> Key: DRILL-4786
> URL: https://issues.apache.org/jira/browse/DRILL-4786
> Project: Apache Drill
> Issue Type: Improvement
> Components: Metadata, Query Planning & Optimization
> Affects Versions: 1.7.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
>
> Consider queries of the following type run against Parquet data with metadata caching:
> {noformat}
> SELECT col FROM `A` WHERE dir0 = 'B`' AND dir1 IN ('1', '2', '3')
> {noformat}
> For such queries, Drill will read the metadata cache file from the top level directory 'A', which is not very efficient since we are only interested in the files from some subdirectories of 'B'. DRILL-4530 improves the performance of such queries when the leaf level directory is a single partition. Here, there are 3 subpartitions due to the IN list. We can build upon the DRILL-4530 enhancement by at least reading the cache file from the immediate parent level `/A/B` instead of the top level.
> The goal of this JIRA is to improve performance for such types of queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)