You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Aman Sinha (JIRA)" <ji...@apache.org> on 2015/10/07 19:09:26 UTC
[jira] [Resolved] (DRILL-3901) Performance regression with doing
Explain of COUNT(*) over 100K files
[ https://issues.apache.org/jira/browse/DRILL-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aman Sinha resolved DRILL-3901.
-------------------------------
Resolution: Fixed
Assignee: Aman Sinha (was: Mehant Baid)
Fix Version/s: 1.2.0
Fixed in 6cd399de8. I will create a separate JIRA for the metadata cache performance improvement.
> Performance regression with doing Explain of COUNT(*) over 100K files
> ---------------------------------------------------------------------
>
> Key: DRILL-3901
> URL: https://issues.apache.org/jira/browse/DRILL-3901
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.1.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Fix For: 1.2.0
>
> Attachments: 0001-DRILL-3901-Don-t-do-early-expansion-of-directory-in-.patch, 0001-DRILL-3901-Don-t-do-early-expansion-of-directory-in-.patch
>
>
> We are seeing a performance regression when doing an Explain of SELECT COUNT(*) over 100K files in a flat directory (no subdirectories) on latest master branch compared to a run that was done on Sept 26. Some initial details (I will have more later):
> {code}
> master branch on Sept 26
> No metadata cache: 71.452 secs
> With metadata cache: 15.804 secs
> Latest master branch
> No metadata cache: 110 secs
> With metadata cache: 32 secs
> {code}
> So, both cases show regression.
> [~mehant] and I took an initial look at this and it appears we might be doing the directory expansion twice.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)