[jira] [Commented] (DRILL-6640) Drill takes long time in planning when there are large number of files in views/tables DFS parent directory

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559144#comment-16559144 ] 

ASF GitHub Bot commented on DRILL-6640:
---------------------------------------

kr-arjun opened a new pull request #1405: DRILL-6640: Modifying DotDrillUtil implementation to avoid using globStatus calls
URL: https://github.com/apache/drill/pull/1405
 
 
   Includes
   1.  Modified DotDrillUtil.getDotDrills implementation to avoid using FS globStatus call with GLOB for a given base file name.
   2. Added unit test cases for the new method.
   3. Code refactoring to include additional comments.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Drill takes long time in planning when there are large number of files in  views/tables DFS parent directory
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6640
>                 URL: https://issues.apache.org/jira/browse/DRILL-6640
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Query Planning & Optimization
>            Reporter: Arjun
>            Assignee: Arjun
>            Priority: Major
>             Fix For: 1.15.0
>
>
> When Drill is used for querying views/ tables, the query planning time increases as the number of files in views/tables parent directory increases. This becomes unacceptably long with complex queries.
> This is caused by globStatus operation on view files using GLOB to retrieve view file status. This can be improved by avoiding the usage of GLOB pattern for Drill metadata files like view files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)