You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Arjun (JIRA)" <ji...@apache.org> on 2018/07/26 18:27:00 UTC

[jira] [Updated] (DRILL-6640) Drill takes long time in planning when there are large number of files in views/tables DFS parent directory

     [ https://issues.apache.org/jira/browse/DRILL-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arjun updated DRILL-6640:
-------------------------
    Description: 
When Drill is used for querying views/ tables, the query planning time increases as the number of files in views/tables parent directory increases. This becomes unacceptably long with complex queries.

This is caused by globStatus operation on view files using GLOB to retrieve view file status. This can be improved by avoiding the usage of GLOB pattern for Drill metadata files like view files.

  was:
When Drill is used for querying views/ tables, the query planning time increases as the number of files in views/tables parent directory increases. This becomes unacceptably long with complex queries that involve multiple joins between such views/tables.

This is caused by globStatus operation on view files using GLOB to retrieve view file status. This can be improved by avoiding the usage of GLOB pattern for Drill metadata files like view files.


> Drill takes long time in planning when there are large number of files in  views/tables DFS parent directory
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6640
>                 URL: https://issues.apache.org/jira/browse/DRILL-6640
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Query Planning &amp; Optimization
>            Reporter: Arjun
>            Assignee: Arjun
>            Priority: Major
>
> When Drill is used for querying views/ tables, the query planning time increases as the number of files in views/tables parent directory increases. This becomes unacceptably long with complex queries.
> This is caused by globStatus operation on view files using GLOB to retrieve view file status. This can be improved by avoiding the usage of GLOB pattern for Drill metadata files like view files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)