You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@drill.apache.org by "Miroslav Holubec (JIRA)" <ji...@apache.org> on 2016/08/17 09:05:20 UTC

[jira] [Commented] (DRILL-4849) Refresh table metadata performance: read only new/updated parquet files

    [ https://issues.apache.org/jira/browse/DRILL-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424170#comment-15424170 ] 

Miroslav Holubec commented on DRILL-4849:
-----------------------------------------

patch on branch including DRILL-4601:
https://github.com/myroch/drill/commit/14c499ab74538f618952b0578a72b9f23785fe5d

m.

> Refresh table metadata performance: read only new/updated parquet files
> -----------------------------------------------------------------------
>
>                 Key: DRILL-4849
>                 URL: https://issues.apache.org/jira/browse/DRILL-4849
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Query Planning & Optimization
>    Affects Versions: 1.7.0
>            Reporter: Miroslav Holubec
>              Labels: metadata, parquet, performance, planning
>
> Currently REFRESH TABLE METADATA takes serious amount of time for many small parquet files. We can actually only read these parquets, which are new or changed. This will require to add modificationTime into file metadata.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)