You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Balazs Jeszenszky (JIRA)" <ji...@apache.org> on 2018/05/08 19:55:00 UTC

[jira] [Created] (IMPALA-6994) Avoid reloading a table's HMS data

Balazs Jeszenszky created IMPALA-6994:
-----------------------------------------

             Summary: Avoid reloading a table's HMS data
                 Key: IMPALA-6994
                 URL: https://issues.apache.org/jira/browse/IMPALA-6994
             Project: IMPALA
          Issue Type: Improvement
          Components: Frontend
    Affects Versions: Impala 2.12.0
            Reporter: Balazs Jeszenszky


Reloading file metadata for HDFS tables (e.g. as a final step in an 'insert') is done via
https://github.com/apache/impala/blob/branch-2.12.0/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L628
, which calls
https://github.com/apache/impala/blob/branch-2.12.0/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java#L1243

HdfsTable.load has no option to only load file metadata. HMS metadata will also be reloaded every time, which is an unnecessary overhead (and potential point of failure) when adding files to existing locations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)