You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Narayanan Venkateswaran (Jira)" <ji...@apache.org> on 2021/06/02 11:06:00 UTC

[jira] [Created] (HIVE-25187) Reduce number of getPartition calls during loadDynamicPartitions for Managed Tables

Narayanan Venkateswaran created HIVE-25187:
----------------------------------------------

             Summary: Reduce number of getPartition calls during loadDynamicPartitions for Managed Tables
                 Key: HIVE-25187
                 URL: https://issues.apache.org/jira/browse/HIVE-25187
             Project: Hive
          Issue Type: Bug
          Components: Hive
            Reporter: Narayanan Venkateswaran


When dynamic partitions are loaded, Hive::loadDynamicPartition loads all partitions from HMS causing heavy load on it. This becomes worse when large number of partitions are present in tables.

Only relevant partitions being loaded in dynamic partitions can be queried from HMS for partition existence.

[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2958]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)