You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Szehon Ho (Jira)" <ji...@apache.org> on 2020/10/12 11:29:00 UTC

[jira] [Created] (HIVE-24263) Create an HMS endpoint to list partition locations

Szehon Ho created HIVE-24263:
--------------------------------

             Summary: Create an HMS endpoint to list partition locations
                 Key: HIVE-24263
                 URL: https://issues.apache.org/jira/browse/HIVE-24263
             Project: Hive
          Issue Type: Improvement
          Components: Standalone Metastore
            Reporter: Szehon Ho


In our company, we have a use-case to get quickly a list of partition locations.  Currently it is done via listPartitions, which is a very heavy operation in terms of memory and performance.

For example, we have an integration from output of a Hive pipeline to Spark jobs that consume directly from HDFS.  It needs to know the partition paths that are available for consumation, and does repeated listPartitions() for this.

As there is already an internal method in the ObjectStore for this done for dropPartitions, it is only a matter of exposing this API to HiveMetaStoreClient.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)