[jira] [Created] (HIVE-5189) make batching in partition retrieval in metastore applicable to more methods

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

Sergey Shelukhin created HIVE-5189:
--------------------------------------

             Summary: make batching in partition retrieval in metastore applicable to more methods
                 Key: HIVE-5189
                 URL: https://issues.apache.org/jira/browse/HIVE-5189
             Project: Hive
          Issue Type: Improvement
            Reporter: Sergey Shelukhin


As indicated in HIVE-5158, Metastore can OOM if retrieving a large number of partitions. For client-side partition filtering, the client applies batching (that would avoid that) by sending parts of the filtered name list in separate request according to configuration.
The batching is not used on filter pushdown path, and when retrieving all partitions (e.g. when the pruner expression is not useful in non-strict mode). HIVE-4914 and pushdown improvements will make this problem somewhat worse by allowing more requests to go to the server.

There needs to be some batching scheme (ideally, a somewhat generic one) that would be applicable to all these paths.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira