You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Chinmay Kulkarni (Jira)" <ji...@apache.org> on 2021/08/17 19:34:00 UTC
[jira] [Created] (HIVE-25463) Add support to standalone Hive
metastore for bulk dropping partitions out of the box
Chinmay Kulkarni created HIVE-25463:
---------------------------------------
Summary: Add support to standalone Hive metastore for bulk dropping partitions out of the box
Key: HIVE-25463
URL: https://issues.apache.org/jira/browse/HIVE-25463
Project: Hive
Issue Type: Improvement
Components: Standalone Metastore
Affects Versions: 3.1.2
Reporter: Chinmay Kulkarni
Today for setups that use standalone metastore, we set the following:
{noformat}
<property>
<name>metastore.expression.proxy</name>
<value>org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy</value>
</property>
{noformat}
However, [DefaultPartitionExpressionProxy|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DefaultPartitionExpressionProxy.java] throws UnsupportedOperationException throughout, thus preventing any partition pruning to be successful.
The effect of this is, other projects that rely on just standalone Hive Metastore (for example Trino) are unable to do things like invoke bulk partition drops using say, [HiveMetaStore.drop_partitions_req|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L5008] via a Thrift client.
We should implement a separate PartitionExpressionProxy implementation for such clients for use with standalone HMS. Note that an implementation like PartitionExpressionForMetastore cannot be used in such cases since standalone HMS installs will/should not have hive-exec dependencies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)