You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/08/05 08:57:00 UTC
[jira] [Commented] (SPARK-32541) MSCK REPAIR TABLE should drop the
missing partitions from metastore
[ https://issues.apache.org/jira/browse/SPARK-32541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171357#comment-17171357 ]
Apache Spark commented on SPARK-32541:
--------------------------------------
User 'lipzhu' has created a pull request for this issue:
https://github.com/apache/spark/pull/29358
> MSCK REPAIR TABLE should drop the missing partitions from metastore
> -------------------------------------------------------------------
>
> Key: SPARK-32541
> URL: https://issues.apache.org/jira/browse/SPARK-32541
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Zhu, Lipeng
> Priority: Major
>
> {{msck repair table <tablename>}} alter table <tablename> recover partitions is often used in environments where the new partitions are loaded as directories on HDFS and users want to create the missing partitions in bulk. However, currently it only supports addition of missing partitions. If there are any partitions which are present in metastore but not on the FileSystem, it should also delete them so that it truly repairs the table metadata.
> In Hive, it already support the grammar `msck repair table drop partitions`.
>
> https://issues.apache.org/jira/browse/HIVE-17824
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org