You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/10/12 01:27:00 UTC

[jira] [Commented] (HIVE-20707) Automatic partition management

    [ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647268#comment-16647268 ] 

Prasanth Jayachandran commented on HIVE-20707:
----------------------------------------------

There is more scope added to this patch

1) Support managed and external tables. Managed tables will get X lock for MSCK.

2) Retention period for partitions can be specified via table property.

3) MSCK repair can now handle retention periods via config.

4) All external tables will now automatically enable partition discovery.

On top of these there are fixes related to ACID, MSCK was acquiring wrong lock, explain locks was not working for msck, fixes related to catalog handling (some metastore APIs was not handling catalog names correctly).

> Automatic partition management
> ------------------------------
>
>                 Key: HIVE-20707
>                 URL: https://issues.apache.org/jira/browse/HIVE-20707
>             Project: Hive
>          Issue Type: New Feature
>    Affects Versions: 4.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Major
>         Attachments: HIVE-20707.1.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK REPAIR command has to be executed manually. To avoid this manual step, external tables can be specified a table property based on which a background metastore thread can sync partitions periodically. Tables can also be specified with partition retention period. Any partition whose age exceeds the retention period will be dropped automatically.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)