You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Chinna Rao Lalam (JIRA)" <ji...@apache.org> on 2016/02/18 15:14:18 UTC

[jira] [Updated] (HIVE-12077) MSCK Repair table should fix partitions in batches

     [ https://issues.apache.org/jira/browse/HIVE-12077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chinna Rao Lalam updated HIVE-12077:
------------------------------------
    Attachment: HIVE-12077.1.patch

> MSCK Repair table should fix partitions in batches 
> ---------------------------------------------------
>
>                 Key: HIVE-12077
>                 URL: https://issues.apache.org/jira/browse/HIVE-12077
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Ryan P
>         Attachments: HIVE-12077.1.patch
>
>
> If a user attempts to run MSCK REPAIR TABLE on a directory with a large number of untracked partitions HMS will OOME. I suspect this is because it attempts to do one large bulk load in an effort to save time. Ultimately this can lead to a collection so large in size that HMS eventually hits an Out of Memory Exception. 
> Instead I suggest that Hive include a configurable batch size that HMS can use to break up the load. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)