You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Subramanyam Pattipaka (JIRA)" <ji...@apache.org> on 2017/02/03 19:56:52 UTC

[jira] [Created] (HIVE-15807) MSCK operations hangs in HiveMetaStoreChecker.checkPartitionDirs

Subramanyam Pattipaka created HIVE-15807:
--------------------------------------------

             Summary: MSCK operations hangs in HiveMetaStoreChecker.checkPartitionDirs
                 Key: HIVE-15807
                 URL: https://issues.apache.org/jira/browse/HIVE-15807
             Project: Hive
          Issue Type: Bug
    Affects Versions: 2.2.0
            Reporter: Subramanyam Pattipaka
            Assignee: Pengcheng Xiong
             Fix For: 2.2.0


The seems to be a regression from HIVE-14511. The operation was hung in checkPartitionDirs. The data has 3 levels of partitions (month, date, id) which has total of 800 partitions.

An example path would look like month=9/day=30/id=12

The default value for hive config hive.mv.files.thread was set to 128. I have attached the jstack of hive process used to run msck command

checkPartitionDirs is implemented as recursive function which uses same pool to submit worker threads. It seems thread pool ran out of thread to do the actual work and all threads seems to be waiting and hung. Please take a look the stack and confirm if this is the case here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)