[jira] [Comment Edited] (HIVE-14840) MSCK not adding the missing partitions to Hive Metastore when the partition names are not in lowercase

["Sushil Kumar S (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15632759#comment-15632759 ] 

Sushil Kumar S edited comment on HIVE-14840 at 11/3/16 1:47 PM:
----------------------------------------------------------------

[~sershe] Hey, since this is due to FS case sensitivity , am planning to move this ticket as an improvement where I am thinking of adding the partitions that are not in Metastore as follows :

For all the partitions that are not in Metastore from MSCK:
  - check if the partition name matches with the name stored in the Metastore ignoring the case 
     - if matches add to a temporary list else ignore that path

Finally bulk add all the partitions from the temporary list to Metastore with the case-sensitive path corresponding to that parition as location

Kindly, give me your suggestion. 
   


was (Author: sushil-k-s):
[~sershe] Hey, since this is a because of FS case sensitivity , am planning to move this ticket as an improvement where I am thinking of adding the partitions that are not in Metastore as follows :

For all the partitions that are not in Metastore from MSCK:
  - check if the partition name matches with the name stored in the Metastore ignoring the case 
     - if matches add to a temporary list else ignore that path

Finally bulk add all the partitions from the temporary list to Metastore with the case-sensitive path corresponding to that parition as location

Kindly, give me your suggestion. 
   

> MSCK not adding the missing partitions to Hive Metastore when the partition names are not in lowercase
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-14840
>                 URL: https://issues.apache.org/jira/browse/HIVE-14840
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 1.2.1
>            Reporter: Sushil Kumar S
>            Assignee: Sushil Kumar S
>            Priority: Minor
>              Labels: hive
>
> Hi,
>   There's is a bug while running MSCK REPAIR TABLE EXTERNAL_TABLE_NAME on Hive 1.2.1, all the partition that are not present in the metastore are being listed but not added if the partition names are not in lowercase, in other words if an external path has a camel case based name and value i.e s3n://some_external_path/myPartition=01 it just gets listed as partition not found in metastore but doesn’t add it.
> However, am not able to run ALTER TABLE MY_EXTERNAL_TABLE RECOVER PARTITIONS; on hive 1.2 and based on the source code from hive-exec am able to see under org/apache/hadoop/hive/ql/parse/HiveParser.g:1001:1 that there's no token matching in the grammar for RECOVER PARTITIONS.
> Example:
> - When external path = s3n://some_external_path/myPartition=01
>        hive> MSCK REPAIR TABLE my_external_table;
>        Partitions not in metastore: my_external_table:mypartition=01
>        Time taken: 1.729 seconds, Fetched: 2 row(s)
>         hive> show partitions foster.ola_raven_raven_users_raw;
>         OK
>         Time taken: 0.901 seconds, Fetched: 0 row(s)
> - When external path = s3n://some_external_path/mypartition=01
>         hive> MSCK REPAIR TABLE my_external_table;
>         Partitions not in metastore: my_external_table:mypartition=01
>         Repair: Added partition to metastore my_external_table:mypartition=01
>         Time taken: 1.729 seconds, Fetched: 2 row(s)
>          hive> show partitions my_external_table;
>          OK
>          mypartition=01
>          Time taken: 1.101 seconds, Fetched: 1 row(s)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)