[jira] Updated: (HIVE-142) Create a metastore check command

["Johan Oskarsson (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johan Oskarsson updated HIVE-142:
---------------------------------

    Attachment: HIVE-142.patch

This is a rough first patch for this issue, I'd appreciate it if someone could have a quick look at it. I'll do some more testing, touch up some javadoc and other minor changes but beyond that it's fairly complete for this first basic version including unit tests. I'd rather not add any other features to it for this ticket.

The patch have bits of HIVE-126 and HIVE-182 in it that are required. Once those are both committed I'll roll a new version of this patch without the overlap. If someone would have time to commit 126 and 182 soon it would be much appreciated.

I have also changed HashMap and AbstractMap to Map in a few places, I can separate that out in a new issue if required.

> Create a metastore check command
> --------------------------------
>
>                 Key: HIVE-142
>                 URL: https://issues.apache.org/jira/browse/HIVE-142
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Metastore
>    Affects Versions: 0.19.0
>            Reporter: Johan Oskarsson
>            Assignee: Johan Oskarsson
>             Fix For: 0.19.0
>
>         Attachments: HIVE-142.patch
>
>
> We need a command to verify that the information in the metastore reflects the data that is on hdfs. For example partitions can be deleted on hdfs but still be in the metastore.
> From Joydeep Sen Sarma, see ticket HIVE-126 for the full comment:
> for a command line interface - one might want to check the entire database or just a table or even just one partition. other metadata checks will also be added over time (for example - do the file types on disk agree with metadata records, bucketing information etc). So, here's a strawman proposal for a new command:
> alter table <DB>[.TABLE [PARTITION-SPEC]] check [TYPE-LIST]
> where TYPE by default is 'all' (check for all kinds of errors), but can be specified to a specific type. For example - in this case - we can have a type called 'partitions' (and then over time we can add other types like 'fileformat' etc.). for v1 - we can just drop the type-list altogether.
> the check command can produce a list of things that need to be done to fix the format (like adding any directories not in the metastore - but in hdfs - to the metastore). actually performing of such steps would require a user confirmation (y/n).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.