[jira] [Commented] (FALCON-703) Monitoring plugin that registers catalog partition

["Shwetha G S (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128371#comment-14128371 ] 

Shwetha G S commented on FALCON-703:
------------------------------------

[~svenkat], Can we use HiveMetaStoreClient instead of HCatClient in HiveCatalogService? HCatClient.dropPartitions() deletes the data for the partition even for external tables. HiveMetaStoreClient.dropPartition() has an option of not deleting the data. I need drop partition without deleting the data

> Monitoring plugin that registers catalog partition
> --------------------------------------------------
>
>                 Key: FALCON-703
>                 URL: https://issues.apache.org/jira/browse/FALCON-703
>             Project: Falcon
>          Issue Type: New Feature
>            Reporter: Shwetha G S
>            Assignee: Shwetha G S
>
> During migration from filesystem based feeds to hcatalog based feeds, there are some cases which are difficult to handle. For example, if any hourly feed is updated at 10th hour to hcatalog, how do we update the dependent process that consumes the whole day of data of that feed. Some instances of that feed for that day will be on hcatalog and some won't be. 
> If we can de-couple the partition registration and assume that all partitions are already on hcatalog even before the migration, then the whole hcatalog migration becomes easier and the feeds can be migrated independently. We can do hcatalog partition registration by listening to jms message for succeeded workflow. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)