[jira] [Comment Edited] (HUDI-499) Allow partition path to be updated with GLOBAL_BLOOM index

["sivabalan narayanan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008135#comment-17008135 ] 

sivabalan narayanan edited comment on HUDI-499 at 1/4/20 7:32 PM:
------------------------------------------------------------------

yes, you are right. basically the HoodieRecord returned from this method will be sent to upsert. If payload is empty, then its considered as delete, if not it could be an insert or update. 

There is also another api in HoodieWriteClient dedicated just for deletes, which takes in JavaRDD<HoodieKey>. But for your impl, guess you have to go with updating payload to Option.empty() during tag locations. 

 


was (Author: shivnarayan):
yes, you are right. basically the HoodieRecord returned from this method will be sent to upsert. If payload is empty, then its considered as delete, if not it could be an insert or update. 

> Allow partition path to be updated with GLOBAL_BLOOM index
> ----------------------------------------------------------
>
>                 Key: HUDI-499
>                 URL: https://issues.apache.org/jira/browse/HUDI-499
>             Project: Apache Hudi (incubating)
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Raymond Xu
>            Priority: Minor
>
> h3. Context
> When a record is to be updated with a new partition path, and when set to GLOBAL_BLOOM as index, the current logic implemented in [https://github.com/apache/incubator-hudi/pull/1091/] ignores the new partition path and update the record in the original partition path.
> h3. Proposed change
> Allow records to be inserted into their new partition paths and delete the records in the old partition paths. A configuration (e.g. {{hoodie.index.bloom.update.partitionpath=true}}) can be added to enable this feature.
> h4. An example use case
> A Hudi dataset manages people info and partitioned by birthday. In most cases, where people info are updated, birthdays are not to be changed (that's why we choose it as partition field). But in some edge cases where birthday info are input wrongly and we want to manually fix it or allow user to updated it occasionally. In this case, option 2 would be helpful in keeping records in the expected partition, so that a query like "show me people who were born after 2000" would work.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)