You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "bjgrunner (via GitHub)" <gi...@apache.org> on 2023/12/15 15:47:10 UTC

[I] [SUPPORT] update partition key value [hudi]

bjgrunner opened a new issue, #10335:
URL: https://github.com/apache/hudi/issues/10335

   We are using Spark 3.4 and scala 3.12, and hudi-spark3.4-bundle_2.12.
   
   I have a table partitioned by event_date, and have another column called previous_event_date. The unique key field is event_id, via bloom index. 
   
   Suppose I have a record with:
   
   Event_id = 1
   Event_date = 14dec2023
   Previous_event_date = 13dec2023
   
   A new record comes in for upsert:
   
   Event_id = 1
   Event_date = 15dec2023
   Previous_event_date = 14dec2023
   
   This is a change to the existing record, but has a new event date. Is there a way I can tell Hudi to do something like:
   
   Update table
   Set event_date=15dec2023, previous_date = 14dec2023
   Where event_id = 1
   And event_date = new record previous event date (14dec2023)
   
   I'm basically wanting to move the record from one partition to another, without having to use a global index, or perform a delete first followed by upsert.
   
   I hope this is clear, I tried my best to explain the requirement.
   
   Thanks. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] update partition key value [hudi]

Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #10335:
URL: https://github.com/apache/hudi/issues/10335#issuecomment-1875573250

   @bjgrunner Feel free to close this issue in case you are good with explanation, else let us know. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] update partition key value [hudi]

Posted by "codope (via GitHub)" <gi...@apache.org>.
codope closed issue #10335: [SUPPORT] update partition key value 
URL: https://github.com/apache/hudi/issues/10335


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] update partition key value [hudi]

Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #10335:
URL: https://github.com/apache/hudi/issues/10335#issuecomment-1862516035

   @bjgrunner I understand your requirement, But without using GLOBAL index, Hudi will consider these records as new record only as they fall in different partition. So I don't think you will be able to achieve it without global index, or perform a delete first followed by INSERT. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] update partition key value [hudi]

Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #10335:
URL: https://github.com/apache/hudi/issues/10335#issuecomment-1884493492

   @bjgrunner Closing this. Feel free to reopen or create a new one for any further queries.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org