[GitHub] [hudi] XinyaoTian commented on pull request #6382: [HUDI-4612][RFC-59] RFC-59 Materials (RFC Proposal) Submission: "Multiple event_time Fields Latest Verification in a Single Table"

[GitBox <gi...@apache.org>]

XinyaoTian commented on PR #6382:
URL: https://github.com/apache/hudi/pull/6382#issuecomment-1236637259

   @prasannarajaperumal Hi Prasanna, thanks for your review :)
   I have read document of RFC-46 carefully according to your suggestion. 
   
   To my understanding, RFC-46 intends to improve the entire design of HoodieRecordPayload, which is extremely awesome and will provide quite a lot benefits. However, this doesn't give Hoodie the ability to verify multiple event-time fields in a single table (Although it may be easier to implement this feature by the new Payload design). What we would like to improve is to give Hudi the ability to JOIN multiple tables in stream-consuming mode without multiple event-time disordering. Therefore, I think we still need to propose this feature since it's really matter to have multiple event-time fields verification in a single Hudi table. 
   
   For your convenience, we can wait for the final landing of the RFC-46 and then implement the feature proposed in this RFC. I promise this feature is very important because people asking for this feature in many place (including Hudi Slack) almost every week. We really need to have MORE THAN ONE event-time fields so than we can ensure the accuracy of events even if there are many JOIN operations sinking to ONE Hudi table.
   
   If there's anything worth to note please contact me! Look forward to receiving your further feedback.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org