[GitHub] [hudi] xushiyan edited a comment on pull request #4565: [HUDI-3215] Solve UT for Spark 3.2

[GitBox <gi...@apache.org>]

xushiyan edited a comment on pull request #4565:
URL: https://github.com/apache/hudi/pull/4565#issuecomment-1022673067


   > > @YannByron let me clarify my thoughts:
   > > 
   > > * event time field can be different from ordering / precombine field: the former is used to calculate latency/freshness; it can be another field say `occurred_at` and the latency will be calculated as `commit_time - occurred_at`, while the records are ordered or precombined using `updated_at`
   > > * there are users who need to set event time and there are those who don't care. For the latter, we can't just set it as the ordering field; we should leave it not set and then nothing will be calculated or persisted in commit metadata
   > > * as you reported "setting returnNullIfNotFound to true does not take effect", then we should fix that instead of masking it by setting event time
   > > 
   > > WDYT?
   > 
   > @xushiyan I remove the codes about setting event.time by precombine.field, but I keep setting ordering.field by precombine.field, is it right? And, for "setting returnNullIfNotFound to true does not take effect", I can't fix it soon. So I just catch the exception in this case to make it work. please review again.
   
   @YannByron yes we should tackle `returnNullIfNotFound` as a follow-up issue and PR. Filed https://issues.apache.org/jira/browse/HUDI-3333


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org