You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "TheR1sing3un (via GitHub)" <gi...@apache.org> on 2025/03/19 02:57:22 UTC
[I] [DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ? [hudi]
TheR1sing3un opened a new issue, #12997:
URL: https://github.com/apache/hudi/issues/12997
From the current code structure, we would treat the merge policy of `COMMIT_TIME_ORDERING` as a separate logic, but from a business perspective, should we treat it as a special case of `EVENT_TIME_ORDERING` where the ORDERING VALUE for each record is the same? For example, right now it's represented by `int: 0`. This way we don't need to maintain two merge policies, we default to `EVENT_TIME_ORDERING`, the same record_key merge policy is handled in the order of `transaction_time` and `event_time`. This will help us in the future to maintain the code and deal with the various merge problems we have encountered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ? [hudi]
Posted by "TheR1sing3un (via GitHub)" <gi...@apache.org>.
TheR1sing3un commented on issue #12997:
URL: https://github.com/apache/hudi/issues/12997#issuecomment-2735238725
> user want to force the merging to be based on input sequence.
If according to what I said above, when we process duplicate record, we update it in turn according to the input sequence, can we also achieve this effect?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ? [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #12997:
URL: https://github.com/apache/hudi/issues/12997#issuecomment-2735228746
Theoratically `COMMIT_TIME_ORDERING` is a special case of `EVENT_TIME_ORDERING`, the reason we have a explicit option for commit time sequence is that user want to force the merging to be based on input sequence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ? [hudi]
Posted by "TheR1sing3un (via GitHub)" <gi...@apache.org>.
TheR1sing3un commented on issue #12997:
URL: https://github.com/apache/hudi/issues/12997#issuecomment-2735198627
@yihua @danny0405 I'd love to hear what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ? [hudi]
Posted by "TheR1sing3un (via GitHub)" <gi...@apache.org>.
TheR1sing3un commented on issue #12997:
URL: https://github.com/apache/hudi/issues/12997#issuecomment-2735234924
in addition, I think we can treat all log records with the same `record_key` as state transition logs similar to state machines (this record's value), and we can handle them sequently according to the actual time of occurrence, i.e., 'instant_time' or 'completion_time in NBCC'. Then 'event_time' or 'ordering value' is our out-of-order processing logic in state transformation. A higher 'ordering value' can allow state transformation, that is, data update. A lower 'odering value' is considered an out-of-order log and can be discarded directly. I think this kind of processing logic can avoid some of the current problems with things
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org