You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Y Ethan Guo (Jira)" <ji...@apache.org> on 2024/12/13 01:27:00 UTC

[jira] [Commented] (HUDI-8425) Revisit precombine/ordering value passing in the file group reader and record buffer

    [ https://issues.apache.org/jira/browse/HUDI-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905337#comment-17905337 ] 

Y Ethan Guo commented on HUDI-8425:
-----------------------------------

This is properly handled and any improvement will be tackled by HUDI-8682.

> Revisit precombine/ordering value passing in the file group reader and record buffer
> ------------------------------------------------------------------------------------
>
>                 Key: HUDI-8425
>                 URL: https://issues.apache.org/jira/browse/HUDI-8425
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Y Ethan Guo
>            Priority: Blocker
>             Fix For: 1.0.1
>
>
> [https://github.com/apache/hudi/pull/11943#discussion_r1805676748]
> Currently we set default ordering value if the precombine/ordering field is not set, so the natural ordering is used (based on processing or instant time).  This is fine given it's only used on the reader path.  However, there's a risk of writing the default ordering field to the storage if the file group reader is used on the writer side for compaction, clustering, etc.  We should revisit precombine/ordering value passing in the file group reader and record buffer to see if we can get rid of such default ordering value and let the record merger handle the cases that the ordering value is null / not present directly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)