You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2023/08/15 03:32:00 UTC

[jira] [Closed] (HUDI-6654) Add new log block header type to store record positions

     [ https://issues.apache.org/jira/browse/HUDI-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo closed HUDI-6654.
---------------------------
    Resolution: Fixed

> Add new log block header type to store record positions
> -------------------------------------------------------
>
>                 Key: HUDI-6654
>                 URL: https://issues.apache.org/jira/browse/HUDI-6654
>             Project: Apache Hudi
>          Issue Type: New Feature
>            Reporter: Ethan Guo
>            Assignee: Ethan Guo
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.14.0
>
>
> To support position-based merging of base and log files, we need to encode positions in the log blocks so that the positions can be used directly, without having to deserialize records or delete keys for OverwriteWithLatest payload, or with ordering values required only for `DefaultHoodieRecordPayload` supporting event time based streaming.  We add a new `HeaderMetadataType` to store the positions in the log block header.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)