You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2022/02/23 16:26:00 UTC

[jira] [Assigned] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout

     [ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sagar Sumit reassigned HUDI-3356:
---------------------------------

    Assignee: Sagar Sumit  (was: Manoj Govindassamy)

> Conversion of write stats to metadata index records should use HoodieData throughout
> ------------------------------------------------------------------------------------
>
>                 Key: HUDI-3356
>                 URL: https://issues.apache.org/jira/browse/HUDI-3356
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: writer-core
>            Reporter: Manoj Govindassamy
>            Assignee: Sagar Sumit
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.11.0
>
>
> HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats to metadata index records to List of HoodieRecords before passing on them to engine specific commit() to prep records. This can OOM driver. We need to use HoodieData<HoodieRecords> throughout. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)