You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Vinoth Chandar (Jira)" <ji...@apache.org> on 2019/10/21 00:42:00 UTC

[jira] [Updated] (HUDI-169) Hoodie 2.0 Rollback could be slow if there are too many files needed to rollback for a partition

     [ https://issues.apache.org/jira/browse/HUDI-169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vinoth Chandar updated HUDI-169:
--------------------------------
    Fix Version/s: 0.5.1

> Hoodie 2.0 Rollback could be slow if there are too many files needed to rollback for a partition
> ------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-169
>                 URL: https://issues.apache.org/jira/browse/HUDI-169
>             Project: Apache Hudi (incubating)
>          Issue Type: Task
>          Components: Common Core, Performance
>            Reporter: Balaji Varadarajan
>            Assignee: Balaji Varadarajan
>            Priority: Major
>             Fix For: 0.5.1
>
>
> The logic which rolls-back a commit performs parallel rollback  only at partition level.  Depending on the input batch, there could be skew in number of files per partition that needs to be rolled-back.
> For partition with too many files to be rolled-back, each rollback appends a log-block to the file-slice. This could potentially take a long time as they are done serially.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)