You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2021/09/21 20:21:00 UTC
[jira] [Assigned] (HUDI-2477) Restore fails after adding rollback
plan and rollback.requested instant
[ https://issues.apache.org/jira/browse/HUDI-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan reassigned HUDI-2477:
-----------------------------------------
Assignee: sivabalan narayanan
> Restore fails after adding rollback plan and rollback.requested instant
> -----------------------------------------------------------------------
>
> Key: HUDI-2477
> URL: https://issues.apache.org/jira/browse/HUDI-2477
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
>
> restore triggers rollback of N commits and then finally will commit the restore. None of rollbacks will be published to timeline.
> But after we have added the rollback.requested instant, restore is breaking w/ metadata enabled.
> Here is what is happening:
> Restore
> schedule rollback for all of N commits. this will produce rollback.requested instants to timeline. Remember we can't skip this publishing, bcoz, rollback action executor depends on this.
> trigger rollback action executor. which will execute the rollback. but this time we may not publish the rollbacks. and so there won't be a rollback completed instant.
> and now to finalize restore, we apply the changes to metadata table before we can commit the restore to datatable. Here is where the issue is. We do check if bootstrapping is required. chances that last synced instant to metadata table is not active anymore in data table and so it triggers a bootstrap. but we do allow bootstrap only if there are no pending operations in datatable. But all rollbacks are surfacing as pending operations and hence we fail here.
>
> This could also be an issue when we try to play with bootstrap in original dataset.
> bootstrap. and for some reason you want to rollback bootstrap. this might end up in this state too.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)