[jira] [Commented] (HUDI-1014) Design and Implement upgrade-downgrade infrastrucutre

["sivabalan narayanan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166002#comment-17166002 ] 

sivabalan narayanan commented on HUDI-1014:
-------------------------------------------

With 0.6.0, Hoodie is switching to marker based rollback and hence location and format of marker files is going to differ. Hence a dataset created in pre 0.6.0 when launching for first time with 0.6.0 if having a pending commit that needs to be rolled back, we need some fix to be put in or in other words some upgrade steps need to be done. 

 
 * Collect into on all existing pending commit file slices. 
 * Delete old marker files and recreate new ones in new format (w/ IOType). 
 * Proceed on as usual. Every write operation will try to rollback any pending rollback and since marker based roll back is enabled and since marker files are in expected format, it should be taken care of automatically. 

 

Similarly, we might introduce a command in hudi-cli to downgrade from 0.6.0 to pre 0.6.0. Users need to do this downgrade step before launching hudi in an older version if their data set was created in 0.6.0. 

 

 

> Design and Implement upgrade-downgrade infrastrucutre
> -----------------------------------------------------
>
>                 Key: HUDI-1014
>                 URL: https://issues.apache.org/jira/browse/HUDI-1014
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: Common Core, Writer Core
>            Reporter: Vinoth Chandar
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>             Fix For: 0.6.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)