[GitHub] [hudi] zherenyu831 commented on issue #2707: [SUPPORT] insert_ovewrite_table failed on archiving

[GitBox <gi...@apache.org>]

zherenyu831 commented on issue #2707:
URL: https://github.com/apache/hudi/issues/2707#issuecomment-811565897


   @jsbali 
   For make everything clear:
   On archiving:
   Since the first replacecommit have 0 partitionToReplaceFileIds, and it failed at
   https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/ReplaceArchivalHelper.java#L73
   
   with error `Positive number of partitions required`
   
   Solution is 
   Manually delete first replacecommit or  ignore deletion on  0 partitionToReplaceFileIds replacecommit  in the code as your mentioned on your ticket
   
   After I deleted the first replacecommit, the archiving finished successfully 
   But I got another issue, 
   https://github.com/apache/hudi/issues/2707#issuecomment-804831651
   
   This is not related to what I deleted, commit time is different
   Once the commits has been deleted on archiving process, hudi tried to load timeline again without refresh it.
   
   I didn't debug more, since I want to ask the developers about how they want to deal with replaceFile deletion.
   seems like someone want to use cleaner to handle it.
   https://github.com/apache/hudi/issues/2707#issuecomment-804849028
   
   Unfortunately, I also found cleaner is not working well with insert_overwrite_table, 
   it only keep one file group
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org