[GitHub] [hudi] vinothchandar edited a comment on issue #1764: [SUPPORT] Commits stays INFLIGHT forever after S3 consistency check fails when Hudi tries to delete duplicate datafiles

[GitBox <gi...@apache.org>]

vinothchandar edited a comment on issue #1764:
URL: https://github.com/apache/hudi/issues/1764#issuecomment-648882567


   @zuyanton thanks for reporting this.. let's work together to resolve this. can you please paste the `.hoodie/hoodie.properties` file? The .inflight file hanging around could be normal or not, depending on the timeline layout version.. So checking for that. 
   
   The consistency check itself is tunable.. See https://github.com/apache/hudi/blob/release-0.5.3/hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java#L92 although I would expect the defaults to be sufficient most of the time (have not seen a lot of issues like this so far).. cc @umehrot2  any suggestions here? 
   
   On the second problem, 
   >like log files with the same fileID as parquet files that were part of failed compaction never get compacted,
   The next run of compaction should try and attempt to complete this inflight compaction again.. cc @bvaradar can you confirm 
   
   a) with inline compaction, a failed compaction would be re-attempted in the next run 
   b) we will perform clean up from .aux for all marker files that we may not have deleted yet (due to such finalize errors).. 
    
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org