[GitHub] [iceberg] ajantha-bhat commented on issue #4225: Compact the iceberg table of V2 to delete two kinds of deleting file and merge the some small datafile

[GitBox <gi...@apache.org>]

ajantha-bhat commented on issue #4225:
URL: https://github.com/apache/iceberg/issues/4225#issuecomment-1050608326


   @AlexLY123 :
   
   compaction already handles merging the delete files. but by default it is disabled. Need to configure the 
   `delete-file-threshold` to enable it.
   https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/actions/BinPackStrategy.java#L87-L88
   
   Also regarding major and minor compaction. Even though we don't have similar to hbase, we can run compaction based on the resource availability by selecting few files itself;
   Currently possible by using the `filter` method (where clause) of `rewrite_data_files` 
   
   Also lot of control is there by configuring parallelism and enabling partial commit during compaction etc.
   cc: @jackye1995 , @RussellSpitzer 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org