[I] Provide indication of intermediate compactions to CompactionConfigurer. [accumulo]

["keith-turner (via GitHub)" <gi...@apache.org>]

keith-turner opened a new issue, #3937:
URL: https://github.com/apache/accumulo/issues/3937

   **Is your feature request related to a problem? Please describe.**
   
   User compactions of a tablet can be scheduled over multiple compaction jobs.  For example if a tablet has 50 files and a use compaction is initiated for it, then the following could happen.
   
    1. 50 files are selected for user compaction
    2. A compaction job compacts 30 of the 50 files. On completion 30 files are removed from selected set and one is added.  So now there are 21 files in the selected set.
    3. A compaction job compacts the remaining 21 files, completing the user compaction for the tablet.
   
   If a user sets per compaction config to use expensive compression, they may not want to apply this to the intermediate compaction in step 2 above as this is a short lived file.  In the CompactionConfigurer there is currently no way for it to know if a compaction is intermediate or not.
   
   **Describe the solution you'd like**
   Add something to [CompactionConfigurer.InputParameters](https://github.com/apache/accumulo/blob/3f51b5e0caedae759eb6ba6a28a0ae8f7bc9d0f2/core/src/main/java/org/apache/accumulo/core/client/admin/compaction/CompactionConfigurer.java#L51) that can indicate if a compaction is intermediate or not.  One possible way to do this would be to add a method like the following to InputParameters.
   
   ```java
   /**
    * If this a user compaction, then returns the selected set of files.  For user compactions, when
    *  getInputFiles().equals(getSelectedFiles()) is true then this is the final compaction in user compaction, 
    * when its not true then this is an intermediate compaction.  For system compactions there is no selected set of files so the empty set is returned.
    */
   public Collection<CompactableFile> getSelectedFiles();
   ```
   
   **Describe alternatives you've considered**
   
   Initially considered adding a method that returns a boolean to indicate if the compaction is intermediate of not.  Thought  making the set of selected files could be more generally useful while still allowing to be known if a compaction is intermediate or not.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Provide indication of intermediate compactions to CompactionConfigurer. [accumulo]

["keith-turner (via GitHub)" <gi...@apache.org>]

keith-turner closed issue #3937: Provide indication of intermediate compactions to CompactionConfigurer.
URL: https://github.com/apache/accumulo/issues/3937


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Provide indication of intermediate compactions to CompactionConfigurer. [accumulo]

["kevinrr888 (via GitHub)" <gi...@apache.org>]

kevinrr888 commented on issue #3937:
URL: https://github.com/apache/accumulo/issues/3937#issuecomment-1856432137

   I would like to work on this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org