[GitHub] [accumulo] drewfarris commented on issue #564: Add multiple compaction thread pools and allow multiple compactions per tablet

[GitBox <gi...@apache.org>]

drewfarris commented on issue #564: Add multiple compaction thread pools and allow multiple compactions per tablet
URL: https://github.com/apache/accumulo/issues/564#issuecomment-564131375
 
 
   I think this is looking good as well, just a couple questions:
   
   Have you thought much about how the `CompactionManager` decides to perform work or decides to cancel scheduled work? Is this implemented via some sort of pluggable strategy and does this mean that there's a new interface we need to design to support this?
   
   I could be misunderstanding the role of the `CompactionPrioritizer` - it seems that this class establishes priorities for jobs that have already been formulated, but perhaps this class actually creates the jobs to run by looking at a set of files to potentially compact?
   
   I like the idea of moving large or compute-intensive jobs out of the tserver process. Interested to see how managing external compactions works out.
   
   In the documentation for `CompactionExecutor` you mention that if a job is too big it will be processed in multiple passes. I wonder how much complexity this will add to the code - perhaps it will be more straightforward if the executor can split jobs - with the notion that an entire job either succeeds or fails, but there's no partial success state?
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services