[GitHub] [accumulo] keith-turner commented on pull request #1605: Fixes #564 adds support multiple compaction executors

[GitBox <gi...@apache.org>]

keith-turner commented on pull request #1605:
URL: https://github.com/apache/accumulo/pull/1605#issuecomment-625944170


   > Would there be value in marking the new properties/APIs as "experimental" until it bears out after a release or two?
   
   That would entail keeping all the old code and then jamming this new code into the old code.  I don't think that would help achieve stability or maintainability.  There are actually a lot of big changes that have built up for 2.1.0.  I think we need to stop accepting new features for 2.1.0 at some point and start heavily testing it and only fixing bugs.
   
   > There are so many new classes. The number of new classes might make the compaction code a bit more logical, but it's a big shift from the current code, and very hard for a newcomer to quickly familiarize themselves with all the code.
   
   Did you look at the javadoc link I posted? The current plugins break down by data and execution.  The CompactionSelector, CompactionConfigurer, and iterators are only concerned with user data and can be set per table or per compaction.  The CompactionPlanner and CompactionDispatcher are only concerned with executing compactions. Planners are configured at the system level and dispatchers are configured per table. Users can set execution hints when initiating a compaction through the API and these hints are made available to the dispatcher and planner.   If this is helpful, I can work this into the javadoc.
   
   > It might help to organize a video chat or slide show to go over all the code changes at a high level, so that other Accumulo devs can quickly assess the new design.
   
   I can do a chat on slack.  One goal of that could be to determine how the written documentation could be improved. 
   
   > I kind of liked the idea of setting a single compaction strategy property that encapsulates all the configuration
   
   This is very vague and I don't know what this would concretely look like.  We could discuss this in the chat.  If you have more specific ideas, it would helpful to write them up somewhere.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org