[GitHub] [accumulo] ctubbsii commented on pull request #2096: External Compactions

[GitBox <gi...@apache.org>]

ctubbsii commented on pull request #2096:
URL: https://github.com/apache/accumulo/pull/2096#issuecomment-842357738


   Based on the documentation in apache/accumulo-website#282 , it seems that the way the external compactions are exposed through to the user in the compaction configuration is through the specially named `externalQueue`, which seems to be a special feature of the default compaction service. Is that correct?
   
   From the perspective of the user configuring things, it seems difficult to reason about how this configuration maps to pluggable services that they are deploying and administering, because the external compaction services aren't configured explicitly, but are an internal feature of a monolithic compaction service implementation. It seems like it will try to "automagically" use external services because of a specially named queue, and not because some plugin like `my.external.compaction.CompactionExecutorService` is configured explicitly as an executor (unless I'm mistaken here; I'm still going through the code).
   
   I'm wondering if there are ways this could be made a bit more modular and the configuration a bit more explicit, so that the default compaction service isn't so monolithic, and users can very explicitly swap out the external executor implementation for another one. This specific implementation could be maintained in a side repository, deployed as a separate optional jar, and configured by users by some class name at an appropriate place in the configuration that specifies a plugin implementation. However, it should also be capable of being easily swapped out for a different external compaction implementation, and I'm not sure that's the case right now.
   
   My main concern is that if the implementation is too monolithic, it creates a maintenance burden to continue to add marginal improvements to the monolithic implementation to satisfy every divergent use case, rather than encourage users to take advantage of the modular nature of our plugins to develop implementations to satisfy specific use cases. Ideally, we'd really want to make things pluggable/modular and the plugin interfaces relatively lightweight.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org