[GitHub] [druid] kroeders opened a new pull request #10484: improve retention rule management with import rules

[GitBox <gi...@apache.org>]

kroeders opened a new pull request #10484:
URL: https://github.com/apache/druid/pull/10484


   Discussion of the general problem of Increase reusability of retention rule configurations is here [Issue #10237 ](https://github.com/apache/druid/issues/10237)
   This pull request implements a new rule type that imports rules from any other rule set, including rules from other data sources. The back end code is the same as the previous PR #10129 and the UI changes have been ported to incorporate the recent UI changes. 
   
   
   ### Description
   
   This branch adds a new ImportRule type that specifies an imported rule set. Whenever rules are used, they are expanded to include the default rules for the cluster. This changes the getRulesWithDefault to also expand any import rules found at the time they are retrieved. This is functionally equivalent to the original list as the rules are replaced before evaluation. Rules are also used in CoordinatorRuleManager in the router, where getRulesWithDefault was implemented again. Here, that implementation is left at one place, in SQLRuleManager and a new method is added to RulesResource to allow CoordinatorRuleManager to retrieve the expanded list of rules without knowing how it was generated. 
   
   In the implementation, the function of getRulesWithDefault has diverged a little bit from the name, maybe a name like getExpandedRules would be better. Also, the implementation for ImportRule is specifically in SQLMetadataRuleManager maybe it should be interface methods or in a utility class for use by other implementations in the future. 
   
   <img width="743" alt="Screen Shot 2020-10-07 at 2 46 29 PM" src="https://user-images.githubusercontent.com/8482587/95377733-f99cc800-08b0-11eb-9a1d-c8132a9de54d.png">
   
   <hr>
   
   This PR has:
   - [X] been self-reviewed.
   - [X] added documentation for new or modified features or behaviors.
   - [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
   - [X] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
   - [X] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
   - [ ] added integration tests.
   - [X] been tested in a test Druid cluster.
   
   <hr>
   
   ##### Key changed/added classes in this PR
    * `SQLMetadataRuleManager`
    * `ImportRule`
    * `CoordinatorRuleManager`
    * `RulesResource`
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org