[jira] [Commented] (SOLR-15857) Add Secret Manager support for ZK ACL credentials

["Haythem Khiri (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645735#comment-17645735 ] 

Haythem Khiri commented on SOLR-15857:
--------------------------------------

This seems to be solved 

> Add Secret Manager support for ZK ACL credentials
> -------------------------------------------------
>
>                 Key: SOLR-15857
>                 URL: https://issues.apache.org/jira/browse/SOLR-15857
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Lamine
>            Priority: Minor
>         Attachments: ZooKeeper_Access_Control.pdf
>
>          Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> _{*}Note{*}: Attached a copy of a ref guide describing this feature in more details and with usage examples._    
> *Problem*
> Solr uses a list of credentials to connect to Zookeeper and to handle ACLs.
>  - 1- In the current implementation, the credentials are passed through command line (system props) or read from a clear text file stored in all cluster hosts. Needless to say this is not safe enough.
>  - 2- On the other hand, the same code to load the credentials is called twice, first by _ZkCredentialsProvider_ to connect to Zookeeper and a second time by _ZkACLProvider_ to create ACLs. The code is also duplicated, although it's only reading from system props.
>   Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to load the credentials from another source (ex a Secret Manager) would also require duplicate the code and make repetitive calls to extract the same credentials. 
>  - 3- There is no way to customize how credentials are passed without recompiling. 
>  
> *Proposed solution*
> Let’s start with problem 2).
> *Problem 2*
>  - Refactor the way how the credentials are injected by passing them as a dependency. One code, called once and injected into the client class. Here the client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}.
>  - Favor composition over inheritance to inject custom credentials loaders without changing the composing (container) class. 
>  - Add a third interface _ZkCredentialsInjector_ whose implementations load ZK credentials from a credentials source to be injected into _ZkCredentialsProvider_ and _ZkACLProvider_
>  - The workflow is:  Credentials source —> _ZkCredentialsInjector_ --> {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper
>  - The _ZkCredentialsInjector_ gets the creds from an external source which get injected into zkCredentialsProvider and zkACLProvider. The "{_}external source{_}" here can be system props, a file, a Secret Manager, or any other local or remote source.
>  
> {code:java}
> public interface ZkCredentialsInjector{     
> List<ZkCredential> getZkCredentials();     
> ...
> }
> {code}
>  
>  
>  - Any class implementing _ZkCredentialsInjector_ can be injected via system props in {_}solr.ini.sh/cmd{_}.
> In the below example _VMParamsZkCredentialsInjector_ is injected. 
> Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and _VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated and replaced with a combination of {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and  {_}VMParamsZkCredentialsInjector{_}.
>  
> {code:java}
>   SOLR_ZK_CREDS_AND_ACLS=“
>      -DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
>      -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider \
>      -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector \
>      -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
>      -DzkDigestReadonlyUsername=readonly-user -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
>  SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code}
>  
>  - Add  {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes to support _Digest_ based scheme ZK authentication/authorization
>  
> {code:java}
> Class DigestZkACLProvider implements ZkACLProvider {  CredentialsInjector credentialsInjector;     
>   ... 
> }
> Class DigestZkCredentialsProvider implements ZkCredentialsProvider{     CredentialsInjector credentialsInjector;     
> ... 
> }{code}
>  
> This concept can be generalized to non-digest schemes (a kind of Strategy pattern) but that would require more refactoring, it can be achieved in a future contribution if this one is accepted.
> Now apply this new feature and add a custom injector to solve problem 1).
> *Problem 1*
>  - Store the credentials in a Secret Manager to have Solr pull them out at startup.
>  - Add _SecretCredentialInjector_ class that contains a dependency interface ({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials from a Secret Manager and delegate the _getZkCredentials_ call.
> {code:java}
> public class SecretCredentialInjector implements ZkCredentialsInjector {
>     ...
>     private SecretCredentialsProvider secretCredentialProvider;
>     
>     public List<ZkCredential> getZkCredentials() {         
>           ...         
>           return secretCredentialProvider.getZkCredentials(secretName);     
>    }
>     ...
> }
> {code}
>  
>  - In this contribution the offered implementating class is _AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret Manager. Tu support any other Secret Manager provider all you need to do is add a class implementing _SecretCredentialsProvider_ and pass it through system props (-{_}DzkSecretCredentialsProvider{_})
>  
>  
> {code:java}
> SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \
>   -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider \
>   -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector \
>       -DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider \
>       -DzkSecretCredentialSecretName=zkSecret \
>       -DzkCredentialsAWSSecretRegion=us-west-2"
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"
>  
> {code}
>  
> *Problem 3*
> A new _contrib_ module ({_}secret-provider{_}) is added where _SecretCredentialsProvider_ implementing classes can be added without the need to add a new dependency to Solr core. All one needs to do after adding a new class is to pass it through system props via _solr.ini.sh/cmd_ file. 
> This module can be used in the future for other secrets injections, not specifically related to zk.
>  
> Thank you in advance for your time and your comments.
>  
>     



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org