[jira] [Commented] (NIFI-8000) Determine how to set ACLs in ZooKeeper when using TLS

["Moncef ABBOUD (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257166#comment-17257166 ] 

Moncef ABBOUD commented on NIFI-8000:
-------------------------------------

Hello [~bbende], 

Thank you for the detailed description and happy new year. 

Regarding the Curator client, I noticed that only Kerberos is supported as a SASL mechanism and it is not possible to set ZK ACLs when using another SASL mechanisms such as DIGEST for instance. How about using ZooDefs.Ids.CREATOR_ALL_ACL to set ACLs in the case of SASL in general and allowing special treatment in the case of Kerberos to leverage removeHostFromPrincipal/removeRealmFromPrincipal ?

In the case of TLS, how about adding the constraint of having the same CN across the different nodes so the ZK ACL would have just the one identity and be accessible to all the nodes having this identity keeping in mind that Subject Alternative Name (SAN) can be used to ensure hostname validation.

Finally, with respect to the differences between the State ZK client and Curator client, how about pushing the ACLs handling logic to a Utils class that would be used by both clients to allow better extensibility and consistency. This would address your second point and would allow leveraging of the removeHostFromPrincipal/removeRealmFromPrincipal in the ZK State manager.

Best regards.

> Determine how to set ACLs in ZooKeeper when using TLS
> -----------------------------------------------------
>
>                 Key: NIFI-8000
>                 URL: https://issues.apache.org/jira/browse/NIFI-8000
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Bryan Bende
>            Priority: Major
>
> We currently have two code paths in NiFi that use ZooKeeper:
> 1) Clustering (uses Curator)
> 2) State Management (uses regular ZK client)
> Recently both of these were updated to support TLS.
> In the past we supported setting ACLs in ZooKeeper when NiFi and ZK were kerberized. In a cluster, each NiFi node would typically have a principal like nifi/HOST@DOMAIN, and we don't wan't the ACL to be set to one specific identity, since then the other nodes wouldn't be able to modify the z-node. 
> ZooKeeper offers properties to control the kerberos identity:
> {code:java}
> kerberos.removeHostFromPrincipal (Java system property: zookeeper.kerberos.removeHostFromPrincipal)
>  You can instruct ZooKeeper to remove the host from the client principal
>  name during authentication. (e.g. the zk/myhost@EXAMPLE.COM client 
> principal will be authenticated in ZooKeeper as zk@EXAMPLE.COM) Default:
>  false
> kerberos.removeRealmFromPrincipal (Java system property: zookeeper.kerberos.removeRealmFromPrincipal)
>  You can instruct ZooKeeper to remove the realm from the client 
> principal name during authentication. (e.g. the zk/myhost@EXAMPLE.COM 
> client principal will be authenticated in ZooKeeper as zk/myhost) 
> Default: false {code}
> So we provided a similar mechanism in nifi.properties:
> {code:java}
> # Zookeeper properties for the authentication scheme used when creating acls on znodes used for cluster management
> # Values supported for nifi.zookeeper.auth.type are "default", which will apply world/anyone rights on znodes
> # and "sasl" which will give rights to the sasl/kerberos identity used to authenticate the nifi node
> # The identity is determined using the value in nifi.kerberos.service.principal and the removeHostFromPrincipal
> # and removeRealmFromPrincipal values (which should align with the kerberos.removeHostFromPrincipal and kerberos.removeRealmFromPrincipal
> # values configured on the zookeeper server).
> nifi.zookeeper.auth.type=x509
> nifi.zookeeper.kerberos.removeHostFromPrincipal=
> nifi.zookeeper.kerberos.removeRealmFromPrincipal= {code}
> So this way all the kerberos principals could be mapped to just "nifi".
> ZooKeeper now supports setting ACLs based on the X509 identity, but it brings us back to the same problem. Given a three node nifi cluster with certificates like:
> {code:java}
> CN=nifi-1, OU=xyz
> CN=nifi-2, OU=xyz
> CN=nifi-3, OU=xyz {code}
> We don't want the ACL to be set such that only one of these specific identities is the owner, we want it to be something like "nifi", but there is no standard way to perform an identity mapping here.
> Current problems:
> 1) ZK State Manager currently always uses OPEN for creating the root node
> [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/state/providers/zookeeper/ZooKeeperStateProvider.java#L431]
> 2) ZK State Manager does not leverage the removeHostFromPrincipal/removeRealmFromPrincipal, so even when doing kerberos it woudln't work correctly.
> 3) Clustering uses a Curator ACL factory that would need to be updated to handle the x509 case
> [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/leader/election/CuratorACLProviderFactory.java]
> I tested the changes for 1 and 3 above and have put them on this branch:
> [https://github.com/apache/nifi/compare/main...bbende:zk-acls?expand=1]
> With these changes, if you set the following:
> nifi.properties
> - nifi.zookeeper.auth.type=x509
> state-management.xml
> - <property name="Root Node">/nifi-state</property>
> - <property name="Access Control">CreatorOnly</property>
> Then check ZK and the ACLs will be correctly set:
> {code:java}
> [zk: 127.0.0.1:2281(CONNECTED) 8] getAcl /nifi
> 'x509,'CN=localhost%2COU=NIFI: cdrwa 
> [zk: 127.0.0.1:2281(CONNECTED) 8] getAcl /nifi-state
> 'x509,'CN=localhost%2COU=NIFI: cdrwa{code}
> Now we are left with the problem of how to make the identity not be specific to only one node.
> Currently we use the "auth" scheme for ZK ACLs, which tells ZK to use the identity that authenticated to ZK. One option might be to provide a way to use the "digest" scheme and have properties that let each node specify a username/password for the digest identity, this way all the nifi nodes would use the same username/password.
> https://zookeeper.apache.org/doc/r3.5.7/zookeeperProgrammers.html#sc_ZooKeeperAccessControl



--
This message was sent by Atlassian Jira
(v8.3.4#803005)