You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2023/08/03 11:29:11 UTC
[kyuubi] branch master updated: [KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 7bc0dbf93 [KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
7bc0dbf93 is described below
commit 7bc0dbf9321a61a646133750b92f22b2bfb34160
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Thu Aug 3 19:29:02 2023 +0800
[KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
### _Why are the changes needed?_
One of the basic ideas of Kerberos authentication is that the Client MUST provide the Server Principal Name (SPN).
[ZOOKEEPER-1467](https://issues.apache.org/jira/browse/ZOOKEEPER-1467) (fixed in 3.6.0, 3.5.7) allows to configure Zookeeper SPN at the client side.
Previously, there is no way but only `-Dzookeeper.server.principal=zookeeper/_HOSTREALM` to set ZK SPN, this PR allows the user to set it via Kyuubi configuration `kyuubi.ha.zookeeper.auth.serverPrincipal`.
In default, if `zookeeper.server.principal` is not configured, it assumes that the Zookeeper server uses `zookeeper/_HOST<SAME_REALM_WITH_CLIENT_PRINCIPAL>`
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes #5130 from pan3793/zk-spn.
Closes #5130
6f0a1bbdc [Cheng Pan] doc
290e172aa [Cheng Pan] docs
a43b0c67a [Cheng Pan] nit
9a33446b7 [Cheng Pan] Allow set Zookeeper server principal
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
---
docs/deployment/settings.md | 53 +++++++++++-----------
.../apache/kyuubi/ha/HighAvailabilityConf.scala | 31 +++++++++----
.../client/zookeeper/ZookeeperClientProvider.scala | 8 +++-
3 files changed, 54 insertions(+), 38 deletions(-)
diff --git a/docs/deployment/settings.md b/docs/deployment/settings.md
index b1b8af143..91c68824c 100644
--- a/docs/deployment/settings.md
+++ b/docs/deployment/settings.md
@@ -262,32 +262,33 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
### Ha
-| Key | Default | Meaning | Type | Since |
-|------------------------------------------------|----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
-| kyuubi.ha.addresses || The connection string for the discovery ensemble | string | 1.6.0 |
-| kyuubi.ha.client.class | org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient | Class name for service discovery client.<ul> <li>Zookeeper: org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient</li> <li>Etcd: org.apache.kyuubi.ha.client.etcd.EtcdDiscoveryClient</li></ul> | string | 1.6.0 |
-| kyuubi.ha.etcd.lease.timeout | PT10S | Timeout for etcd keep alive lease. The kyuubi server will know the unexpected loss of engine after up to this seconds. | duration | 1.6.0 |
-| kyuubi.ha.etcd.ssl.ca.path | <undefined> | Where the etcd CA certificate file is stored. | string | 1.6.0 |
-| kyuubi.ha.etcd.ssl.client.certificate.path | <undefined> | Where the etcd SSL certificate file is stored. | string | 1.6.0 |
-| kyuubi.ha.etcd.ssl.client.key.path | <undefined> | Where the etcd SSL key file is stored. | string | 1.6.0 |
-| kyuubi.ha.etcd.ssl.enabled | false | When set to true, will build an SSL secured etcd client. | boolean | 1.6.0 |
-| kyuubi.ha.namespace | kyuubi | The root directory for the service to deploy its instance uri | string | 1.6.0 |
-| kyuubi.ha.zookeeper.acl.enabled | false | Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
-| kyuubi.ha.zookeeper.auth.digest | <undefined> | The digest auth string is used for ZooKeeper authentication, like: username:password. | string | 1.3.2 |
-| kyuubi.ha.zookeeper.auth.keytab | <undefined> | Location of the Kyuubi server's keytab is used for ZooKeeper authentication. | string | 1.3.2 |
-| kyuubi.ha.zookeeper.auth.principal | <undefined> | Name of the Kerberos principal is used for ZooKeeper authentication. | string | 1.3.2 |
-| kyuubi.ha.zookeeper.auth.type | NONE | The type of ZooKeeper authentication, all candidates are <ul><li>NONE</li><li> KERBEROS</li><li> DIGEST</li></ul> | string | 1.3.2 |
-| kyuubi.ha.zookeeper.connection.base.retry.wait | 1000 | Initial amount of time to wait between retries to the ZooKeeper ensemble | int | 1.0.0 |
-| kyuubi.ha.zookeeper.connection.max.retries | 3 | Max retry times for connecting to the ZooKeeper ensemble | int | 1.0.0 |
-| kyuubi.ha.zookeeper.connection.max.retry.wait | 30000 | Max amount of time to wait between retries for BOUNDED_EXPONENTIAL_BACKOFF policy can reach, or max time until elapsed for UNTIL_ELAPSED policy to connect the zookeeper ensemble | int | 1.0.0 |
-| kyuubi.ha.zookeeper.connection.retry.policy | EXPONENTIAL_BACKOFF | The retry policy for connecting to the ZooKeeper ensemble, all candidates are: <ul><li>ONE_TIME</li><li> N_TIME</li><li> EXPONENTIAL_BACKOFF</li><li> BOUNDED_EXPONENTIAL_BACKOFF</li><li> UNTIL_ELAPSED</li></ul> | string | 1.0.0 |
-| kyuubi.ha.zookeeper.connection.timeout | 15000 | The timeout(ms) of creating the connection to the ZooKeeper ensemble | int | 1.0.0 |
-| kyuubi.ha.zookeeper.engine.auth.type | NONE | The type of ZooKeeper authentication for the engine, all candidates are <ul><li>NONE</li><li> KERBEROS</li><li> DIGEST</li></ul> | string | 1.3.2 |
-| kyuubi.ha.zookeeper.namespace | kyuubi | (deprecated) The root directory for the service to deploy its instance uri | string | 1.0.0 |
-| kyuubi.ha.zookeeper.node.creation.timeout | PT2M | Timeout for creating ZooKeeper node | duration | 1.2.0 |
-| kyuubi.ha.zookeeper.publish.configs | false | When set to true, publish Kerberos configs to Zookeeper. Note that the Hive driver needs to be greater than 1.3 or 2.0 or apply HIVE-11581 patch. | boolean | 1.4.0 |
-| kyuubi.ha.zookeeper.quorum || (deprecated) The connection string for the ZooKeeper ensemble | string | 1.0.0 |
-| kyuubi.ha.zookeeper.session.timeout | 60000 | The timeout(ms) of a connected session to be idled | int | 1.0.0 |
+| Key | Default | Meaning | Type | Since |
+|------------------------------------------------|----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
+| kyuubi.ha.addresses || The connection string for the discovery ensemble | string | 1.6.0 |
+| kyuubi.ha.client.class | org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient | Class name for service discovery client.<ul> <li>Zookeeper: org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient</li> <li>Etcd: org.apache.kyuubi.ha.client.etcd.EtcdDiscoveryClient</li></ul> | string | 1.6.0 |
+| kyuubi.ha.etcd.lease.timeout | PT10S | Timeout for etcd keep alive lease. The kyuubi server will know the unexpected loss of engine after up to this seconds. | duration | 1.6.0 |
+| kyuubi.ha.etcd.ssl.ca.path | <undefined> | Where the etcd CA certificate file is stored. | string | 1.6.0 |
+| kyuubi.ha.etcd.ssl.client.certificate.path | <undefined> | Where the etcd SSL certificate file is stored. | string | 1.6.0 |
+| kyuubi.ha.etcd.ssl.client.key.path | <undefined> | Where the etcd SSL key file is stored. | string | 1.6.0 |
+| kyuubi.ha.etcd.ssl.enabled | false | When set to true, will build an SSL secured etcd client. | boolean | 1.6.0 |
+| kyuubi.ha.namespace | kyuubi | The root directory for the service to deploy its instance uri | string | 1.6.0 |
+| kyuubi.ha.zookeeper.acl.enabled | false | Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
+| kyuubi.ha.zookeeper.auth.digest | <undefined> | The digest auth string is used for ZooKeeper authentication, like: username:password. | string | 1.3.2 |
+| kyuubi.ha.zookeeper.auth.keytab | <undefined> | Location of the Kyuubi server's keytab that is used for ZooKeeper authentication. | string | 1.3.2 |
+| kyuubi.ha.zookeeper.auth.principal | <undefined> | Kerberos principal name that is used for ZooKeeper authentication. | string | 1.3.2 |
+| kyuubi.ha.zookeeper.auth.serverPrincipal | <undefined> | Kerberos principal name of ZooKeeper Server. It only takes effect when Zookeeper client's version at least 3.5.7 or 3.6.0 or applies ZOOKEEPER-1467. To use Zookeeper 3.6 client, compile Kyuubi with `-Pzookeeper-3.6`. | string | 1.8.0 |
+| kyuubi.ha.zookeeper.auth.type | NONE | The type of ZooKeeper authentication, all candidates are <ul><li>NONE</li><li> KERBEROS</li><li> DIGEST</li></ul> | string | 1.3.2 |
+| kyuubi.ha.zookeeper.connection.base.retry.wait | 1000 | Initial amount of time to wait between retries to the ZooKeeper ensemble | int | 1.0.0 |
+| kyuubi.ha.zookeeper.connection.max.retries | 3 | Max retry times for connecting to the ZooKeeper ensemble | int | 1.0.0 |
+| kyuubi.ha.zookeeper.connection.max.retry.wait | 30000 | Max amount of time to wait between retries for BOUNDED_EXPONENTIAL_BACKOFF policy can reach, or max time until elapsed for UNTIL_ELAPSED policy to connect the zookeeper ensemble | int | 1.0.0 |
+| kyuubi.ha.zookeeper.connection.retry.policy | EXPONENTIAL_BACKOFF | The retry policy for connecting to the ZooKeeper ensemble, all candidates are: <ul><li>ONE_TIME</li><li> N_TIME</li><li> EXPONENTIAL_BACKOFF</li><li> BOUNDED_EXPONENTIAL_BACKOFF</li><li> UNTIL_ELAPSED</li></ul> | string | 1.0.0 |
+| kyuubi.ha.zookeeper.connection.timeout | 15000 | The timeout(ms) of creating the connection to the ZooKeeper ensemble | int | 1.0.0 |
+| kyuubi.ha.zookeeper.engine.auth.type | NONE | The type of ZooKeeper authentication for the engine, all candidates are <ul><li>NONE</li><li> KERBEROS</li><li> DIGEST</li></ul> | string | 1.3.2 |
+| kyuubi.ha.zookeeper.namespace | kyuubi | (deprecated) The root directory for the service to deploy its instance uri | string | 1.0.0 |
+| kyuubi.ha.zookeeper.node.creation.timeout | PT2M | Timeout for creating ZooKeeper node | duration | 1.2.0 |
+| kyuubi.ha.zookeeper.publish.configs | false | When set to true, publish Kerberos configs to Zookeeper. Note that the Hive driver needs to be greater than 1.3 or 2.0 or apply HIVE-11581 patch. | boolean | 1.4.0 |
+| kyuubi.ha.zookeeper.quorum || (deprecated) The connection string for the ZooKeeper ensemble | string | 1.0.0 |
+| kyuubi.ha.zookeeper.session.timeout | 60000 | The timeout(ms) of a connected session to be idled | int | 1.0.0 |
### Kinit
diff --git a/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala b/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
index eba069da2..28305ac52 100644
--- a/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
+++ b/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
@@ -91,22 +91,33 @@ object HighAvailabilityConf {
.checkValues(AuthTypes.values.map(_.toString))
.createWithDefault(AuthTypes.NONE.toString)
+ val HA_ZK_AUTH_SERVER_PRINCIPAL: OptionalConfigEntry[String] =
+ buildConf("kyuubi.ha.zookeeper.auth.serverPrincipal")
+ .doc("Kerberos principal name of ZooKeeper Server. It only takes effect when " +
+ "Zookeeper client's version at least 3.5.7 or 3.6.0 or applies ZOOKEEPER-1467. " +
+ "To use Zookeeper 3.6 client, compile Kyuubi with `-Pzookeeper-3.6`.")
+ .version("1.8.0")
+ .stringConf
+ .createOptional
+
val HA_ZK_AUTH_PRINCIPAL: ConfigEntry[Option[String]] =
buildConf("kyuubi.ha.zookeeper.auth.principal")
- .doc("Name of the Kerberos principal is used for ZooKeeper authentication.")
+ .doc("Kerberos principal name that is used for ZooKeeper authentication.")
.version("1.3.2")
.fallbackConf(KyuubiConf.SERVER_PRINCIPAL)
- val HA_ZK_AUTH_KEYTAB: ConfigEntry[Option[String]] = buildConf("kyuubi.ha.zookeeper.auth.keytab")
- .doc("Location of the Kyuubi server's keytab is used for ZooKeeper authentication.")
- .version("1.3.2")
- .fallbackConf(KyuubiConf.SERVER_KEYTAB)
+ val HA_ZK_AUTH_KEYTAB: ConfigEntry[Option[String]] =
+ buildConf("kyuubi.ha.zookeeper.auth.keytab")
+ .doc("Location of the Kyuubi server's keytab that is used for ZooKeeper authentication.")
+ .version("1.3.2")
+ .fallbackConf(KyuubiConf.SERVER_KEYTAB)
- val HA_ZK_AUTH_DIGEST: OptionalConfigEntry[String] = buildConf("kyuubi.ha.zookeeper.auth.digest")
- .doc("The digest auth string is used for ZooKeeper authentication, like: username:password.")
- .version("1.3.2")
- .stringConf
- .createOptional
+ val HA_ZK_AUTH_DIGEST: OptionalConfigEntry[String] =
+ buildConf("kyuubi.ha.zookeeper.auth.digest")
+ .doc("The digest auth string is used for ZooKeeper authentication, like: username:password.")
+ .version("1.3.2")
+ .stringConf
+ .createOptional
val HA_ZK_CONN_MAX_RETRIES: ConfigEntry[Int] =
buildConf("kyuubi.ha.zookeeper.connection.max.retries")
diff --git a/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/zookeeper/ZookeeperClientProvider.scala b/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/zookeeper/ZookeeperClientProvider.scala
index eaffedd12..d0749c8d9 100644
--- a/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/zookeeper/ZookeeperClientProvider.scala
+++ b/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/zookeeper/ZookeeperClientProvider.scala
@@ -108,7 +108,11 @@ object ZookeeperClientProvider extends Logging {
throw new IOException(s"${HA_ZK_AUTH_KEYTAB.key}: $keytab does not exists")
}
System.setProperty("zookeeper.sasl.clientconfig", "KyuubiZooKeeperClient")
- val serverPrincipal = KyuubiHadoopUtils.getServerPrincipal(principal)
+ conf.get(HA_ZK_AUTH_SERVER_PRINCIPAL).foreach { zkServerPrincipal =>
+ // ZOOKEEPER-1467 allows configuring SPN in client
+ System.setProperty("zookeeper.server.principal", zkServerPrincipal)
+ }
+ val zkClientPrincipal = KyuubiHadoopUtils.getServerPrincipal(principal)
// HDFS-16591 makes breaking change on JaasConfiguration
val jaasConf = DynConstructors.builder()
.impl( // Hadoop 3.3.5 and above
@@ -124,7 +128,7 @@ object ZookeeperClientProvider extends Logging {
classOf[String],
classOf[String])
.build[Configuration]()
- .newInstance("KyuubiZooKeeperClient", serverPrincipal, keytab)
+ .newInstance("KyuubiZooKeeperClient", zkClientPrincipal, keytab)
Configuration.setConfiguration(jaasConf)
case _ =>
}