You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2023/08/15 18:35:58 UTC
[kyuubi] branch branch-1.7 updated: [KYUUBI #5158] Allow embedded Zookeeper binding IP address
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.7 by this push:
new def4f4ae9 [KYUUBI #5158] Allow embedded Zookeeper binding IP address
def4f4ae9 is described below
commit def4f4ae9f0066f87879559eb806362e42f80ea3
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Wed Aug 16 01:53:51 2023 +0800
[KYUUBI #5158] Allow embedded Zookeeper binding IP address
### _Why are the changes needed?_
This PR provides a new configuration `kyuubi.zookeeper.embedded.client.use.hostname` to control whether use hostname or IP address on binding, to improve the out-of-box experience of K8s case, similar to `kyuubi.frontend.connection.url.use.hostname`.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
Closes #5158 from pan3793/zk-ip.
Closes #5158
260dae987 [Cheng Pan] nit
3ac0c430e [Cheng Pan] doc
de50f65e8 [Cheng Pan] false
890b13bd9 [Cheng Pan] Allow embedded Zookeeper binding IP address
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
(cherry picked from commit 4e4a4e11538dfe6a490368a214bf019af6b4dd2c)
Signed-off-by: Cheng Pan <ch...@apache.org>
---
docs/deployment/settings.md | 25 +++++++++++-----------
.../kyuubi/zookeeper/EmbeddedZookeeper.scala | 8 ++++++-
.../apache/kyuubi/zookeeper/ZookeeperConf.scala | 7 ++++++
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/docs/deployment/settings.md b/docs/deployment/settings.md
index f079b5b44..a8ad219fa 100644
--- a/docs/deployment/settings.md
+++ b/docs/deployment/settings.md
@@ -421,18 +421,19 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
### Zookeeper
-| Key | Default | Meaning | Type | Since |
-|--------------------------------------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|-------|
-| kyuubi.zookeeper.embedded.client.port | 2181 | clientPort for the embedded ZooKeeper server to listen for client connections, a client here could be Kyuubi server, engine, and JDBC client | int | 1.2.0 |
-| kyuubi.zookeeper.embedded.client.port.address | <undefined> | clientPortAddress for the embedded ZooKeeper server to | string | 1.2.0 |
-| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. | string | 1.2.0 |
-| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log . | string | 1.2.0 |
-| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server | string | 1.0.0 |
-| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
-| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
-| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
-| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
-| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |
+| Key | Default | Meaning | Type | Since |
+|--------------------------------------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-------|
+| kyuubi.zookeeper.embedded.client.port | 2181 | clientPort for the embedded ZooKeeper server to listen for client connections, a client here could be Kyuubi server, engine, and JDBC client | int | 1.2.0 |
+| kyuubi.zookeeper.embedded.client.port.address | <undefined> | clientPortAddress for the embedded ZooKeeper server to | string | 1.2.0 |
+| kyuubi.zookeeper.embedded.client.use.hostname | false | When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address. | boolean | 1.7.2 |
+| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. | string | 1.2.0 |
+| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log . | string | 1.2.0 |
+| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server | string | 1.0.0 |
+| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
+| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
+| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
+| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
+| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |
## Spark Configurations
diff --git a/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala b/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala
index 0e980132a..1ce3f49e3 100644
--- a/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala
+++ b/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/EmbeddedZookeeper.scala
@@ -43,7 +43,13 @@ class EmbeddedZookeeper extends AbstractService("EmbeddedZookeeper") {
val maxClientCnxns = conf.get(ZK_MAX_CLIENT_CONNECTIONS)
val minSessionTimeout = conf.get(ZK_MIN_SESSION_TIMEOUT)
val maxSessionTimeout = conf.get(ZK_MAX_SESSION_TIMEOUT)
- host = conf.get(ZK_CLIENT_PORT_ADDRESS).getOrElse(findLocalInetAddress.getCanonicalHostName)
+ host = conf.get(ZK_CLIENT_PORT_ADDRESS).getOrElse {
+ if (conf.get(ZK_CLIENT_USE_HOSTNAME)) {
+ findLocalInetAddress.getCanonicalHostName
+ } else {
+ findLocalInetAddress.getHostAddress
+ }
+ }
zks = new ZooKeeperServer(dataDirectory, dataDirectory, tickTime)
zks.setMinSessionTimeout(minSessionTimeout)
diff --git a/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala b/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
index c6256a5e3..6ef494896 100644
--- a/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
+++ b/kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
@@ -49,6 +49,13 @@ object ZookeeperConf {
.stringConf
.createOptional
+ val ZK_CLIENT_USE_HOSTNAME: ConfigEntry[Boolean] =
+ buildConf("kyuubi.zookeeper.embedded.client.use.hostname")
+ .doc("When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address.")
+ .version("1.7.2")
+ .booleanConf
+ .createWithDefault(false)
+
val ZK_DATA_DIR: ConfigEntry[String] = buildConf("kyuubi.zookeeper.embedded.data.dir")
.doc("dataDir for the embedded zookeeper server where stores the in-memory database" +
" snapshots and, unless specified otherwise, the transaction log of updates to the database.")