You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "touchida (via GitHub)" <gi...@apache.org> on 2023/06/30 05:56:40 UTC
[GitHub] [kyuubi] touchida opened a new issue, #5010: [Bug]
touchida opened a new issue, #5010:
URL: https://github.com/apache/kyuubi/issues/5010
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
In our company, we'd like to provide the two types of HA endpoints to users at the same time: Load Balancer using VIP and Zookeeper service discovery mode.
It is because some clients such as PyHive do not support the latter.
However, when we set `kyuubi.frontend.bind.host=0.0.0.0`, Zookeeper service discovery mode will not work, since the `serviceUri` stored in the znode also becomes `0.0.0.0`.
In more details, when we set
- `kyuubi.frontend.bind.host=0.0.0.0`
- `kyuubi.frontend.thrift.binary.bind.port=10009`
The znode becomes as follows:
```
[zk: some_zk_host:some_zk_port(CONNECTED) 0] ls /some_kyuubi_zk_namespace
[serviceUri=0.0.0.0:10009;version=1.8.0-SNAPSHOT;sequence=0000000007]
```
HiveServer2 supports the two types of endpoints at the same time by `hive.server2.thrift.bind.host=""` (default value).
For the details, read the below source code (carefully).
- https://github.com/apache/hive/blob/877a009/service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java#L85
- https://github.com/apache/hive/blob/877a009/service/src/java/org/apache/hive/service/server/HiveServer2.java#L311
### Affects Version(s)
master
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
```yaml
`kyuubi.frontend.bind.host=0.0.0.0`
```
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1614180378
Emm... so you expect that Kyuubi binds `0.0.0.0`(all address), but use another address for registering? Sound reasonable, but unfortunately, Kyuubi does not support it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] touchida commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "touchida (via GitHub)" <gi...@apache.org>.
touchida commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1614176527
@pan3793
Thanks for your comment!
> Kyuubi will try to bind the first non-loopback address if kyuubi.frontend.bind.host is absent
You're right.
In that case, however, the problem is that we cannot access Kyuubi via Load Balancer VIP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] touchida commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "touchida (via GitHub)" <gi...@apache.org>.
touchida commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1614194995
@pan3793
> Emm... so you expect that Kyuubi binds 0.0.0.0(all address), but use another address for registering?
Yup!
I'm now considering something like the following fix.
I think this will be the same behavior as HiveServer2 with `hive.server2.thrift.bind.host=""`.
```
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
index 7532beccb..e497f2fa5 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
@@ -113,6 +113,10 @@ abstract class TFrontendService(name: String)
override def connectionUrl: String = {
checkInitialized()
val host = serverHost match {
+ case Some("0.0.0.0") if conf.get(FRONTEND_CONNECTION_URL_USE_HOSTNAME) =>
+ Utils.findLocalInetAddress.getCanonicalHostName
+ case Some("0.0.0.0") =>
+ Utils.findLocalInetAddress
case Some(h) => h // respect user's setting ahead
case None if conf.get(FRONTEND_CONNECTION_URL_USE_HOSTNAME) =>
serverAddr.getCanonicalHostName
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 closed issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
URL: https://github.com/apache/kyuubi/issues/5010
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1614171563
Kyuubi will try to bind the first non-loopback address if `kyuubi.frontend.bind.host` is absent, is this behavior you expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] touchida commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "touchida (via GitHub)" <gi...@apache.org>.
touchida commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1617131061
@pan
Your idea looks better than mine above, since it will be helpful when we want to use a wildcard certificate to enable SSL but the domains of server hostnames and the certificate are different.
If there is no objection, I'll submit a PR based on yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1617266683
@touchida please go ahead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #5010: [Bug] ZK service discovery mode does not work when front end bind host is set to 0.0.0.0
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5010:
URL: https://github.com/apache/kyuubi/issues/5010#issuecomment-1614203801
sounds reasonable, also cc @yaooqinn
another approach I thought of is to introduce a new configuration: `kyuubi.frontend.advisor.host` which is dedicated to connecting, which is more flexible but also more complex
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org