You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/11/16 01:23:20 UTC
[GitHub] [incubator-kyuubi] hddong opened a new issue #1392: [Umbrella] Support service discovery through etcd
hddong opened a new issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues) and found no similar issues.
### Describe the proposal
On k8s, if we not had a zk yeet, we can use the existing etcd for service discovery.
### Task list
-[]
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-973661795
look good to me, can these be put in the same module？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
hddong commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-973644924
cc @pan3793 @yaooqinn @ulysses-you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] timothy65535 commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
timothy65535 commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-975033252
hi, it seems we still need zookeeper cluster, we expose kyuubi server to zookeeper, hive client can connect to server by zookeeper. how the hive client use etcd?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
hddong commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-975459970
> hi, it seems we still need zookeeper cluster, we expose ...how the hive client use etcd?
Yes, it does not support server discovery by default. Maybe we have two solutions: [zetcd](https://github.com/etcd-io/zetcd)(It hasn't been updated for a while), or we can provide a client or jar support etcd. And, on kubernetes, may be one server is acceptable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-971501315
Would you please describe your plan in more detail?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
hddong commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-973644283
Sure.
## The modules:
```
kyuubi-ha
- kyuubi-ha-common
- kyuubi-ha-zookeeper
- kyuubi-ha-etcd
```
## Implementation
There are mainly two classes, `ServiceDiscovery` and `ZooKeeperClientProvider`, for other modules to use.
1. `ServiceDiscovery`
- use a new class `AbstractInstanceClient` instead of `zkClient`
- Move static functions in `ServiceDiscovery` to `DiscoveryClient`
`DiscoveryClient` contains discovery api( E.g `createServiceNodeAndGetPath`, `getServiceNodesInfo`) and client api (E.g `getChildren`, `delete`).
2. `ZooKeeperClientProvider`
Abstract as `DiscoveryClientProvider`
## Usages
In other modules, original:
```scala
withZkClient(conf) { framework =>
createServiceNodeAndGetPath(conf, framework, uniqueNamespace, "localhost:10000")
framework.delete().forPath(nodePath)
}
```
we can change to:
```scala
withDiscoveryClient(conf) { framework =>
framework.createServiceNodeAndGetPath(conf, uniqueNamespace, "localhost:10000")
framework.delete(nodePath)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] hddong commented on issue #1392: [Umbrella] Support service discovery through etcd
Posted by GitBox <gi...@apache.org>.
hddong commented on issue #1392:
URL: https://github.com/apache/incubator-kyuubi/issues/1392#issuecomment-973668629
> look good to me, can these be put in the same module？
Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org