You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/07/13 07:26:10 UTC
[GitHub] [incubator-kyuubi] wForget opened a new issue #786: Engine load balancing strategy
wForget opened a new issue #786:
URL: https://github.com/apache/incubator-kyuubi/issues/786
### 1. Describe the feature
The last engine is acquired by default. Is there any plan to support other engine load balancing strategies?
Such as load balancing according to the current number of sessions of the engine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on issue #786: Engine load balancing strategy
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on issue #786:
URL: https://github.com/apache/incubator-kyuubi/issues/786#issuecomment-878858349
Good idea, actually we have some discussion about this internal.
Maybe we can have something like engine pool to provide the ability of load balancing. And about the strategies, we can support `round robin`, `session/statment-number-weight-based` or other user defined strategy which is used the interface provided by this framework.
As you konw, Kyuubi is stateless in a cluster so it should be considered first how to share the engine pool status between several Kyuubi services. So we need more details and some design of this.
It's also very helpful if you can share some use case of this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] wForget commented on issue #786: Engine load balancing strategy
Posted by GitBox <gi...@apache.org>.
wForget commented on issue #786:
URL: https://github.com/apache/incubator-kyuubi/issues/786#issuecomment-879907673
Hi @ulysses-you @yaooqinn
A simple implementation of this feature in #802 , can you help me review it and give some suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #786: Engine load balancing strategy
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #786:
URL: https://github.com/apache/incubator-kyuubi/issues/786#issuecomment-878858464
For engine, we'd call it service discovery, not load balancing. The engine discovery logic is better to bind with client requests, a.k.a, JDBC connections, as an engine is stateful with cache tables, udf, etc. We already have implemented 3 share levels, CONNECTION/USER/SERVER for different kinds of usage scenarios.
- For CONNECTION mode, the engine and all states are within a JDBC connection.
- For USER mode, the engine and all states might/could share across all JDBC connections of a user with a `subDomain` if any.
For `The last engine is acquired by default`, it's a potential bug since `1.2.0-rc`s as a distributed lock for engine creation. When an engine is created but killed forcefully and externally, the service uri of this engine will not get cleared. And another thread might get a malformed engine after it gains the lock. For this part, I think we need to introduce a retry and recreation of engine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org