You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "lsm1 (via GitHub)" <gi...@apache.org> on 2024/08/05 12:01:19 UTC
[PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
lsm1 opened a new pull request, #6585:
URL: https://github.com/apache/kyuubi/pull/6585
# :mag: Description
## Issue References ๐
<!-- Append the issue number after #. If there is no issue for you to link create one or -->
<!-- If there are no issues to link, please provide details here. -->
This pull request fixes #
## Describe Your Solution ๐ง
The current FileSessionConfAdvisor only supports configuring a single file through kyuubi.session.conf.profiler. However, users may need to use multiple configuration files, and the existing setup does not support this. To address this need, we have updated the FileSessionConfAdvisor to support multiple configuration files.
## Types of changes :bookmark:
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan ๐งช
#### Behavior Without This Pull Request :coffin:
#### Behavior With This Pull Request :tada:
#### Related Unit Tests
---
# Checklist ๐
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6585:
URL: https://github.com/apache/kyuubi/pull/6585#issuecomment-2269034442
## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/6585?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `11 lines` in your changes missing coverage. Please review.
> Project coverage is 0.00%. Comparing base [(`063a192`)](https://app.codecov.io/gh/apache/kyuubi/commit/063a192c7a67d5588de7f2a1ce5c5e6d8d5869e0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`fcedc4f`)](https://app.codecov.io/gh/apache/kyuubi/commit/fcedc4fe3cf78d871e3d0548e32603abc37d7c2a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 13 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/kyuubi/pull/6585?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...apache/kyuubi/session/FileSessionConfAdvisor.scala](https://app.codecov.io/gh/apache/kyuubi/pull/6585?src=pr&el=tree&filepath=kyuubi-server%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fsession%2FFileSessionConfAdvisor.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0ZpbGVTZXNzaW9uQ29uZkFkdmlzb3Iuc2NhbGE=) | 0.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/6585?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://app.codecov.io/gh/apache/kyuubi/pull/6585?src=pr&el=tree&filepath=kyuubi-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fconfig%2FKyuubiConf.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/6585?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #6585 +/- ##
=======================================
Coverage 0.00% 0.00%
=======================================
Files 677 677
Lines 41799 42186 +387
Branches 5709 5803 +94
=======================================
- Misses 41799 42186 +387
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/6585?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
Posted by "lsm1 (via GitHub)" <gi...@apache.org>.
lsm1 commented on code in PR #6585:
URL: https://github.com/apache/kyuubi/pull/6585#discussion_r1711295766
##########
kyuubi-server/src/test/scala/org/apache/kyuubi/plugin/PluginLoaderSuite.scala:
##########
@@ -67,6 +67,19 @@ class PluginLoaderSuite extends KyuubiFunSuite {
assert(clusterAConfFromCache.get("kyuubi.ha.namespace") == "kyuubi-ns-a")
assert(clusterAConfFromCache.get("kyuubi.zk.ha.namespace") == null)
assert(clusterAConfFromCache.size() == 5)
+
+ conf.set(KyuubiConf.SESSION_CONF_PROFILE, Seq("cluster-a", "cluster-b"))
+ val clusterABConf =
Review Comment:
In this case, since the value of `kyuubi.ha.namespace` is different, the value from `cluster-b` will replace the value from `cluster-a`. I will add more key-value pairs to `cluster-b` to refine this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #6585:
URL: https://github.com/apache/kyuubi/pull/6585#discussion_r1708654096
##########
kyuubi-server/src/test/scala/org/apache/kyuubi/plugin/PluginLoaderSuite.scala:
##########
@@ -67,6 +67,19 @@ class PluginLoaderSuite extends KyuubiFunSuite {
assert(clusterAConfFromCache.get("kyuubi.ha.namespace") == "kyuubi-ns-a")
assert(clusterAConfFromCache.get("kyuubi.zk.ha.namespace") == null)
assert(clusterAConfFromCache.size() == 5)
+
+ conf.set(KyuubiConf.SESSION_CONF_PROFILE, Seq("cluster-a", "cluster-b"))
+ val clusterABConf =
Review Comment:
This case does not seem to reflect the scenario of multiple profiles.
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/session/FileSessionConfAdvisor.scala:
##########
@@ -51,15 +51,18 @@ object FileSessionConfAdvisor extends Logging {
reloadInterval,
TimeUnit.MILLISECONDS)
.build(new CacheLoader[String, JMap[String, String]] {
- override def load(profile: String): JMap[String, String] = {
- val propsFile = Utils.getPropertiesFile(s"kyuubi-session-$profile.conf")
- propsFile match {
- case None =>
- error("File not found: $KYUUBI_CONF_DIR/" + s"kyuubi-session-$profile.conf")
- Collections.emptyMap()
- case Some(_) =>
- Utils.getPropertiesFromFile(propsFile).asJava
+ override def load(profiles: String): JMap[String, String] = {
Review Comment:
Should the key of sessionConfCache be one profile?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
Posted by "lsm1 (via GitHub)" <gi...@apache.org>.
lsm1 closed pull request #6585: Support Multiple Configuration Files in FileSessionConfAdvisor
URL: https://github.com/apache/kyuubi/pull/6585
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Support Multiple Configuration Files in FileSessionConfAdvisor [kyuubi]
Posted by "lsm1 (via GitHub)" <gi...@apache.org>.
lsm1 commented on PR #6585:
URL: https://github.com/apache/kyuubi/pull/6585#issuecomment-2283068316
Merge to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org