You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "dnskr (via GitHub)" <gi...@apache.org> on 2024/07/10 16:15:46 UTC

[PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

dnskr opened a new pull request, #6530:
URL: https://github.com/apache/kyuubi/pull/6530

   # :mag: Description
   ## Issue References ๐Ÿ”—
   
   This pull request fixes https://github.com/apache/kyuubi/issues/6523
   
   ## Describe Your Solution ๐Ÿ”ง
   
   `KyuubiServer` respects configuration properties provided as `--conf` cli options with highest priority.
   
   
   ## Types of changes :bookmark:
   <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [x] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing functionality to change)
   
   ## Test Plan ๐Ÿงช
   
   #### Behavior Without This Pull Request :coffin:
   REST endpoint starts at `10099` port:
   ```shell
   bin/kyuubi run --conf kyuubi.frontend.rest.bind.port=9999
   ...
   2024-07-10 17:57:24.998 INFO main org.apache.kyuubi.server.KyuubiRestFrontendService: Service[KyuubiRestFrontendService] is started.
   2024-07-10 17:57:24.999 INFO main org.apache.kyuubi.server.KyuubiRestFrontendService: Exposing REST endpoint at: http://0.0.0.0:10099
   2024-07-10 17:57:24.999 INFO main org.apache.kyuubi.server.KyuubiServer: Service[KyuubiServer] is started.
   ```
   
   #### Behavior With This Pull Request :tada:
   REST endpoint starts at `9999` port:
   ```shell
   bin/kyuubi run --conf kyuubi.frontend.rest.bind.port=9999
   ...
   2024-07-10 17:49:41.659 INFO main org.apache.kyuubi.server.KyuubiRestFrontendService: Service[KyuubiRestFrontendService] is started.
   2024-07-10 17:49:41.660 INFO main org.apache.kyuubi.server.KyuubiRestFrontendService: Exposing REST endpoint at: http://0.0.0.0:9999
   2024-07-10 17:49:41.660 INFO main org.apache.kyuubi.server.KyuubiServer: Service[KyuubiServer] is started.
   ```
   
   #### Related Unit Tests
   `kyuubi-common/src/test/scala/org/apache/kyuubi/config/KyuubiConfSuite.scala`
   
   ---
   
   # Checklist ๐Ÿ“
   <!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
   <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
   
   - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
   
   **Be nice. Be informative.**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#discussion_r1675628506


##########
bin/kyuubi:
##########
@@ -102,7 +102,7 @@ if [[ -n ${YARN_CONF_DIR} ]]; then
   KYUUBI_CLASSPATH="${KYUUBI_CLASSPATH}:${YARN_CONF_DIR}"
 fi
 
-cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS"
+cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS ${@:2}"

Review Comment:
   it also allows `bin/kyuubi` without command. I think we can shift the consumed args and pass `$@` here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "dnskr (via GitHub)" <gi...@apache.org>.
dnskr commented on PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#issuecomment-2220970441

   I'm not familiar with the source code well enough yet, so any feedback, suggestions or even an alternative PR are highly appreciated. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#issuecomment-2225259251

   the rest change lgtm, thanks for working on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#issuecomment-2224485018

   ## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/6530?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   Attention: Patch coverage is `0%` with `12 lines` in your changes missing coverage. Please review.
   > Project coverage is 0.00%. Comparing base [(`53b3911`)](https://app.codecov.io/gh/apache/kyuubi/commit/53b3911935a3b0d39cec495edf8750df3c9e0c61?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`4977af1`)](https://app.codecov.io/gh/apache/kyuubi/commit/4977af1683f4a69d08add613da0eac47a89e4622?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   | [Files](https://app.codecov.io/gh/apache/kyuubi/pull/6530?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../scala/org/apache/kyuubi/server/KyuubiServer.scala](https://app.codecov.io/gh/apache/kyuubi/pull/6530?src=pr&el=tree&filepath=kyuubi-server%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fserver%2FKyuubiServer.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXJ2ZXIvS3l1dWJpU2VydmVyLnNjYWxh) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/6530?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://app.codecov.io/gh/apache/kyuubi/pull/6530?src=pr&el=tree&filepath=kyuubi-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fconfig%2FKyuubiConf.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/6530?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@          Coverage Diff           @@
   ##           master   #6530   +/-   ##
   ======================================
     Coverage    0.00%   0.00%           
   ======================================
     Files         677     677           
     Lines       41787   41792    +5     
     Branches     5707    5707           
   ======================================
   - Misses      41787   41792    +5     
   ```
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/6530?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#issuecomment-2233444552

   also cc @cxzl25 @cfmcgrady @hddong 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#issuecomment-2235247819

   Thanks, merged to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#discussion_r1677266627


##########
bin/kyuubi:
##########
@@ -102,7 +102,7 @@ if [[ -n ${YARN_CONF_DIR} ]]; then
   KYUUBI_CLASSPATH="${KYUUBI_CLASSPATH}:${YARN_CONF_DIR}"
 fi
 
-cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS"
+cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS ${@:2}"

Review Comment:
   > Why do we need to support running Kyuubi as a daemon with `bin/kyuubi` without `start` command?
   
   this is a behavior from the very beginning, let's keep it to avoid breaking the user experience
   
   > ... so in current implementation `restart` command requires users to provide the same `--conf` options was used with `start` command.
   
   I see, and it is reasonable to me.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "dnskr (via GitHub)" <gi...@apache.org>.
dnskr commented on code in PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#discussion_r1676027345


##########
bin/kyuubi:
##########
@@ -102,7 +102,7 @@ if [[ -n ${YARN_CONF_DIR} ]]; then
   KYUUBI_CLASSPATH="${KYUUBI_CLASSPATH}:${YARN_CONF_DIR}"
 fi
 
-cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS"
+cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS ${@:2}"

Review Comment:
   @pan3793 Thanks, I missed that case.
   
   Why do we need to support running Kyuubi as a daemon with `bin/kyuubi` without `start` command? At first look, it would be more expected to me to print help message if no command provided by user. I can create a PR if you think it is a good change.
   
   Also I found another interesting case. As a user, I would expect `restart` to launch Kyuubi with the same configs was used to launch Kyuubi with `start` command initially. Not sure how to manage it automatically, so in current implementation `restart` command requires users to provide the same `--conf` options was used with `start` command.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "dnskr (via GitHub)" <gi...@apache.org>.
dnskr commented on code in PR #6530:
URL: https://github.com/apache/kyuubi/pull/6530#discussion_r1680616450


##########
bin/kyuubi:
##########
@@ -102,7 +102,7 @@ if [[ -n ${YARN_CONF_DIR} ]]; then
   KYUUBI_CLASSPATH="${KYUUBI_CLASSPATH}:${YARN_CONF_DIR}"
 fi
 
-cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS"
+cmd="${RUNNER} ${KYUUBI_JAVA_OPTS} -cp ${KYUUBI_CLASSPATH} $CLASS ${@:2}"

Review Comment:
   @pan3793 I added commit to address your initial suggestion. Please, have a look if it looks good.
   
   ---
   > > Why do we need to support running Kyuubi as a daemon with `bin/kyuubi` without `start` command?
   > 
   > this is a behavior from the very beginning, let's keep it to avoid breaking the user experience
   > 
   
   Right. It is make sense to leave it as is, at least before Kyuubi 2.0
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] [KYUUBI #6523] Support configuration with `--conf` options [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #6530: [KYUUBI #6523] Support configuration with `--conf` options
URL: https://github.com/apache/kyuubi/pull/6530


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org