You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "bowenliang123 (via GitHub)" <gi...@apache.org> on 2023/07/04 08:40:14 UTC
[GitHub] [kyuubi] bowenliang123 opened a new pull request, #5016: Support to disable custom app name for spark engine
bowenliang123 opened a new pull request, #5016:
URL: https://github.com/apache/kyuubi/pull/5016
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/CONTRIBUTING.html
2. If the PR is related to an issue in https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
- support to disable custom app name for spark engine, for either share level of USER/GROUP/SERVER.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 closed pull request #5016: Support ignoring custom app name for Spark engine
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 closed pull request #5016: Support ignoring custom app name for Spark engine
URL: https://github.com/apache/kyuubi/pull/5016
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620930907
It happened to set a custom spark app name for one time task on the user/client side without notifying the Kyuubi's administrator. Plus, he's using USER / SERVER share level and the app name was contaminated as the one time data task name. From the perspective of Kyuubi's administrator, it's hard to tell it's a shared spark SQL engine on YARN with such custom app name.
Therefore, this feature mainly enables the administrator of Kyuubi and the resource provider, to reject or ignore custom app name with the share level of USER / SERVER / GROUP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1630213547
> How about introducing a `kyuubi.session.conf.sharable.engine.ignore.list`?
It's a considerable approach for this.
It's introducing a new concept `sharable engine` or `sharable share level`. And this config may mix configs for different engine types as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620951844
Seems we have `kyuubi.session.conf.ignore.list`, the only difference is it applies to all share levels?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] link3280 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "link3280 (via GitHub)" <gi...@apache.org>.
link3280 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620925358
I'm curious to know why users set the custom app name in the first place if they want the default one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 closed pull request #5016: Support ignoring custom app name for Spark engine
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #5016: Support ignoring custom app name for Spark engine
URL: https://github.com/apache/kyuubi/pull/5016
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620968687
Yes. The CONNECTION level should not be limited to custom app name.
And this feature may be extended to combine share level and the customer app name as the final app name, if needed in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] codecov-commenter commented on pull request #5016: Support to disable custom app name for spark engine
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1619950445
## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/5016?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#5016](https://app.codecov.io/gh/apache/kyuubi/pull/5016?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (e1c92df) into [master](https://app.codecov.io/gh/apache/kyuubi/commit/04ee20bd196a863f2eb755ec040135f2d0d3b265?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (04ee20b) will **not change** coverage.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #5016 +/- ##
======================================
Coverage 0.00% 0.00%
======================================
Files 563 563
Lines 31167 31177 +10
Branches 4070 4073 +3
======================================
- Misses 31167 31177 +10
```
| [Impacted Files](https://app.codecov.io/gh/apache/kyuubi/pull/5016?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://app.codecov.io/gh/apache/kyuubi/pull/5016?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...ain/scala/org/apache/kyuubi/engine/EngineRef.scala](https://app.codecov.io/gh/apache/kyuubi/pull/5016?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvRW5naW5lUmVmLnNjYWxh) | `0.00% <0.00%> (ø)` | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620942801
No problem, it was a real case in my production environment. It's embarrassinging that our administrator found it difficult to identify the target YARN app of engine instance in the list of mixed app names of MR/Spark/and etc when he was investigating other engine issues.
And let's see how other users interested in this minor feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] link3280 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "link3280 (via GitHub)" <gi...@apache.org>.
link3280 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1620937185
@bowenliang123 Thanks for your input! It might apply to Flink as well. Perhaps I should make a related PR for the Flink engine later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on pull request #5016: Support ignoring custom app name for Spark engine
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5016:
URL: https://github.com/apache/kyuubi/pull/5016#issuecomment-1630178006
How about introducing a `kyuubi.session.conf.sharable.engine.ignore.list`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org