You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "Yikf (via GitHub)" <gi...@apache.org> on 2023/12/30 03:43:57 UTC
[PR] [KYUUBI #5935] Engine YARN mode support proxy user [kyuubi]
Yikf opened a new pull request, #5936:
URL: https://github.com/apache/kyuubi/pull/5936
# :mag: Description
## Issue References ๐
<!-- Append the issue number after #. If there is no issue for you to link create one or -->
<!-- If there are no issues to link, please provide details here. -->
This pull request fixes https://github.com/apache/kyuubi/issues/5935
## Describe Your Solution ๐ง
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
## Types of changes :bookmark:
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan ๐งช
#### Behavior Without This Pull Request :coffin:
#### Behavior With This Pull Request :tada:
#### Related Unit Tests
---
# Checklist ๐
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy appUser [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #5936:
URL: https://github.com/apache/kyuubi/pull/5936#discussion_r1438487393
##########
kyuubi-common/src/main/scala/org/apache/kyuubi/engine/deploy/yarn/EngineYarnModeSubmitter.scala:
##########
@@ -92,6 +94,28 @@ abstract class EngineYarnModeSubmitter extends Logging {
def engineExtraJars(): Seq[File] = Seq.empty
protected def submitApplication(): Unit = {
+ val user = kyuubiConf.getOption(KYUUBI_SESSION_USER_KEY)
Review Comment:
we actually have seen that limitation of always using session user as the engine launch user, let's define a dedicated `proxyUser: Option[String]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy appUser [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5936:
URL: https://github.com/apache/kyuubi/pull/5936#issuecomment-1872472443
BTW, for hadoop-testing environment, the recommended `HIVE_HADOOP_CLASSPATH` is `/opt/hadoop/share/hadoop/client/*`, there are only Hadoop Shaded client jars
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy appUser [kyuubi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5936:
URL: https://github.com/apache/kyuubi/pull/5936#issuecomment-1872453337
## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: `8 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`5d59cf1`)](https://app.codecov.io/gh/apache/kyuubi/commit/5d59cf1d6a0ab973286804690a2a0d4f2ac32b4a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 61.21% compared to head [(`08a959a`)](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 61.13%.
| [Files](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...i/engine/deploy/yarn/EngineYarnModeSubmitter.scala](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9lbmdpbmUvZGVwbG95L3lhcm4vRW5naW5lWWFybk1vZGVTdWJtaXR0ZXIuc2NhbGE=) | 0.00% | [8 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #5936 +/- ##
============================================
- Coverage 61.21% 61.13% -0.08%
Complexity 23 23
============================================
Files 621 621
Lines 36864 36872 +8
Branches 5014 5016 +2
============================================
- Hits 22566 22543 -23
- Misses 11869 11889 +20
- Partials 2429 2440 +11
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/5936?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy appUser [kyuubi]
Posted by "Yikf (via GitHub)" <gi...@apache.org>.
Yikf closed pull request #5936: [KYUUBI #5935] Engine YARN mode support proxy appUser
URL: https://github.com/apache/kyuubi/pull/5936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy appUser [kyuubi]
Posted by "Yikf (via GitHub)" <gi...@apache.org>.
Yikf commented on code in PR #5936:
URL: https://github.com/apache/kyuubi/pull/5936#discussion_r1452004696
##########
kyuubi-common/src/main/scala/org/apache/kyuubi/engine/deploy/yarn/EngineYarnModeSubmitter.scala:
##########
@@ -92,6 +94,28 @@ abstract class EngineYarnModeSubmitter extends Logging {
def engineExtraJars(): Seq[File] = Seq.empty
protected def submitApplication(): Unit = {
+ val user = kyuubiConf.getOption(KYUUBI_SESSION_USER_KEY)
Review Comment:
The user retrieved in this context is identified as appUser, which is supplied via the ProcessBuilder using the --conf parameter. However, the naming of the configuration key associated with this user appears to be somewhat unclear.
Are you suggesting that the key name should be updated for clarity, or are you indicating that appUser does not represent the intended proxy user?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #5935] Engine YARN mode support proxy user [kyuubi]
Posted by "Yikf (via GitHub)" <gi...@apache.org>.
Yikf commented on PR #5936:
URL: https://github.com/apache/kyuubi/pull/5936#issuecomment-1872442529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org