You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/09/28 17:06:23 UTC
[GitHub] [incubator-kyuubi] cxzl25 opened a new pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
cxzl25 opened a new pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177
### _Why are the changes needed?_
https://github.com/apache/incubator-kyuubi/issues/1176
When `kyuubi.ha.zookeeper.acl.enabled=true`, both service and engine will use zookeeper acl to create znode, but engine has no keytab information and cannot write information to zookeeper, throwing an exception.
```java
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /kyuubi_USER/XXXX
at org.apache.zookeeper.KeeperException.create(KeeperException.java:124)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792)
at org.apache.kyuubi.shade.org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:740)
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724798765
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724842837
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
good point, seems branch-1.3 would fail. Can you send a PR target to branch-1.3 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
pan3793 commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-941022834
Change milestone to 1.4.0, see #1221
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-929416927
# Current

# Fix

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718332276
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionManager.scala
##########
@@ -48,14 +49,18 @@ class KyuubiSessionManager private (name: String) extends SessionManager(name) {
val username = Option(user).filter(_.nonEmpty).getOrElse("anonymous")
+ val sessionConf = this.getConf.getUserDefaults(user)
+ if (!sessionConf.get(HA_ZK_ACL_ENGINE_ENABLED)) {
Review comment:
Move to `ZooKeeperACLProvider`
##########
File path: kyuubi-ha/src/test/scala/org/apache/kyuubi/ha/client/ServiceDiscoverySuite.scala
##########
@@ -98,17 +100,31 @@ class ServiceDiscoverySuite extends KerberizedTestHelper {
}
test("acl for zookeeper") {
- val provider = new ZooKeeperACLProvider(conf)
- val acl = provider.getDefaultAcl
- assert(acl.size() === 1)
- assert(acl === ZooDefs.Ids.OPEN_ACL_UNSAFE)
-
- val conf1 = conf.clone.set(HA_ZK_ACL_ENABLED, true)
- val acl1 = new ZooKeeperACLProvider(conf1).getDefaultAcl
- assert(acl1.size() === 2)
- val expected = ZooDefs.Ids.READ_ACL_UNSAFE
- expected.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
Review comment:
This will modify the original list
##########
File path: kyuubi-ha/src/test/scala/org/apache/kyuubi/ha/client/ServiceDiscoverySuite.scala
##########
@@ -98,17 +100,31 @@ class ServiceDiscoverySuite extends KerberizedTestHelper {
}
test("acl for zookeeper") {
- val provider = new ZooKeeperACLProvider(conf)
- val acl = provider.getDefaultAcl
- assert(acl.size() === 1)
- assert(acl === ZooDefs.Ids.OPEN_ACL_UNSAFE)
-
- val conf1 = conf.clone.set(HA_ZK_ACL_ENABLED, true)
- val acl1 = new ZooKeeperACLProvider(conf1).getDefaultAcl
- assert(acl1.size() === 2)
- val expected = ZooDefs.Ids.READ_ACL_UNSAFE
- expected.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
Review comment:
This (`expected.addAll`) will modify the original list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-929508909
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1177](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4a28756) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/191d30e2ac642dc441979c61780a6de4417464e0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (191d30e) will **increase** coverage by `0.04%`.
> The diff coverage is `78.75%`.
> :exclamation: Current head 4a28756 differs from pull request most recent head 0b7cc2e. Consider uploading reports for the commit 0b7cc2e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1177 +/- ##
============================================
+ Coverage 78.79% 78.84% +0.04%
Complexity 93 93
============================================
Files 182 182
Lines 6654 6683 +29
Branches 785 784 -1
============================================
+ Hits 5243 5269 +26
- Misses 961 966 +5
+ Partials 450 448 -2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/session/AbstractSession.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0Fic3RyYWN0U2Vzc2lvbi5zY2FsYQ==) | `95.40% <0.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `63.88% <46.15%> (-1.39%)` | :arrow_down: |
| [...g/apache/kyuubi/session/KyuubiSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25NYW5hZ2VyLnNjYWxh) | `94.73% <80.00%> (-2.33%)` | :arrow_down: |
| [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `79.16% <100.00%> (-7.20%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.75% <100.00%> (-0.14%)` | :arrow_down: |
| [.../engine/spark/session/SparkSQLSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU1FMU2Vzc2lvbk1hbmFnZXIuc2NhbGE=) | `92.10% <100.00%> (+0.43%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.33% <100.00%> (+0.08%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/log/OperationLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL09wZXJhdGlvbkxvZy5zY2FsYQ==) | `97.82% <100.00%> (+2.17%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/session/SessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25NYW5hZ2VyLnNjYWxh) | `56.36% <100.00%> (+3.42%)` | :arrow_up: |
| ... and [4 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [191d30e...0b7cc2e](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-929417763
https://github.com/apache/incubator-kyuubi/blob/4f74aba50e683fc7d58e793f92ff6db75e09f7b8/kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/ZooKeeperClientProvider.scala#L91-L104
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724832305
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
`branch-1.3` lacks `HA_ZK_ENGINE_REF_ID` configuration to distinguish between service and engine.
It should be compilation failure now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724856786
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
Do you mean to merge https://github.com/apache/incubator-kyuubi/issues/1032 into `branch-1.3`? And change the version of `ha.engine.ref.id` to 1.3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718117750
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionManager.scala
##########
@@ -48,14 +49,18 @@ class KyuubiSessionManager private (name: String) extends SessionManager(name) {
val username = Option(user).filter(_.nonEmpty).getOrElse("anonymous")
+ val sessionConf = this.getConf.getUserDefaults(user)
+ if (!sessionConf.get(HA_ZK_ACL_ENGINE_ENABLED)) {
Review comment:
It‘s a bit hacky, can we use HA_ZK_ACL_ENGINE_ENABLED at the caller side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-929416927
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] ulysses-you commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724798209
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
@cxzl25 sorry for the late, can we send a followup change the version to 1.3.1 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
pan3793 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r719085787
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/ZooKeeperACLProvider.scala
##########
@@ -33,11 +33,20 @@ class ZooKeeperACLProvider(conf: KyuubiConf) extends ACLProvider {
*/
override lazy val getDefaultAcl: java.util.List[ACL] = {
val nodeAcls = new java.util.ArrayList[ACL]
- if (conf.get(HighAvailabilityConf.HA_ZK_ACL_ENABLED)) {
+
+ def addACL(): Unit = {
// Read all to the world
nodeAcls.addAll(ZooDefs.Ids.READ_ACL_UNSAFE)
// Create/Delete/Write/Admin to the authenticated user
nodeAcls.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
+ }
+
+ if (conf.get(HighAvailabilityConf.HA_ZK_ACL_ENABLED) &&
+ conf.get(HighAvailabilityConf.HA_ZK_ENGINE_REF_ID).isEmpty) {
Review comment:
@ulysses-you does this sufficient to distinguish between the server side and the engine side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-938281581
thanks, merged to master/1.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718332912
##########
File path: kyuubi-ha/src/test/scala/org/apache/kyuubi/ha/client/ServiceDiscoverySuite.scala
##########
@@ -98,17 +100,31 @@ class ServiceDiscoverySuite extends KerberizedTestHelper {
}
test("acl for zookeeper") {
- val provider = new ZooKeeperACLProvider(conf)
- val acl = provider.getDefaultAcl
- assert(acl.size() === 1)
- assert(acl === ZooDefs.Ids.OPEN_ACL_UNSAFE)
-
- val conf1 = conf.clone.set(HA_ZK_ACL_ENABLED, true)
- val acl1 = new ZooKeeperACLProvider(conf1).getDefaultAcl
- assert(acl1.size() === 2)
- val expected = ZooDefs.Ids.READ_ACL_UNSAFE
- expected.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
Review comment:
This (`expected.addAll`) will modify the original list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718332912
##########
File path: kyuubi-ha/src/test/scala/org/apache/kyuubi/ha/client/ServiceDiscoverySuite.scala
##########
@@ -98,17 +100,31 @@ class ServiceDiscoverySuite extends KerberizedTestHelper {
}
test("acl for zookeeper") {
- val provider = new ZooKeeperACLProvider(conf)
- val acl = provider.getDefaultAcl
- assert(acl.size() === 1)
- assert(acl === ZooDefs.Ids.OPEN_ACL_UNSAFE)
-
- val conf1 = conf.clone.set(HA_ZK_ACL_ENABLED, true)
- val acl1 = new ZooKeeperACLProvider(conf1).getDefaultAcl
- assert(acl1.size() === 2)
- val expected = ZooDefs.Ids.READ_ACL_UNSAFE
- expected.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
Review comment:
This will modify the original list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718332276
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionManager.scala
##########
@@ -48,14 +49,18 @@ class KyuubiSessionManager private (name: String) extends SessionManager(name) {
val username = Option(user).filter(_.nonEmpty).getOrElse("anonymous")
+ val sessionConf = this.getConf.getUserDefaults(user)
+ if (!sessionConf.get(HA_ZK_ACL_ENGINE_ENABLED)) {
Review comment:
Move to `ZooKeeperACLProvider`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r719102228
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/client/ZooKeeperACLProvider.scala
##########
@@ -33,11 +33,20 @@ class ZooKeeperACLProvider(conf: KyuubiConf) extends ACLProvider {
*/
override lazy val getDefaultAcl: java.util.List[ACL] = {
val nodeAcls = new java.util.ArrayList[ACL]
- if (conf.get(HighAvailabilityConf.HA_ZK_ACL_ENABLED)) {
+
+ def addACL(): Unit = {
// Read all to the world
nodeAcls.addAll(ZooDefs.Ids.READ_ACL_UNSAFE)
// Create/Delete/Write/Admin to the authenticated user
nodeAcls.addAll(ZooDefs.Ids.CREATOR_ALL_ACL)
+ }
+
+ if (conf.get(HighAvailabilityConf.HA_ZK_ACL_ENABLED) &&
+ conf.get(HighAvailabilityConf.HA_ZK_ENGINE_REF_ID).isEmpty) {
Review comment:
I don’t seem to find other ways to distinguish server and engine, or use main class to distinguish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r718117750
##########
File path: kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiSessionManager.scala
##########
@@ -48,14 +49,18 @@ class KyuubiSessionManager private (name: String) extends SessionManager(name) {
val username = Option(user).filter(_.nonEmpty).getOrElse("anonymous")
+ val sessionConf = this.getConf.getUserDefaults(user)
+ if (!sessionConf.get(HA_ZK_ACL_ENGINE_ENABLED)) {
Review comment:
It‘s a bit hacky, can we use HA_ZK_ACL_ENGINE_ENABLED at the caller side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] codecov-commenter commented on pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#issuecomment-929508909
# [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1177](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4a28756) into [master](https://codecov.io/gh/apache/incubator-kyuubi/commit/191d30e2ac642dc441979c61780a6de4417464e0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (191d30e) will **increase** coverage by `0.04%`.
> The diff coverage is `78.75%`.
> :exclamation: Current head 4a28756 differs from pull request most recent head 0b7cc2e. Consider uploading reports for the commit 0b7cc2e to get more accurate results
[](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1177 +/- ##
============================================
+ Coverage 78.79% 78.84% +0.04%
Complexity 93 93
============================================
Files 182 182
Lines 6654 6683 +29
Branches 785 784 -1
============================================
+ Hits 5243 5269 +26
- Misses 961 966 +5
+ Partials 450 448 -2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ubi/engine/spark/operation/PlanOnlyStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vUGxhbk9ubHlTdGF0ZW1lbnQuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...la/org/apache/kyuubi/session/AbstractSession.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0Fic3RyYWN0U2Vzc2lvbi5zY2FsYQ==) | `95.40% <0.00%> (ø)` | |
| [...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9TcGFya1NRTEVuZ2luZS5zY2FsYQ==) | `63.88% <46.15%> (-1.39%)` | :arrow_down: |
| [...g/apache/kyuubi/session/KyuubiSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLXNlcnZlci9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL0t5dXViaVNlc3Npb25NYW5hZ2VyLnNjYWxh) | `94.73% <80.00%> (-2.33%)` | :arrow_down: |
| [...uubi/engine/spark/events/EventLoggingService.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9ldmVudHMvRXZlbnRMb2dnaW5nU2VydmljZS5zY2FsYQ==) | `79.16% <100.00%> (-7.20%)` | :arrow_down: |
| [...uubi/engine/spark/operation/ExecuteStatement.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9vcGVyYXRpb24vRXhlY3V0ZVN0YXRlbWVudC5zY2FsYQ==) | `88.75% <100.00%> (-0.14%)` | :arrow_down: |
| [.../engine/spark/session/SparkSQLSessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZXh0ZXJuYWxzL2t5dXViaS1zcGFyay1zcWwtZW5naW5lL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUva3l1dWJpL2VuZ2luZS9zcGFyay9zZXNzaW9uL1NwYXJrU1FMU2Vzc2lvbk1hbmFnZXIuc2NhbGE=) | `92.10% <100.00%> (+0.43%)` | :arrow_up: |
| [...in/scala/org/apache/kyuubi/config/KyuubiConf.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9jb25maWcvS3l1dWJpQ29uZi5zY2FsYQ==) | `95.33% <100.00%> (+0.08%)` | :arrow_up: |
| [...org/apache/kyuubi/operation/log/OperationLog.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9vcGVyYXRpb24vbG9nL09wZXJhdGlvbkxvZy5zY2FsYQ==) | `97.82% <100.00%> (+2.17%)` | :arrow_up: |
| [...ala/org/apache/kyuubi/session/SessionManager.scala](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-a3l1dWJpLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2t5dXViaS9zZXNzaW9uL1Nlc3Npb25NYW5hZ2VyLnNjYWxh) | `56.36% <100.00%> (+3.42%)` | :arrow_up: |
| ... and [4 more](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [191d30e...0b7cc2e](https://codecov.io/gh/apache/incubator-kyuubi/pull/1177?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] cxzl25 commented on a change in pull request #1177: [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on a change in pull request #1177:
URL: https://github.com/apache/incubator-kyuubi/pull/1177#discussion_r724808940
##########
File path: kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
##########
@@ -48,6 +48,13 @@ object HighAvailabilityConf {
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
+ val HA_ZK_ACL_ENGINE_ENABLED: ConfigEntry[Boolean] =
+ buildConf("ha.zookeeper.acl.engine.enabled")
+ .doc("Set to true if the zookeeper ensemble is kerberized at engine side.")
+ .version("1.4.0")
Review comment:
see https://github.com/apache/incubator-kyuubi/pull/1193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org