You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/07/08 15:42:26 UTC
[GitHub] [hadoop] aajisaka opened a new pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
aajisaka opened a new pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190
JIRA: HADOOP-17794
Documented the following parameters:
```
hadoop.kms.authentication.zk-dt-secret-manager.enable
hadoop.kms.authentication.zk-dt-secret-manager.kerberos.keytab
hadoop.kms.authentication.zk-dt-secret-manager.kerberos.principal
hadoop.kms.authentication.zk-dt-secret-manager.zkConnectionString
hadoop.kms.authentication.zk-dt-secret-manager.znodeWorkingPath
hadoop.kms.authentication.zk-dt-secret-manager.zkAuthType
```
They are required to setup multiple KMS instances.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#discussion_r666863490
##########
File path: hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
##########
@@ -791,10 +791,62 @@ This secret sharing can be done using a Zookeeper service which is configured in
$H4 Delegation Tokens
Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens too.
+Under HA, every KMS instance must verify the delegation token given by another KMS instance.
+To do this, all the KMS instances must use ZKDelegationTokenSecretManager to retrieve
+the TokenIdentifiers and DelegationKeys from ZooKeeper.
-Under HA, A KMS instance must verify the delegation token given by another KMS instance, by checking the shared secret used to sign the delegation token. To do this, all KMS instances must be able to retrieve the shared secret from ZooKeeper.
+Sample configuration:
Review comment:
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
tasanuma commented on pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#issuecomment-877203600
Merged it. Thanks for your contribution, @aajisaka!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma merged pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
tasanuma merged pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#issuecomment-877246671
Thank you @tasanuma.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#issuecomment-876589562
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 39s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 45m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 32s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not generate ASF License warnings. |
| | | 62m 16s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3190/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3190 |
| Optional Tests | dupname asflicense mvnsite codespell |
| uname | Linux c1256f465fc6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 194aa0c6be7fa73e1c92bba4922f8193d24af0c7 |
| Max. process+thread count | 674 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3190/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] tasanuma commented on a change in pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
tasanuma commented on a change in pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#discussion_r666759085
##########
File path: hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
##########
@@ -791,10 +791,62 @@ This secret sharing can be done using a Zookeeper service which is configured in
$H4 Delegation Tokens
Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens too.
+Under HA, every KMS instance must verify the delegation token given by another KMS instance.
+To do this, all the KMS instances must use ZKDelegationTokenSecretManager to retrieve
+the TokenIdentifiers and DelegationKeys from ZooKeeper.
-Under HA, A KMS instance must verify the delegation token given by another KMS instance, by checking the shared secret used to sign the delegation token. To do this, all KMS instances must be able to retrieve the shared secret from ZooKeeper.
+Sample configuration:
Review comment:
How about adding the path of the configurations?
```suggestion
Sample configuration in `etc/hadoop/kms-site.xml`:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3190: HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3190:
URL: https://github.com/apache/hadoop/pull/3190#issuecomment-877145473
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 57m 42s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not generate ASF License warnings. |
| | | 79m 12s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3190/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3190 |
| Optional Tests | dupname asflicense mvnsite codespell |
| uname | Linux e1a1755ad4cd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4b6c0fdb072a7d1861fb9e9e3f8dbc9ec2855e7b |
| Max. process+thread count | 721 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-kms U: hadoop-common-project/hadoop-kms |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3190/2/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org