You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "zhangxiping1 (via GitHub)" <gi...@apache.org> on 2023/08/04 08:20:38 UTC
[GitHub] [hadoop] zhangxiping1 opened a new pull request, #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
zhangxiping1 opened a new pull request, #5921:
URL: https://github.com/apache/hadoop/pull/5921
### Description of PR
RBF: We changed the hadoop.security.auth_to_local configuration of one router, the other routers stopped working
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1723187323
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 39s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 8s | | trunk passed |
| +1 :green_heart: | compile | 18m 13s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 34s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 38s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 6s | | the patch passed |
| +1 :green_heart: | compile | 17m 28s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 28s | | the patch passed |
| +1 :green_heart: | compile | 16m 33s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 4m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 34s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 45s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 19m 11s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 239m 52s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not generate ASF License warnings. |
| | | 503m 58s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5921 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 71cdb505207d 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f79870bcf9ca3fb22f6672689d28e810c7d68dbd |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/4/testReport/ |
| Max. process+thread count | 2326 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1295083257
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
Review Comment:
I mean the constructor doesnt really need to be declared explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1722681250
window 10 environment, the test case runs successfully ，try to build again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri merged PR #5921:
URL: https://github.com/apache/hadoop/pull/5921
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1295084050
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
+ delegationTokenMaxLifetime,
+ delegationTokenRenewInterval,
+ delegationTokenRemoverScanInterval);
+ }
+
+ @Override
+ public DelegationTokenIdentifier createIdentifier() {
+ return null;
+ }
+
+ @Override
+ public void logExpireTokens(Collection<DelegationTokenIdentifier> expiredTokens) throws IOException {
+ super.logExpireTokens(expiredTokens);
+ }
+ }
+
+ @Test
+ public void testLogExpireTokensWhenChangeRules() {
+ MyDelegationTokenSecretManager myDtSecretManager =
+ new MyDelegationTokenSecretManager(10 * 1000, 10 * 1000, 10 * 1000, 10 * 1000);
+ setRules("RULE:[2:$1@$0](SomeUser.*)s/.*/SomeUser/");
+ DelegationTokenIdentifier dtId = new DelegationTokenIdentifier(
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"));
+ Set<DelegationTokenIdentifier> expiredTokens = new HashSet();
+ expiredTokens.add(dtId);
+
+ setRules("RULE:[2:$1@$0](OtherUser.*)s/.*/OtherUser/");
+ // rules was modified, causing the existing tokens (May be loaded from other storage systems like zookeeper)
+ // to fail to match the kerberos rules,
+ // return an exception that cannot be handled
+ try {
+ myDtSecretManager.logExpireTokens(expiredTokens);
+ } catch (Exception e) {
+ Assert.fail("Exception in logExpireTokens");
Review Comment:
But for the unit test, you can just do:
```
public void testLogExpireTokensWhenChangeRules() {
...
setRules("RULE:[2:$1@$0](SomeUser.*)s/.*/SomeUser/");
...
setRules("RULE:[2:$1@$0](OtherUser.*)s/.*/OtherUser/");
myDtSecretManager.logExpireTokens(expiredTokens);
}
```
No need to catch the exception, just let it surface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1707729841
@Hexiaoqiao @goiri Does this PR still need to be improved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1686158033
@goiri Modified according to your suggestion, can you help to see what is wrong with the test case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1716495453
@Hexiaoqiao can you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1720133126
I'm a little worried about the tests, those like TestRollingUpgrade seem fairly consistent but it might be because of something else. Can we double check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1718187234
This should be good to go.
Can you rebase to trigger a new build?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1291733842
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java:
##########
@@ -81,7 +81,12 @@ class AbstractDelegationTokenSecretManager<TokenIdent
= DelegationTokenSecretManagerMetrics.create();
private String formatTokenId(TokenIdent id) {
- return "(" + id + ")";
+ try {
+ return "(" + id + ")";
+ } catch (Exception e) {
+ LOG.warn("Exception in formatTokenId", e);
Review Comment:
What can this exception be?
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
+ delegationTokenMaxLifetime,
+ delegationTokenRenewInterval,
+ delegationTokenRemoverScanInterval);
+ }
+
+ @Override
+ public DelegationTokenIdentifier createIdentifier() {
+ return null;
+ }
+
+ @Override
+ public void logExpireTokens(Collection<DelegationTokenIdentifier> expiredTokens) throws IOException {
+ super.logExpireTokens(expiredTokens);
+ }
+ }
+
+ @Test
+ public void testLogExpireTokensWhenChangeRules() {
+ MyDelegationTokenSecretManager myDtSecretManager =
+ new MyDelegationTokenSecretManager(10 * 1000, 10 * 1000, 10 * 1000, 10 * 1000);
+ setRules("RULE:[2:$1@$0](SomeUser.*)s/.*/SomeUser/");
+ DelegationTokenIdentifier dtId = new DelegationTokenIdentifier(
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"));
+ Set<DelegationTokenIdentifier> expiredTokens = new HashSet();
+ expiredTokens.add(dtId);
+
+ setRules("RULE:[2:$1@$0](OtherUser.*)s/.*/OtherUser/");
+ // rules was modified, causing the existing tokens (May be loaded from other storage systems like zookeeper)
+ // to fail to match the kerberos rules,
+ // return an exception that cannot be handled
+ try {
+ myDtSecretManager.logExpireTokens(expiredTokens);
+ } catch (Exception e) {
+ Assert.fail("Exception in logExpireTokens");
Review Comment:
If you just let the exception go through, it will fail the test anyway.
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
Review Comment:
Doesn't this happen automatically?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1717454090
ok ，Thank you ，Can we end this PR now? Merge into the trunk branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1680352528
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 56s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 43s | | trunk passed |
| +1 :green_heart: | compile | 18m 58s | | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 18m 17s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 28s | | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 45s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 12s | | the patch passed |
| +1 :green_heart: | compile | 18m 11s | | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 18m 11s | | the patch passed |
| +1 :green_heart: | compile | 17m 57s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 4m 36s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 9s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 41m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 19m 1s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 237m 14s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 7s | | The patch does not generate ASF License warnings. |
| | | 524m 56s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5921 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b5a23c4b9d0f 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 05db88a35ba7ad994d4e9f9872f0808ed025c8c1 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/2/testReport/ |
| Max. process+thread count | 2591 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1720367350
> I'm a little worried about the tests, those like TestRollingUpgrade seem fairly consistent but it might be because of something else. Can we double check?
ok ，I'll check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1292903312
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
+ delegationTokenMaxLifetime,
+ delegationTokenRenewInterval,
+ delegationTokenRemoverScanInterval);
+ }
+
+ @Override
+ public DelegationTokenIdentifier createIdentifier() {
+ return null;
+ }
+
+ @Override
+ public void logExpireTokens(Collection<DelegationTokenIdentifier> expiredTokens) throws IOException {
+ super.logExpireTokens(expiredTokens);
+ }
+ }
+
+ @Test
+ public void testLogExpireTokensWhenChangeRules() {
+ MyDelegationTokenSecretManager myDtSecretManager =
+ new MyDelegationTokenSecretManager(10 * 1000, 10 * 1000, 10 * 1000, 10 * 1000);
+ setRules("RULE:[2:$1@$0](SomeUser.*)s/.*/SomeUser/");
+ DelegationTokenIdentifier dtId = new DelegationTokenIdentifier(
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"),
+ new Text("SomeUser/HOST@EXAMPLE.COM"));
+ Set<DelegationTokenIdentifier> expiredTokens = new HashSet();
+ expiredTokens.add(dtId);
+
+ setRules("RULE:[2:$1@$0](OtherUser.*)s/.*/OtherUser/");
+ // rules was modified, causing the existing tokens (May be loaded from other storage systems like zookeeper)
+ // to fail to match the kerberos rules,
+ // return an exception that cannot be handled
+ try {
+ myDtSecretManager.logExpireTokens(expiredTokens);
+ } catch (Exception e) {
+ Assert.fail("Exception in logExpireTokens");
Review Comment:
After fixing the code, there will be no exceptions, and if there are exceptions, the code has not been fixed properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1292902814
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
Review Comment:
Yes, the ExpiredTokenRemover thread will operate automatically, which is selected to be called automatically in the test case to determine if an exception has been generated. If an exception has been generated, it indicates that it has not been fixed.Any suggestions for improvement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1665913649
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 18m 0s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 35m 43s | | trunk passed |
| +1 :green_heart: | compile | 18m 9s | | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 16m 34s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 0s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 4s | | the patch passed |
| +1 :green_heart: | compile | 17m 20s | | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 17m 20s | | the patch passed |
| +1 :green_heart: | compile | 16m 26s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 31s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/1/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 53 unchanged - 0 fixed = 56 total (was 53) |
| +1 :green_heart: | mvnsite | 3m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 56s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 51s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 230m 5s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 7s | | The patch does not generate ASF License warnings. |
| | | 510m 3s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5921 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 74605a0faec8 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc0504ac98b9af96f257405b8717f4ea25e3f26b |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/1/testReport/ |
| Max. process+thread count | 2466 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1295294855
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/TestDelegationToken.java:
##########
@@ -376,4 +382,61 @@ public void testDelegationTokenIdentifierToString() throws Exception {
" for SomeUser with renewer JobTracker",
dtId.toStringStable());
}
+
+ public static class MyDelegationTokenSecretManager extends
+ AbstractDelegationTokenSecretManager<DelegationTokenIdentifier> {
+ /**
+ * Create a secret manager
+ *
+ * @param delegationKeyUpdateInterval the number of milliseconds for rolling
+ * new secret keys.
+ * @param delegationTokenMaxLifetime the maximum lifetime of the delegation
+ * tokens in milliseconds
+ * @param delegationTokenRenewInterval how often the tokens must be renewed
+ * in milliseconds
+ * @param delegationTokenRemoverScanInterval how often the tokens are scanned
+ * for expired tokens in milliseconds
+ */
+ public MyDelegationTokenSecretManager(long delegationKeyUpdateInterval,
+ long delegationTokenMaxLifetime, long delegationTokenRenewInterval,
+ long delegationTokenRemoverScanInterval) {
+ super(delegationKeyUpdateInterval,
Review Comment:
> I mean the constructor doesnt really need to be declared explicitly.
Thank you for your suggestion. Let me revise it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "goiri (via GitHub)" <gi...@apache.org>.
goiri commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1723920841
Unfortunate that these tests are so flaky.
It'd be good to tackle TestDirectoryScanner.
I'll go ahead merging this as it is unrelated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1719037994
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 38m 15s | | trunk passed |
| +1 :green_heart: | compile | 18m 40s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 10s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 5m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 12s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 37s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 7m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 59s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | | the patch passed |
| +1 :green_heart: | compile | 17m 59s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 59s | | the patch passed |
| +1 :green_heart: | compile | 17m 40s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 4m 42s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 34s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 41m 39s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 19m 7s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 166m 55s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +0 :ok: | asflicense | 1m 9s | | ASF License check generated no output? |
| | | 439m 39s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.balancer.TestBalancerRPCDelay |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.blockmanagement.TestSequentialBlockId |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | hadoop.hdfs.server.common.TestJspHelper |
| | hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5921 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bce36a6aab16 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 797d58b754645123661dde18f5c9e5fae06e1a16 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/3/testReport/ |
| Max. process+thread count | 3135 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5921/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "Hexiaoqiao (via GitHub)" <gi...@apache.org>.
Hexiaoqiao commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1292958875
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java:
##########
@@ -81,7 +81,12 @@ class AbstractDelegationTokenSecretManager<TokenIdent
= DelegationTokenSecretManagerMetrics.create();
private String formatTokenId(TokenIdent id) {
- return "(" + id + ")";
+ try {
+ return "(" + id + ")";
+ } catch (Exception e) {
+ LOG.warn("Exception in formatTokenId", e);
Review Comment:
As HDFS-17138 mentioned, here will throw NoMatchingRule if set unexpected kerberos rules.
https://github.com/apache/hadoop/blob/8d95c588d2df0048b0d3eb711d74bf34bf4ae3c4/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java#L428-L430
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhangxiping1 commented on a diff in pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…
Posted by "zhangxiping1 (via GitHub)" <gi...@apache.org>.
zhangxiping1 commented on code in PR #5921:
URL: https://github.com/apache/hadoop/pull/5921#discussion_r1292904295
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java:
##########
@@ -81,7 +81,12 @@ class AbstractDelegationTokenSecretManager<TokenIdent
= DelegationTokenSecretManagerMetrics.create();
private String formatTokenId(TokenIdent id) {
- return "(" + id + ")";
+ try {
+ return "(" + id + ")";
+ } catch (Exception e) {
+ LOG.warn("Exception in formatTokenId", e);
Review Comment:
I'm sorry I reply late, here will generate java. lang. IllegalArgumentException
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org