You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/12/20 12:53:09 UTC
[GitHub] [hadoop] surendralilhore opened a new pull request, #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
surendralilhore opened a new pull request, #5248:
URL: https://github.com/apache/hadoop/pull/5248
…in same JVM.
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
steveloughran commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1055492356
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:
##########
@@ -529,6 +529,13 @@ private void setLogin(LoginContext login) {
user.setLogin(login);
}
+ /** This method is only helpful for HadoopLoginContext*/
+ public boolean isLoginSuccess() {
+ LoginContext login = user.getLogin();
+ return (login instanceof HadoopLoginContext)
+ ? ((HadoopLoginContext) login).isLoginSuccess() : true;
Review Comment:
nit, put : on a newline
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
+ } else if (UserGroupInformation.isLoginTicketBased()) {
+ UserGroupInformation.getLoginUser().reloginFromTicketCache();
+ }
+ try {
+ // try processing message again
+ saslResponse = processSaslMessage(saslMessage);
+ AUDITLOG.info("Retry " + AUTH_SUCCESSFUL_FOR + this.toString()
+ + ":" + attemptingUser + " after failure");
+ } catch (IOException exp) {
+ tce = (IOException) getTrueCause(e);
Review Comment:
big assumption there about wrapped cause and its class. does it hold
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
+ } else if (UserGroupInformation.isLoginTicketBased()) {
+ UserGroupInformation.getLoginUser().reloginFromTicketCache();
+ }
+ try {
+ // try processing message again
+ saslResponse = processSaslMessage(saslMessage);
+ AUDITLOG.info("Retry " + AUTH_SUCCESSFUL_FOR + this.toString()
Review Comment:
use slf4j syntax
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
+ } else if (UserGroupInformation.isLoginTicketBased()) {
+ UserGroupInformation.getLoginUser().reloginFromTicketCache();
+ }
+ try {
+ // try processing message again
+ saslResponse = processSaslMessage(saslMessage);
Review Comment:
maybe log at debug that this is about to be retried
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
Review Comment:
this the right log level?
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:
##########
@@ -2083,6 +2090,10 @@ private static class HadoopLoginContext extends LoginContext {
this.conf = conf;
}
+ public boolean isLoginSuccess() {
Review Comment:
+javadoc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1061327469
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
Thanks @cnauroth.
@liuml07 Agree for new force API for re-login.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
cnauroth commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1053628775
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
If I trace through the chain of these re-login methods, they end up passing `false` for `ignoreLastLoginTime`. They'll skip the re-login and early exit if insufficient time (default 60 seconds) has elapsed since last login. Would that still leave a server potentially in a bad state for up to 60 seconds?
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:
##########
@@ -529,6 +529,13 @@ private void setLogin(LoginContext login) {
user.setLogin(login);
}
+ /** This method is only helpful for HadoopLoginContext*/
Review Comment:
There is a minor checkstyle warning here asking for a period at the end of the sentence.
However, perhaps consider expanding a bit. `HadoopLoginContext` is a private inner class, so probably best not to discuss it in a public Javadoc. You could discuss how this method checks for a successful Kerberos login, or defaults to `true` if not using Kerberos.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
liuml07 commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1370342353
It has been a while since last time I check the security code. My understanding fades away as security is complex and risky. Consider my comments non-binding here.
It makes sense to have to force re-login for addressing the issue.
1. Is it possible to figure out some unit tests (not necessarily NN+JN case) for Server and/or UGI? Even the current code change is straightforward, it may be broken by mistake or misunderstanding in future.
2. Do we need `Server#canTryForceLogin` to be thread-safe for multiple connections?
3. Is it clear to extract the new code in `Server` to a private helper method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1058376241
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
Thanks @cnauroth.
Added new API `forceReloginFromTicketCache()` and using both the force API in `Server.java`
>A drawback is that it's potentially a dangerous API if used incorrectly, because it could spam the KDC.
I have added check to use force login API only once in `Server.java` after failure and if it fails again then it will wait for 60 seconds. Handling this by adding **canTryForceLogin** in `Server.java.`
> We could add that, but expanding the public API footprint of UserGroupInformation should not be taken lightly.
Mostly people will use it for new development and they should aware of use case.
> Ideally, I'd like to get a second opinion from one more committer.
@liuml07 Please can you give your opinion here as you reviewed HADOOP-17159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1056248810
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
> Would that still leave a server potentially in a bad state for up to 60 seconds?
Yes, for 60 seconds server will in bad state. Earlier only option was to restart the server.
Below it the test log for 60 second from my test cluster :
`2022-12-23 10:27:19,117 INFO ipc.Server - Auth successful for hive/host1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS)
2022-12-23 10:27:19,121 INFO authorize.ServiceAuthorizationManager - Authorization successful for hive/host1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
2022-12-23 10:27:27,048 ERROR namenode.NameNode - Dummy logout thread...
org.apache.hadoop.security.KerberosAuthException: Login failure for user: nn/host1-xxxxxxxxxxxx@XXX.SERVER.COM javax.security.auth.login.LoginException: Re-login failed
at org.apache.hadoop.security.UserGroupInformation.unprotectedRelogin(UserGroupInformation.java:1203)
at org.apache.hadoop.hdfs.server.namenode.NameNode$2.run(NameNode.java:1590)
Caused by: javax.security.auth.login.LoginException: Re-login failed
at org.apache.hadoop.security.UserGroupInformation.unprotectedRelogin(UserGroupInformation.java:1188)
... 1 more
2022-12-23 10:27:28,786 WARN ipc.Server - Auth failed for 10.x.y.z:46879:null (GSS initiate failed) with true cause: (GSS initiate failed)
2022-12-23 10:27:28,786 INFO ipc.Server - Initiating re-login from IPC Server
2022-12-23 10:27:28,786 INFO ipc.Server - Doing login from keytab
2022-12-23 10:27:28,786 WARN security.UserGroupInformation - Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=1671791247048
.
.
.
.
.
2022-12-23 10:28:27,618 WARN ipc.Server - Auth failed for 10.x.y.z:45329:null (GSS initiate failed) with true cause: (GSS initiate failed)
2022-12-23 10:28:27,619 INFO ipc.Server - Initiating re-login from IPC Server
2022-12-23 10:28:27,619 INFO ipc.Server - Doing login from keytab
2022-12-23 10:28:27,652 INFO ipc.Server - Retry Auth successful for 10.x.y.z:45329:null after failure
2022-12-23 10:28:27,655 INFO ipc.Server - Auth successful for hive/hn1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS)
2022-12-23 10:28:27,667 INFO authorize.ServiceAuthorizationManager - Authorization successful for hive/hn1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1056306460
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
@cnauroth , any suggestion to handle this 60 second delay ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
cnauroth commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1058507029
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:
##########
@@ -1287,14 +1314,18 @@ private void reloginFromKeytab(boolean checkTGT, boolean ignoreLastLoginTime)
@InterfaceAudience.Public
@InterfaceStability.Evolving
public void reloginFromTicketCache() throws IOException {
- if (!shouldRelogin() || !isFromTicket()) {
+ reloginFromTicketCache(false);
+ }
+
+ private void reloginFromTicketCache(boolean ignoreLastLoginTime) throws IOException {
+ if (!shouldRelogin() || !isFromTicket()) {
Review Comment:
There is a Checkstyle warning here about indentation:
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:1321: if (!shouldRelogin() || !isFromTicket()) {: 'if' has incorrect indentation level 5, expected level should be 4. [Indentation]
```
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:
##########
@@ -529,6 +529,16 @@ private void setLogin(LoginContext login) {
user.setLogin(login);
}
+ /** This method checks for a successful Kerberos login
Review Comment:
This is generating a new JavaDoc warning:
```
[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5248/ubuntu-focal/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:535: warning: no @return
[ERROR] public boolean isLoginSuccess() {
[ERROR] ^
```
Additionally, I suggest sticking to the existing style of line break right after the opening and an asterisk on each line:
```
/**
* line 1
* line 2
*
* @return foo
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1369031680
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 23s | | trunk passed |
| +1 :green_heart: | compile | 23m 7s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 35s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| -1 :x: | javadoc | 1m 15s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/4/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 15s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 28s | | the patch passed |
| +1 :green_heart: | compile | 20m 23s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| -1 :x: | javadoc | 1m 7s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/4/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 14s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 21s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not generate ASF License warnings. |
| | | 205m 52s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4f38a72f4e33 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d2412f13af91a2fe9f91f2890f280223f235c2bc |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/4/testReport/ |
| Max. process+thread count | 1251 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1359664117
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 55s | | trunk passed |
| +1 :green_heart: | compile | 23m 15s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 33s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 44s | | trunk passed |
| -1 :x: | javadoc | 1m 17s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 23s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 22m 29s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 29s | | the patch passed |
| +1 :green_heart: | compile | 20m 27s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 1m 9s | [/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt) | hadoop-common-project/hadoop-common: The patch generated 1 new + 223 unchanged - 0 fixed = 224 total (was 223) |
| +1 :green_heart: | mvnsite | 1m 39s | | the patch passed |
| -1 :x: | javadoc | 1m 7s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 5s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 20s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not generate ASF License warnings. |
| | | 207m 7s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8e7a86fb979a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 95092a08213ef0662441fc41cb1e744db5e2ed3d |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/testReport/ |
| Max. process+thread count | 1251 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1363842249
> @surendralilhore , thank you for the patch. I entered a few questions. Additionally, can you please describe if you've been able to simulate the problem in testing to confirm that this patch fixes it? (I assume unit testing isn't practical for this.)
Yes, I simulated it by adding one thread in namenode and that thread will do logout in 2 minute after stating namenode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1370905767
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 17s | | trunk passed |
| +1 :green_heart: | compile | 23m 4s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 18s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| -1 :x: | javadoc | 1m 15s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/5/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 49s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 58s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 22m 24s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 24s | | the patch passed |
| +1 :green_heart: | compile | 20m 49s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 40s | | the patch passed |
| -1 :x: | javadoc | 1m 3s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/5/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 4s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 29s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not generate ASF License warnings. |
| | | 206m 0s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 17d088e9be4d 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f4a755b08bb7bcce7b6044f55e9db9be53dcd829 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/5/testReport/ |
| Max. process+thread count | 1251 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore merged pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore merged PR #5248:
URL: https://github.com/apache/hadoop/pull/5248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1058364619
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
+ } else if (UserGroupInformation.isLoginTicketBased()) {
+ UserGroupInformation.getLoginUser().reloginFromTicketCache();
+ }
+ try {
+ // try processing message again
+ saslResponse = processSaslMessage(saslMessage);
+ AUDITLOG.info("Retry " + AUTH_SUCCESSFUL_FOR + this.toString()
+ + ":" + attemptingUser + " after failure");
+ } catch (IOException exp) {
+ tce = (IOException) getTrueCause(e);
Review Comment:
There is proper null check inside getTrueCause, it will not return null.
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
+ } else if (UserGroupInformation.isLoginTicketBased()) {
+ UserGroupInformation.getLoginUser().reloginFromTicketCache();
+ }
+ try {
+ // try processing message again
+ saslResponse = processSaslMessage(saslMessage);
+ AUDITLOG.info("Retry " + AUTH_SUCCESSFUL_FOR + this.toString()
+ + ":" + attemptingUser + " after failure");
+ } catch (IOException exp) {
+ tce = (IOException) getTrueCause(e);
Review Comment:
There is proper null check inside getTrueCause(), it will not return null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1368849593
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 34s | | trunk passed |
| +1 :green_heart: | compile | 25m 27s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 56s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 42s | | trunk passed |
| -1 :x: | javadoc | 1m 16s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 28s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 22m 23s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 23s | | the patch passed |
| +1 :green_heart: | compile | 20m 22s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 22s | | the patch passed |
| -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/artifact/out/blanks-eol.txt) | The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 37s | | the patch passed |
| -1 :x: | javadoc | 1m 7s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 21s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 20s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not generate ASF License warnings. |
| | | 208m 53s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 566e48aa644e 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3eb7b95ba7f386e8da862e9d6e2be84671ae067b |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/testReport/ |
| Max. process+thread count | 3159 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
cnauroth commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1061900219
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -153,6 +154,13 @@ public abstract class Server {
private ExceptionsHandler exceptionsHandler = new ExceptionsHandler();
private Tracer tracer;
private AlignmentContext alignmentContext;
+
+ /**
+ * Allow server to do force Kerberos re-login once after failure irrespective
+ * of the last login time.
+ */
+ private AtomicBoolean canTryForceLogin = new AtomicBoolean(true);
Review Comment:
Please mark this as `final`.
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -3322,6 +3346,27 @@ protected Server(String bindAddress, int port,
metricsUpdaterInterval, metricsUpdaterInterval, TimeUnit.MILLISECONDS);
}
+ private synchronized void doKerberosRelogin() throws IOException {
+ if(UserGroupInformation.getLoginUser().isLoginSuccess()){
+ return;
+ }
+ LOG.warn("Initiating re-login from IPC Server");
+ if (canTryForceLogin.get()) {
Review Comment:
I suggest shortening this a little to:
```
if (canTryForceLogin.compareAndSet(true, false)) {
if (UserGroupInformation.isLoginKeytabBased()) {
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1058378415
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
Review Comment:
Changed it to Warn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1056240772
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
@cnauroth Thanks for review.
>@surendralilhore , thank you for the patch. I entered a few questions. Additionally, can you please describe if you've been able to simulate the problem in testing to confirm that this patch fixes it? (I assume unit testing isn't practical for this.)
Yes, I simulated it by adding one thread in namenode and that thread will do logout in 2 minute after stating namenode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1374252688
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 16s | | trunk passed |
| +1 :green_heart: | compile | 23m 14s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 26s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| -1 :x: | javadoc | 1m 15s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/6/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 23s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 22m 24s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 24s | | the patch passed |
| +1 :green_heart: | compile | 20m 25s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 39s | | the patch passed |
| -1 :x: | javadoc | 1m 6s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/6/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 8s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 19s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. |
| | | 205m 29s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9f1d5a9caf11 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3f0538ef948fdca63943cefd43a224b6be61f875 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/6/testReport/ |
| Max. process+thread count | 3159 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1374897142
Thanks @cnauroth , @liuml07 , @ayushtkn and @steveloughran for review..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1056248810
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
> Would that still leave a server potentially in a bad state for up to 60 seconds?
Yes, for 60 seconds server will in bad state. Earlier only option was to restart the server.
Below is the test log for 60 second from my test cluster, after 60 second it is successfully logged-in :
```
2022-12-23 10:27:19,117 INFO ipc.Server - Auth successful for hive/host1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS)
2022-12-23 10:27:19,121 INFO authorize.ServiceAuthorizationManager - Authorization successful for hive/host1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
2022-12-23 10:27:27,048 ERROR namenode.NameNode - Dummy logout thread...
org.apache.hadoop.security.KerberosAuthException: Login failure for user: nn/host1-xxxxxxxxxxxx@XXX.SERVER.COM javax.security.auth.login.LoginException: Re-login failed
at org.apache.hadoop.security.UserGroupInformation.unprotectedRelogin(UserGroupInformation.java:1203)
at org.apache.hadoop.hdfs.server.namenode.NameNode$2.run(NameNode.java:1590)
Caused by: javax.security.auth.login.LoginException: Re-login failed
at org.apache.hadoop.security.UserGroupInformation.unprotectedRelogin(UserGroupInformation.java:1188)
... 1 more
2022-12-23 10:27:28,786 WARN ipc.Server - Auth failed for 10.x.y.z:46879:null (GSS initiate failed) with true cause: (GSS initiate failed)
2022-12-23 10:27:28,786 INFO ipc.Server - Initiating re-login from IPC Server
2022-12-23 10:27:28,786 INFO ipc.Server - Doing login from keytab
2022-12-23 10:27:28,786 WARN security.UserGroupInformation - Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=1671791247048
.
.
.
.
.
2022-12-23 10:28:27,618 WARN ipc.Server - Auth failed for 10.x.y.z:45329:null (GSS initiate failed) with true cause: (GSS initiate failed)
2022-12-23 10:28:27,619 INFO ipc.Server - Initiating re-login from IPC Server
2022-12-23 10:28:27,619 INFO ipc.Server - Doing login from keytab
2022-12-23 10:28:27,652 INFO ipc.Server - Retry Auth successful for 10.x.y.z:45329:null after failure
2022-12-23 10:28:27,655 INFO ipc.Server - Auth successful for hive/hn1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS)
2022-12-23 10:28:27,667 INFO authorize.ServiceAuthorizationManager - Authorization successful for hive/hn1-xxxxxxxxxxxx@XXX.SERVER.COM (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.protocol.ClientProtocol
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1366808788
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 36s | | trunk passed |
| +1 :green_heart: | compile | 23m 15s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 22m 30s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 52s | | trunk passed |
| -1 :x: | javadoc | 1m 18s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 22m 20s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 20s | | the patch passed |
| +1 :green_heart: | compile | 20m 29s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 1m 9s | [/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt) | hadoop-common-project/hadoop-common: The patch generated 1 new + 223 unchanged - 0 fixed = 224 total (was 223) |
| +1 :green_heart: | mvnsite | 1m 40s | | the patch passed |
| -1 :x: | javadoc | 1m 9s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 9s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 18s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not generate ASF License warnings. |
| | | 209m 36s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5248 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8a804bdf1d5c 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 67e9d558910969083094776a5c438d97812bc866 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/testReport/ |
| Max. process+thread count | 1845 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5248/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
cnauroth commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1057844215
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
For keytab usage, there is `UserGroupInformation#forceReloginFromKeytab()`, which always does the login regardless of time since last login. There is no equivalent `forceReloginFromTicketCache()` though. We could add that, but expanding the public API footprint of `UserGroupInformation` should not be taken lightly. Ideally, I'd like to get a second opinion from one more committer. I think it's the right thing to do. A drawback is that it's potentially a dangerous API if used incorrectly, because it could spam the KDC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#issuecomment-1370726197
Thanks @liuml07 for review.
> 1. Is it possible to figure out some unit tests (not necessarily NN+JN case) for Server and/or UGI? Even the current code change is straightforward, it may be broken by mistake or misunderstanding in future.
Adding UT for this scenario is difficult. Passing Sasl message to server without any proper channel and making it fail is difficult.
> 2. Do we need `Server#canTryForceLogin` to be thread-safe for multiple connections?
Changed `Server#canTryForceLogin` to `AtomicBoolean`
> 3. Is it clear to extract the new code in `Server` to a private helper method?
Re-login logic extracted in new method and made it synchronized to avoid multiple re-relogin in concurrent scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a diff in pull request #5248: HADOOP-18581 : Handle Server KDC re-login when Server and Client run …
Posted by GitBox <gi...@apache.org>.
surendralilhore commented on code in PR #5248:
URL: https://github.com/apache/hadoop/pull/5248#discussion_r1061327469
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:
##########
@@ -2206,7 +2206,25 @@ private void saslProcess(RpcSaslProto saslMessage)
AUDITLOG.warn(AUTH_FAILED_FOR + this.toString() + ":"
+ attemptingUser + " (" + e.getLocalizedMessage()
+ ") with true cause: (" + tce.getLocalizedMessage() + ")");
- throw tce;
+ if (!UserGroupInformation.getLoginUser().isLoginSuccess()) {
+ LOG.info("Initiating re-login from IPC Server");
+ if (UserGroupInformation.isLoginKeytabBased()) {
+ UserGroupInformation.getLoginUser().reloginFromKeytab();
Review Comment:
Thanks @cnauroth.
@liuml07, Agreed for new force API for re-login.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org