You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/07/13 21:25:18 UTC
[GitHub] [hadoop] KevinWikant opened a new pull request, #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
KevinWikant opened a new pull request, #4563:
URL: https://github.com/apache/hadoop/pull/4563
### Description of PR
Applications which call YARN RMAdminCLI (i.e. YARN ResourceManager client) synchronously can be blocked for up to 15 minutes with the default configuration of "yarn.resourcemanager.connect.max-wait.ms"; this is not an issue in of itself, but there is a non-retryable IllegalArgumentException exception thrown within the YARN ResourceManager client that is getting swallowed & treated as a retryable "connection exception" meaning that it gets retried for 15 minutes.
The purpose of this JIRA (and PR) is to modify the YARN client so that it does not retry on this non-retryable exception.
See JIRA for additional details: https://issues.apache.org/jira/browse/YARN-11210
### How was this patch tested?
- Create Kerberized YARN cluster
- Run YARN rmadmin client & validate it completes successfully
```
> yarn rmadmin -refreshNodes;
22/07/13 15:30:45 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8033
>> Success is silent in client logs, but can be seen in the ResourceManager logs <<
```
- Unset the value of "yarn.resourcemanager.principal" is "yarn-site.xml"
- Run YARN rmadmin client & validate it retries for 15 minutes
```
> yarn rmadmin -refreshNodes;
22/06/28 14:23:45 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8033
22/06/28 14:23:46 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8033. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
...
22/06/28 14:23:55 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8033. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
22/06/28 14:23:56 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8033. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
...
22/06/28 14:24:05 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8033. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
22/06/28 14:24:05 INFO retry.RetryInvocationHandler: java.net.ConnectException: Your endpoint configuration is wrong; For more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort, while invoking ResourceManagerAdministrationProtocolPBClientImpl.refreshNodes over null after 1 failover attempts. Trying to failover after sleeping for 27166ms.
22/06/28 14:24:32 INFO retry.RetryInvocationHandler: java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "0.0.0.0/0.0.0.0"; destination host is: "0.0.0.0":8033; , while invoking ResourceManagerAdministrationProtocolPBClientImpl.refreshNodes over null after 2 failover attempts. Trying to failover after sleeping for 22291ms.
...
22/06/28 14:37:57 INFO retry.RetryInvocationHandler: java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "0.0.0.0/0.0.0.0"; destination host is: "0.0.0.0":8033; , while invoking ResourceManagerAdministrationProtocolPBClientImpl.refreshNodes over null after 28 failover attempts. Trying to failover after sleeping for 26721ms.
22/06/28 14:38:23 INFO retry.RetryInvocationHandler: java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "0.0.0.0/0.0.0.0"; destination host is: "0.0.0.0":8033; , while invoking ResourceManagerAdministrationProtocolPBClientImpl.refreshNodes over null after 29 failover attempts. Trying to failover after sleeping for 27641ms.
refreshNodes: Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "0.0.0.0/0.0.0.0"; destination host is: "0.0.0.0":8033;
```
- Modify YARN client runtime classpath to contain the changes in this PR
- Run YARN rmadmin client & validate it fails after 1 try (tested in both federation enabled & non-federation enabled clusters)
```
> yarn rmadmin -refreshNodes;
22/07/13 17:37:50 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8033
22/07/13 17:37:50 WARN ipc.Client: Exception encountered while connecting to the server
javax.security.sasl.SaslException: Bad Kerberos server principal configuration [Caused by java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name]
at org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:237)
at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:159)
at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:397)
at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:630)
at org.apache.hadoop.ipc.Client$Connection.access$2200(Client.java:424)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:825) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:821)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:821)
at org.apache.hadoop.ipc.Client$Connection.access$3700(Client.java:424)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1612)
at org.apache.hadoop.ipc.Client.call(Client.java:1442)
at org.apache.hadoop.ipc.Client.call(Client.java:1395)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy7.refreshNodes(Unknown Source)
at org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceManagerAdministrationProtocolPBClientImpl.refreshNodes(ResourceManagerAdministrationProtocolPBClientImpl.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
at com.sun.proxy.$Proxy8.refreshNodes(Unknown Source)
at org.apache.hadoop.yarn.client.cli.RMAdminCLI.refreshNodes(RMAdminCLI.java:349)
at org.apache.hadoop.yarn.client.cli.RMAdminCLI.refreshNodes(RMAdminCLI.java:423)
at org.apache.hadoop.yarn.client.cli.RMAdminCLI.handleRefreshNodes(RMAdminCLI.java:917)
at org.apache.hadoop.yarn.client.cli.RMAdminCLI.run(RMAdminCLI.java:816)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.yarn.client.cli.RMAdminCLI.main(RMAdminCLI.java:1027)
Caused by: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
at org.apache.hadoop.security.SaslRpcClient.getServerPrincipal(SaslRpcClient.java:332)
at org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:233)
... 35 more
refreshNodes: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: Bad Kerberos server principal configuration [Caused by java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name]; Host Details : local host is: "0.0.0.0/0.0.0.0"; destination host is: "0.0.0.0":8033;
```
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [n/a] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [n/a] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [n/a] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] PrabhuJoseph commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
PrabhuJoseph commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194965682
Thanks @KevinWikant for the patch and @ashutoshcipher for additional review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194363279
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 11s | | trunk passed |
| +1 :green_heart: | compile | 25m 9s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 47s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 59s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 24s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 14s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 33s | | the patch passed |
| +1 :green_heart: | compile | 24m 11s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 11s | | the patch passed |
| +1 :green_heart: | compile | 21m 53s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 22s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/artifact/out/results-checkstyle-root.txt) | root: The patch generated 6 new + 229 unchanged - 3 fixed = 235 total (was 232) |
| +1 :green_heart: | mvnsite | 4m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 49s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 16s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 22s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 55s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 31s | | hadoop-yarn-api in the patch passed. |
| +1 :green_heart: | unit | 5m 20s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not generate ASF License warnings. |
| | | 269m 59s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4563 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3b4ee9e513bd 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 28b2c47600cc36e20a866efbf5127228f7582735 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/testReport/ |
| Max. process+thread count | 3134 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1184714889
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 15s | | trunk passed |
| +1 :green_heart: | compile | 23m 0s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 40s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 35s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 20s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 51s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 39s | | the patch passed |
| +1 :green_heart: | compile | 22m 12s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 12s | | the patch passed |
| +1 :green_heart: | compile | 20m 29s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 19s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/2/artifact/out/results-checkstyle-root.txt) | root: The patch generated 8 new + 229 unchanged - 3 fixed = 237 total (was 232) |
| +1 :green_heart: | mvnsite | 5m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 40s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 19s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 1s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 47s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 50s | | hadoop-yarn-api in the patch passed. |
| +1 :green_heart: | unit | 5m 37s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 35s | | The patch does not generate ASF License warnings. |
| | | 262m 38s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4563 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2bb50814f343 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a1d43322497a72a4f4889538d3a27b196051be8f |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/2/testReport/ |
| Max. process+thread count | 2999 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] KevinWikant commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
KevinWikant commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194540951
@PrabhuJoseph I have addressed all the checkstyle issues except for the SaslRpcClient Indentation issue which, as I mentioned, would result in inconsistent indentation if addressed:
```
case KERBEROS: {
if (ugi.getRealAuthenticationMethod().getAuthMethod() !=
AuthMethod.KERBEROS) {
LOG.debug("client isn't using kerberos");
return null;
}
final String serverPrincipal;
try {
serverPrincipal = getServerPrincipal(authType);
} catch (IllegalArgumentException ex) {
// YARN-11210: getServerPrincipal can throw IllegalArgumentException if Kerberos
// configuration is bad, this is surfaced as a non-retryable SaslException
throw new SaslException("Bad Kerberos server principal configuration", ex);
}
if (serverPrincipal == null) {
LOG.debug("protocol doesn't use kerberos");
return null;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] PrabhuJoseph merged pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
PrabhuJoseph merged PR #4563:
URL: https://github.com/apache/hadoop/pull/4563
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1183864896
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 0s | | trunk passed |
| +1 :green_heart: | compile | 23m 1s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 39s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 48s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 20s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 54s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 40s | | the patch passed |
| +1 :green_heart: | compile | 22m 14s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 14s | | the patch passed |
| +1 :green_heart: | compile | 20m 32s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 49s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/1/artifact/out/results-checkstyle-root.txt) | root: The patch generated 8 new + 229 unchanged - 3 fixed = 237 total (was 232) |
| +1 :green_heart: | mvnsite | 5m 29s | | the patch passed |
| -1 :x: | javadoc | 1m 38s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 4m 20s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 7s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 52s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 51s | | hadoop-yarn-api in the patch passed. |
| +1 :green_heart: | unit | 5m 30s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 36s | | The patch does not generate ASF License warnings. |
| | | 263m 22s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4563 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 25775d402ad9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 514c5dfa2a19f8dc635fefdb73bea4c4d875cf86 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/1/testReport/ |
| Max. process+thread count | 1263 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4563/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] KevinWikant commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
KevinWikant commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1193994496
## RetryPolicies - RedundantModifier
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryPolicies.java:602: public OtherThanRemoteAndSaslExceptionDependentRetry(RetryPolicy defaultPolicy,:5: Redundant 'public' modifier. [RedundantModifier]
```
Correcting this checkstyle issue because I am touching this line anyway
## RetryPolicies - EmptyBlock
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/retry/RetryPolicies.java:614: if (e instanceof RemoteException || isSaslFailure(e)) {:61: Must have at least one statement. [EmptyBlock]
```
The code had the empty block before the change:
```
if (e instanceof RemoteException || isSaslFailure(e)) {
// do nothing
} else {
```
Will refactor this because required change is minimal
## SaslRpcClient - Indentation
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:240: final String serverPrincipal;: 'block' child has incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:241: try {: 'try' has incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:242: serverPrincipal = getServerPrincipal(authType);: 'try' child has incorrect indentation level 10, expected level should be 8. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:243: } catch (IllegalArgumentException ex) {: 'try rcurly' has incorrect indentation level 8, expected level should be 6. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:246: throw new SaslException("Bad Kerberos server principal configuration", ex);: 'catch' child has incorrect indentation level 10, expected level should be 8. [Indentation]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java:247: }: 'catch rcurly' has incorrect indentation level 8, expected level should be 6. [Indentation]
```
This is what the code looks like after applying this checkstyle rule:
```
case KERBEROS: {
if (ugi.getRealAuthenticationMethod().getAuthMethod() !=
AuthMethod.KERBEROS) {
LOG.debug("client isn't using kerberos");
return null;
}
final String serverPrincipal;
try {
serverPrincipal = getServerPrincipal(authType);
} catch (IllegalArgumentException ex) {
// YARN-11210: getServerPrincipal can throw IllegalArgumentException if Kerberos
// configuration is bad, this is surfaced as a non-retryable SaslException
throw new SaslException("Bad Kerberos server principal configuration", ex);
}
if (serverPrincipal == null) {
LOG.debug("protocol doesn't use kerberos");
return null;
}
```
Not sure why its recommending this indentation level, but it doesn't align with the way this code is currently structured. Going to recommend leaving this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1194107879
LGTM +1(non-binding)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] PrabhuJoseph commented on pull request #4563: YARN-11210. Fix YARN RMAdminCLI retry logic for non-retryable kerbero…
Posted by GitBox <gi...@apache.org>.
PrabhuJoseph commented on PR #4563:
URL: https://github.com/apache/hadoop/pull/4563#issuecomment-1193974096
Thanks @KevinWikant for the patch. LGTM, will push it once the checkstyle issues are addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org