You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/07/29 14:37:19 UTC
[GitHub] [hadoop] ashutoshcipher opened a new pull request, #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
ashutoshcipher opened a new pull request, #4658:
URL: https://github.com/apache/hadoop/pull/4658
### Description of PR
Fix Bug while disabling proxy failover with Federation
When one disables the use of RM fail over proxy with federation, there is a bug checking a wrong/parent flag `yarn.federation.enabled` whether the federation is used instead of the fail over feature flag `yarn.federation.failover.enabled` of federation. Without this change, when fail over feature is disabled, node manager cannot be started.
JIRA - YARN-11237
### How was this patch tested?
Tested on EMR cluster and modified existing test.
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r935946681
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -98,8 +98,7 @@ protected static <T> T createRMProxy(final Configuration configuration,
YarnConfiguration conf = (configuration instanceof YarnConfiguration)
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
- RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ RetryPolicy retryPolicy = createRetryPolicy(conf, isFailoverEnabled(conf));
Review Comment:
@goiri - I think the test is covering the required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1208381720
@goiri - Thanks for review and approval. Can you commit/merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1201582890
@goiri - Can you please help with review? I am have addressed you comments in mu last commit. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1203292119
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 16m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 9s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 23s | | trunk passed |
| +1 :green_heart: | compile | 9m 55s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 8m 50s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 53s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 42s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 0s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 9m 3s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 3s | | the patch passed |
| +1 :green_heart: | compile | 8m 45s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 8m 45s | [/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/3/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 650 unchanged - 1 fixed = 652 total (was 651) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 2m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 26s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 23s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 58s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 32s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 28m 55s | | hadoop-yarn-client in the patch passed. |
| +1 :green_heart: | asflicense | 1m 35s | | The patch does not generate ASF License warnings. |
| | | 208m 1s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4658 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 619d4ecf842a 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 53b7f3013c16c4045f834388c9296ff6446df171 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/3/testReport/ |
| Max. process+thread count | 743 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1199433810
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r935936356
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -303,4 +302,17 @@ protected static RetryPolicy createRetryPolicy(Configuration conf,
return RetryPolicies.retryOtherThanRemoteAndSaslException(
RetryPolicies.TRY_ONCE_THEN_FAIL, exceptionToPolicyMap);
}
+
+ private static boolean isFailoverEnabled(YarnConfiguration conf) {
+ if (HAUtil.isHAEnabled(conf)) {
+ // Considering Resource Manager HA is enabled.
+ return true;
+ }
+ if (HAUtil.isFederationEnabled(conf) && HAUtil.isFederationFailoverEnabled(conf)) {
+ //Considering both federation and federation failover are enabled.
Review Comment:
Ping
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -98,8 +98,7 @@ protected static <T> T createRMProxy(final Configuration configuration,
YarnConfiguration conf = (configuration instanceof YarnConfiguration)
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
- RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ RetryPolicy retryPolicy = createRetryPolicy(conf, isFailoverEnabled(conf));
Review Comment:
Is that test covering everything we need?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r934980277
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -98,8 +98,7 @@ protected static <T> T createRMProxy(final Configuration configuration,
YarnConfiguration conf = (configuration instanceof YarnConfiguration)
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
- RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ RetryPolicy retryPolicy = createRetryPolicy(conf, isFailoverEnabled(conf));
Review Comment:
Can we add a test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r934980055
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -303,4 +302,17 @@ protected static RetryPolicy createRetryPolicy(Configuration conf,
return RetryPolicies.retryOtherThanRemoteAndSaslException(
RetryPolicies.TRY_ONCE_THEN_FAIL, exceptionToPolicyMap);
}
+
+ private static boolean isFailoverEnabled(YarnConfiguration conf) {
+ if (HAUtil.isHAEnabled(conf)) {
+ // Considering Resource Manager HA is enabled.
+ return true;
+ }
+ if (HAUtil.isFederationEnabled(conf) && HAUtil.isFederationFailoverEnabled(conf)) {
+ //Considering both federation and federation failover are enabled.
Review Comment:
Space missing before "Considering"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r935105525
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -98,8 +98,7 @@ protected static <T> T createRMProxy(final Configuration configuration,
YarnConfiguration conf = (configuration instanceof YarnConfiguration)
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
- RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ RetryPolicy retryPolicy = createRetryPolicy(conf, isFailoverEnabled(conf));
Review Comment:
@goiri - I have updated the test `testProxyProvider` as part this change. Are you recommending to add unit test case for `isFailoverEnabled` method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
goiri merged PR #4658:
URL: https://github.com/apache/hadoop/pull/4658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r935105330
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -303,4 +302,17 @@ protected static RetryPolicy createRetryPolicy(Configuration conf,
return RetryPolicies.retryOtherThanRemoteAndSaslException(
RetryPolicies.TRY_ONCE_THEN_FAIL, exceptionToPolicyMap);
}
+
+ private static boolean isFailoverEnabled(YarnConfiguration conf) {
+ if (HAUtil.isHAEnabled(conf)) {
+ // Considering Resource Manager HA is enabled.
+ return true;
+ }
+ if (HAUtil.isFederationEnabled(conf) && HAUtil.isFederationFailoverEnabled(conf)) {
+ //Considering both federation and federation failover are enabled.
Review Comment:
I will do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1199802642
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 7s | | trunk passed |
| +1 :green_heart: | compile | 9m 57s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 10m 23s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 43s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 38s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 52s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 9m 3s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 3s | | the patch passed |
| +1 :green_heart: | compile | 8m 43s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 8m 43s | [/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 650 unchanged - 1 fixed = 652 total (was 651) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 24s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 22s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 59s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 55s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 31s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 29m 1s | | hadoop-yarn-client in the patch passed. |
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not generate ASF License warnings. |
| | | 193m 53s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4658 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 160c13accc5e 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f5885a6f0d740041de2e23545fe41fd5d3be5f49 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/1/testReport/ |
| Max. process+thread count | 760 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1199855207
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 55s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 25s | | trunk passed |
| +1 :green_heart: | compile | 10m 33s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 22s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 20s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 3s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 9m 58s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 58s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 9m 7s | [/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/2/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 650 unchanged - 1 fixed = 652 total (was 651) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 57s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 57s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 56s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 12s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 6s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 28m 10s | | hadoop-yarn-client in the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not generate ASF License warnings. |
| | | 195m 24s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4658 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5a58c5762cc4 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 826c3c1738ef3697477fbdc51d6ea34b35a2dac2 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/2/testReport/ |
| Max. process+thread count | 631 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4658/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r935953045
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -303,4 +302,17 @@ protected static RetryPolicy createRetryPolicy(Configuration conf,
return RetryPolicies.retryOtherThanRemoteAndSaslException(
RetryPolicies.TRY_ONCE_THEN_FAIL, exceptionToPolicyMap);
}
+
+ private static boolean isFailoverEnabled(YarnConfiguration conf) {
+ if (HAUtil.isHAEnabled(conf)) {
+ // Considering Resource Manager HA is enabled.
+ return true;
+ }
+ if (HAUtil.isFederationEnabled(conf) && HAUtil.isFederationFailoverEnabled(conf)) {
+ //Considering both federation and federation failover are enabled.
Review Comment:
Added in my last commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r933372728
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -99,7 +99,8 @@ protected static <T> T createRMProxy(final Configuration configuration,
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ (HAUtil.isHAEnabled(conf) || (HAUtil.isFederationEnabled(conf)
Review Comment:
This boolean is not easy to read.
Can we extract with some nice comment and reuse it?
Something like:
```
/**
* Explain the logic
*/
private static boolean isFailoveEnabled(conf) {
if (HAUtil.isHAEnabled(conf)) {
// Comment explaining why
return true;
}
if (HAUtil.isFederationEnabled(conf) && HAUtil.isFederationFailoverEnabled(conf)) {
// Comment explaining why
return true;
}
return false;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on code in PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#discussion_r933396020
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java:
##########
@@ -99,7 +99,8 @@ protected static <T> T createRMProxy(final Configuration configuration,
? (YarnConfiguration) configuration
: new YarnConfiguration(configuration);
RetryPolicy retryPolicy = createRetryPolicy(conf,
- (HAUtil.isHAEnabled(conf) || HAUtil.isFederationFailoverEnabled(conf)));
+ (HAUtil.isHAEnabled(conf) || (HAUtil.isFederationEnabled(conf)
Review Comment:
Thanks @goiri for reviewing the PR and sharing your comments. I have address your comments in my last commit - Can you please review it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher commented on pull request #4658: YARN-11237. Fix Bug while disabling proxy failover with Federation
Posted by GitBox <gi...@apache.org>.
ashutoshcipher commented on PR #4658:
URL: https://github.com/apache/hadoop/pull/4658#issuecomment-1203217395
Thanks @goiri for review. Thanks @slfan1989 for additional review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org