You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "virajjasani (via GitHub)" <gi...@apache.org> on 2023/09/11 23:45:16 UTC
[GitHub] [hbase] virajjasani opened a new pull request, #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
virajjasani opened a new pull request, #5391:
URL: https://github.com/apache/hbase/pull/5391
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1722367839
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 36s | master passed |
| +1 :green_heart: | compile | 1m 35s | master passed |
| +1 :green_heart: | shadedjars | 5m 50s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 57s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 29s | the patch passed |
| +1 :green_heart: | compile | 1m 27s | the patch passed |
| +1 :green_heart: | javac | 1m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 41s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 1s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 10s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 2m 8s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 225m 28s | hbase-server in the patch passed. |
| | | 260m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 539f37ce1d9e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a80b341b6e |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/testReport/ |
| Max. process+thread count | 4657 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719578931
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 12s | master passed |
| +1 :green_heart: | compile | 1m 4s | master passed |
| +1 :green_heart: | shadedjars | 4m 53s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | the patch passed |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. |
| -1 :x: | unit | 219m 36s | hbase-server in the patch failed. |
| | | 245m 54s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a736ce46156f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/testReport/ |
| Max. process+thread count | 4624 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323315239
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +307,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e)) {
Review Comment:
sounds good @Apache9, addressed in the latest revision
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719163063
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 18s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 50s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. |
| -1 :x: | unit | 220m 29s | hbase-server in the patch failed. |
| | | 247m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5548c6912d1d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/testReport/ |
| Max. process+thread count | 4609 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326663603
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
Use a for loop here? And why only test 4 levels?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
Review Comment:
Better move this block of comments to the if condition in the caller method? I mean the section start from 'Notice that, it is safe blabla'. The numberOfAttemptsSoFar == 0 test is there.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
Review Comment:
Do we really need to test the log message here? Is it possible that we get a SaslException but the rpc connection is still alive?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
Review Comment:
Better also add a comment here to mention that, a sasl error means that we can not set up the rpc connection successfully, so we can make sure that the region server has not received our request.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
+ Throwable cause = e.getCause();
+ if (cause == null) {
+ return false;
+ }
+ if (isSaslError(cause)) {
+ return true;
+ }
+ cause = cause.getCause();
+ if (cause == null) {
+ return false;
+ }
+ if (isSaslError(cause)) {
+ return true;
+ }
+ cause = cause.getCause();
+ if (cause == null) {
+ return false;
+ }
+ if (isSaslError(cause)) {
+ return true;
+ }
+ cause = cause.getCause();
+ if (cause == null) {
+ return false;
+ }
+ return isSaslError(cause);
+ }
+
+ private boolean isSaslError(Throwable cause) {
Review Comment:
Please do not use the same method name here, as IOException is also a Throwable, although this is valid in Java, but it will confuse the developers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1322634863
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +307,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e)) {
Review Comment:
I think here we need to follow the same pattern with CallQueueTooBigException, only if this is the first try, we can make sure that the task has not been sent yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326848730
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
I think we could just use a loop to get the cause until cause is null, to check all the exceptions on chain. And we also need to handle RemoteException specially, to unwrap it instead of just calling getCause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1722328824
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 35s | master passed |
| +1 :green_heart: | compile | 3m 38s | master passed |
| +1 :green_heart: | checkstyle | 1m 6s | master passed |
| +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 47s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 38s | the patch passed |
| +1 :green_heart: | compile | 3m 38s | the patch passed |
| +1 :green_heart: | javac | 3m 38s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 7s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 8s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| | | 38m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 576b04507580 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a80b341b6e |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1327008715
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
Yes, it is in the loop
```
while (true) {
cause = cause.getCause();
if (cause == null) {
return false;
}
if (isThrowableOfTypeSasl(cause)) {
return true;
}
}
```
`isThrowableOfTypeSasl` does the unwrap and checks for type of exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716418580
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 23s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 31s | master passed |
| +1 :green_heart: | compile | 0m 36s | master passed |
| +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 15s | the patch passed |
| +1 :green_heart: | compile | 0m 35s | the patch passed |
| +1 :green_heart: | javac | 0m 35s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 218m 43s | hbase-server in the patch passed. |
| | | 239m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 902d08a84050 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 02c63cdec2 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/testReport/ |
| Max. process+thread count | 4466 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716799818
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 25s | master passed |
| +1 :green_heart: | compile | 1m 39s | master passed |
| +1 :green_heart: | shadedjars | 5m 30s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 1s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 13s | the patch passed |
| +1 :green_heart: | compile | 1m 40s | the patch passed |
| +1 :green_heart: | javac | 1m 40s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 34s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 1s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 4s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 50s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 237m 59s | hbase-server in the patch passed. |
| | | 272m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 385a8905f657 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bac3198e59 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/testReport/ |
| Max. process+thread count | 4240 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718473049
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 25s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 10s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 47s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 52s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 0s | hbase-server in the patch passed. |
| | | 244m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e38bec87317b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/testReport/ |
| Max. process+thread count | 4638 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1723867619
Thanks for the reviews! Let me merge and backport this today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1721044083
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 33s | master passed |
| +1 :green_heart: | compile | 1m 26s | master passed |
| +1 :green_heart: | shadedjars | 5m 38s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 40s | the patch passed |
| +1 :green_heart: | compile | 1m 22s | the patch passed |
| +1 :green_heart: | javac | 1m 22s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 50s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 57s | hbase-server in the patch passed. |
| | | 247m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 16276c2c9d43 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/testReport/ |
| Max. process+thread count | 4466 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719653331
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 21s | master passed |
| +1 :green_heart: | compile | 1m 56s | master passed |
| +1 :green_heart: | shadedjars | 5m 59s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 57s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 32s | the patch passed |
| +1 :green_heart: | javac | 1m 32s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 39s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 51s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 250m 54s | hbase-server in the patch passed. |
| | | 285m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 95b16f51651b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/testReport/ |
| Max. process+thread count | 4657 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326836964
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
Review Comment:
let me add javadoc instead and make it look generic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1714925029
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 4s | master passed |
| +1 :green_heart: | compile | 0m 46s | master passed |
| +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 45s | the patch passed |
| +1 :green_heart: | compile | 0m 45s | the patch passed |
| +1 :green_heart: | javac | 0m 45s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 215m 45s | hbase-server in the patch passed. |
| | | 238m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e16f7e91a50e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 97d512be7c |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/testReport/ |
| Max. process+thread count | 4693 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323678057
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
Duo's point is that, if some attempts were already successful, it might have triggered region transition already and we might be in the middle of another sub-procedure when we encounter this.
I also think num of attempts should not matter as we won't be able to make any progress anyways, but then it takes our discussion back to the parent Jira HBASE-28048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323864706
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
@virajjasani Better to introduce a method may be called
```
boolean hasNotReachedRegionServerYet(IOException e)
```
In this method we could test for both sasl error and call queue too big, and also other exception types in the future. So we do not need to add extra condition every time when we want to add new exception type tests in scheduleForRetry method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1324992854
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
not sure what is the best place to keep this, anywhere else in hbase-common would also work
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325395461
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
ok, this is now taken care of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719245986
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 34s | master passed |
| +1 :green_heart: | compile | 3m 37s | master passed |
| +1 :green_heart: | checkstyle | 1m 8s | master passed |
| +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 43s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 35s | the patch passed |
| +1 :green_heart: | compile | 3m 38s | the patch passed |
| +1 :green_heart: | javac | 3m 38s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 13s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| | | 38m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux e6ed2d77e7c9 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716420087
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 8s | master passed |
| +1 :green_heart: | compile | 0m 47s | master passed |
| +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 47s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed |
| +1 :green_heart: | javac | 0m 46s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 217m 16s | hbase-server in the patch passed. |
| | | 240m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 6e136eef94e6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 02c63cdec2 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/testReport/ |
| Max. process+thread count | 4674 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323674395
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
NettyRpcConnection is package private, hence can't be accessed from hbase-server
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326837861
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
Review Comment:
yes, we have come across this as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716783464
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 30s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 4m 56s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 48s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 54s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 20s | hbase-server in the patch passed. |
| | | 245m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 52c49b02ac5a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bac3198e59 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/testReport/ |
| Max. process+thread count | 4620 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325295375
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServerInFirstAttempt(IOException e) {
Review Comment:
I mean we could extract a method for testing exception only, and then we test numberOfAttemptsSoFar outside the method...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323672892
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
Does it matter how many attempts we have had so far if now we are getting a SASL error?
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
This is weird, please don't put these kinds of constants in HConstants. There are too many unrelated concerns there already.
Public static string constant in some other file, even this one, is preferred.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718850992
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 38s | master passed |
| +1 :green_heart: | compile | 3m 37s | master passed |
| +1 :green_heart: | checkstyle | 1m 6s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 35s | the patch passed |
| +1 :green_heart: | compile | 3m 39s | the patch passed |
| +1 :green_heart: | javac | 3m 39s | the patch passed |
| -0 :warning: | checkstyle | 0m 15s | hbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 9s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| | | 38m 31s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 28a19edbaab4 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1328010092
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +330,56 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ /**
+ * The category of exceptions where we can ensure that the request has not yet been received
+ * and/or processed by the target regionserver yet and hence we can determine whether it is safe
+ * to choose different regionserver as the target.
+ * @param e IOException thrown by the underlying rpc framework.
+ * @return true if the exception belongs to the category where the regionserver has not yet
+ * received the request yet.
+ */
+ private boolean unableToConnectToServer(IOException e) {
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ Throwable cause = e;
+ while (true) {
+ cause = cause.getCause();
+ if (cause == null) {
+ return false;
+ }
+ if (isThrowableOfTypeSasl(cause)) {
+ return true;
+ }
+ }
+ }
+
+ private boolean isThrowableOfTypeSasl(Throwable cause) {
+ if (cause instanceof IOException) {
+ IOException unwrappedException = unwrapException((IOException) cause);
+ return unwrappedException instanceof SaslException
+ || (unwrappedException.getMessage() != null && unwrappedException.getMessage()
+ .contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS));
+ }
+ return false;
+ }
Review Comment:
much better, let me update the PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1721431777
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 21s | master passed |
| +1 :green_heart: | compile | 1m 6s | master passed |
| +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 31s | hbase-server in the patch passed. |
| | | 245m 3s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 43dc168f39d4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ff2c10c1c2 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/testReport/ |
| Max. process+thread count | 4646 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326837575
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
it's based on the examples we have seen so far, e.g.
`procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed due to java.io.IOException: Call to address=rs1:61020 failed on local exception: java.io.IOException: org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS initiate failed, try=0, retrying...
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1721079638
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 13s | master passed |
| +1 :green_heart: | compile | 1m 18s | master passed |
| +1 :green_heart: | shadedjars | 5m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 44s | the patch passed |
| +1 :green_heart: | compile | 1m 18s | the patch passed |
| +1 :green_heart: | javac | 1m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 15s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 33s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 247m 9s | hbase-server in the patch passed. |
| | | 277m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 67ec010e69e0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/testReport/ |
| Max. process+thread count | 4615 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] d-c-manning commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "d-c-manning (via GitHub)" <gi...@apache.org>.
d-c-manning commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1327886285
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +330,56 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ /**
+ * The category of exceptions where we can ensure that the request has not yet been received
+ * and/or processed by the target regionserver yet and hence we can determine whether it is safe
+ * to choose different regionserver as the target.
+ * @param e IOException thrown by the underlying rpc framework.
+ * @return true if the exception belongs to the category where the regionserver has not yet
+ * received the request yet.
+ */
+ private boolean unableToConnectToServer(IOException e) {
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ Throwable cause = e;
+ while (true) {
+ cause = cause.getCause();
+ if (cause == null) {
+ return false;
+ }
+ if (isThrowableOfTypeSasl(cause)) {
+ return true;
+ }
+ }
+ }
+
+ private boolean isThrowableOfTypeSasl(Throwable cause) {
+ if (cause instanceof IOException) {
+ IOException unwrappedException = unwrapException((IOException) cause);
+ return unwrappedException instanceof SaslException
+ || (unwrappedException.getMessage() != null && unwrappedException.getMessage()
+ .contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS));
+ }
+ return false;
+ }
Review Comment:
Can the unwrapping be simplified, since `unwrapException` won't do anything unless it's a `RemoteException`?
```suggestion
private boolean isSaslError(IOException e) {
Throwable cause = e;
while (true) {
if (cause instanceof IOException) {
IOException unwrappedCause = unwrapException((IOException) cause);
if (
unwrappedCause instanceof SaslException || (unwrappedCause.getMessage() != null
&& unwrappedCause.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
) {
return true;
}
}
cause = cause.getCause();
if (cause == null) {
return false;
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1740326368
Thank you for the reviews @Apache9 @mnpoonia @d-c-manning @apurtell !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718274909
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 39s | master passed |
| +1 :green_heart: | compile | 3m 39s | master passed |
| +1 :green_heart: | checkstyle | 1m 6s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 47s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 38s | the patch passed |
| +1 :green_heart: | compile | 3m 38s | the patch passed |
| +1 :green_heart: | javac | 3m 38s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 7s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 5s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 13s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| | | 38m 46s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux bea567ee7318 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326866700
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
> handle RemoteException specially, to unwrap it instead of just calling getCause
yes, that is taken care of:
```
private boolean isThrowableOfTypeSasl(Throwable cause) {
if (cause instanceof IOException) {
IOException unwrappedException = unwrapException((IOException) cause);
return unwrappedException instanceof SaslException
|| (unwrappedException.getMessage() != null && unwrappedException.getMessage()
.contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS));
}
return false;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1714804454
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | master passed |
| +1 :green_heart: | compile | 4m 2s | master passed |
| +1 :green_heart: | checkstyle | 1m 0s | master passed |
| +1 :green_heart: | spotless | 1m 11s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 11s | the patch passed |
| +1 :green_heart: | compile | 4m 7s | the patch passed |
| +1 :green_heart: | javac | 4m 7s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 59s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 13m 26s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| -1 :x: | spotless | 0m 54s | patch has 24 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 2m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 48m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux efd08c853e36 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 97d512be7c |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 80 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1714926692
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 33s | master passed |
| +1 :green_heart: | compile | 0m 36s | master passed |
| +1 :green_heart: | shadedjars | 4m 55s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 14s | the patch passed |
| +1 :green_heart: | compile | 0m 35s | the patch passed |
| +1 :green_heart: | javac | 0m 35s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 220m 23s | hbase-server in the patch failed. |
| | | 241m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux cec8f50715ed 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 97d512be7c |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/testReport/ |
| Max. process+thread count | 4624 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1720775087
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 41s | master passed |
| +1 :green_heart: | compile | 3m 36s | master passed |
| +1 :green_heart: | checkstyle | 1m 5s | master passed |
| +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 40s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 39s | the patch passed |
| +1 :green_heart: | compile | 3m 41s | the patch passed |
| +1 :green_heart: | javac | 3m 41s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 6s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 13s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
| | | 38m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux df89016b6fd4 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/7/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719189028
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 54s | master passed |
| +1 :green_heart: | compile | 1m 19s | master passed |
| +1 :green_heart: | shadedjars | 5m 9s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 44s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 14s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 15s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 234m 48s | hbase-server in the patch passed. |
| | | 264m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 971044ec98cb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/testReport/ |
| Max. process+thread count | 4670 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323674395
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
initially wanted to keep in this class but since it belongs to hbase-client, hbase-server RSProcedureDispatcher can't access it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326840580
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
Review Comment:
done, added javadoc and made the comments more generic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326875214
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
+ ) {
+ return true;
+ }
+ // check 4 level of cause
Review Comment:
I mean after unwraping, you still need to go back to the get cause loop, not only test one time...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326847669
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
Review Comment:
OK, I misunderstood the logic here. So here even if it is not a SaslException but the log message is 'relogin in progress', we will also consider it as a SaslExcception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1721500608
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 24s | master passed |
| +1 :green_heart: | compile | 1m 41s | master passed |
| +1 :green_heart: | shadedjars | 6m 1s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 34s | the patch passed |
| +1 :green_heart: | compile | 1m 41s | the patch passed |
| +1 :green_heart: | javac | 1m 41s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 3s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 53s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 45s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 57s | hbase-client in the patch passed. |
| -1 :x: | unit | 255m 12s | hbase-server in the patch failed. |
| | | 290m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux bf955f47f4ee 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ff2c10c1c2 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/testReport/ |
| Max. process+thread count | 4740 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1721144416
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 2s | master passed |
| +1 :green_heart: | compile | 4m 53s | master passed |
| +1 :green_heart: | checkstyle | 1m 31s | master passed |
| +1 :green_heart: | spotless | 0m 52s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 22s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 59s | the patch passed |
| +1 :green_heart: | compile | 4m 35s | the patch passed |
| +1 :green_heart: | javac | 4m 35s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 17s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 40s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 45s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 47m 46s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux a113162005ec 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ff2c10c1c2 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 77 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/8/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325387382
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
i understand but i am not sure what is the best place to keep this in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1323863142
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
Yes, exactly as @virajjasani said, we need to make sure that the remote procedure has not been sent to the rs then we are safe to quit and choose another rs, otherwise the only safe way is to rely on SCP to tell us the rs is dead so we are safe to quit here.
So if we hit another error, like connection timed out the first time, then we are not sure whether we have already send the procedure to the rs, then no matter what the exceptions are in the following retries, we are not safe to quit.
In the real world, if authentication is not configured correctly, then it is likely we will get sasl error at the first try, so the code is enough to cover the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325293263
##########
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##########
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
Please do not put it in HConstants, it is IA.Public.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716165368
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | master passed |
| +1 :green_heart: | compile | 4m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 2s | master passed |
| +1 :green_heart: | spotless | 1m 12s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 41s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 2s | the patch passed |
| +1 :green_heart: | compile | 4m 4s | the patch passed |
| +1 :green_heart: | javac | 4m 4s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 0s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 14m 46s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 54s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 2m 6s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 48m 48s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 615942d13827 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 02c63cdec2 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/2/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1716486288
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 50s | master passed |
| +1 :green_heart: | compile | 3m 40s | master passed |
| +1 :green_heart: | checkstyle | 1m 7s | master passed |
| -1 :x: | spotless | 0m 44s | branch has 1 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 2m 47s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 38s | the patch passed |
| +1 :green_heart: | compile | 3m 38s | the patch passed |
| +1 :green_heart: | javac | 3m 38s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 4s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 15s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. |
| +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 3m 12s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. |
| | | 39m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 8d8d356b8b7b 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / bac3198e59 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/artifact/yetus-general-check/output/branch-spotless.txt |
| Max. process+thread count | 80 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/3/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718460178
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 25s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 6s | master passed |
| +1 :green_heart: | compile | 1m 21s | master passed |
| +1 :green_heart: | shadedjars | 5m 16s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 46s | the patch passed |
| +1 :green_heart: | compile | 1m 20s | the patch passed |
| +1 :green_heart: | javac | 1m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 17s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 16s | hbase-server in the patch passed. |
| | | 248m 2s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b87086d92b18 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 84ccae368f |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/testReport/ |
| Max. process+thread count | 4674 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1326838784
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##########
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+ private boolean unableToConnectToServer(IOException e) {
+ // This exception is thrown in the rpc framework, where we can make sure that the call has not
+ // been executed yet, so it is safe to mark it as fail. Especially for open a region, we'd
+ // better choose another region server.
+ // Notice that, it is safe to quit only if this is the first time we send request to region
+ // server. Maybe the region server has accepted our request the first time, and then there is
+ // a network error which prevents we receive the response, and the second time we hit a
+ // CallQueueTooBigException, obviously it is not safe to quit here, otherwise it may lead to a
+ // double assign...
+ if (e instanceof CallQueueTooBigException) {
+ LOG.warn("request to {} failed due to {}, try={}, this usually because"
+ + " server is overloaded, give up", serverName, e, numberOfAttemptsSoFar);
+ return true;
+ }
+ if (isSaslError(e)) {
+ LOG.warn("{} is not reachable; give up after first attempt", serverName, e);
+ return true;
+ }
+ return false;
+ }
+
+ private boolean isSaslError(IOException e) {
+ if (
+ e instanceof SaslException || (e.getMessage() != null
+ && e.getMessage().contains(RpcConnectionConstants.RELOGIN_IS_IN_PROGRESS))
Review Comment:
`procedure.RSProcedureDispatcher - request to rs-2,61020,1692796686773 failed due to java.io.IOException: Call to address=rs-2:61020 failed on local exception: java.io.IOException: Can not send request because relogin is in progress., try=0, retrying..`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #5391:
URL: https://github.com/apache/hbase/pull/5391#issuecomment-1722364415
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 18s | master passed |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 45s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 16s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 50s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 218m 17s | hbase-server in the patch passed. |
| | | 245m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/5391 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux be9d57817f4d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a80b341b6e |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/testReport/ |
| Max. process+thread count | 4591 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/9/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani merged pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani merged PR #5391:
URL: https://github.com/apache/hbase/pull/5391
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org