You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/03/09 22:52:52 UTC
[GitHub] [hbase] markrmiller opened a new pull request #1259: HBASE-23951:
Avoid high speed recursion trap in AsyncRequestFutureImpl.
markrmiller opened a new pull request #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259
While working on branch-2, I ran into an issue where a retryable error kept occurring and code in AsyncRequestFutureImpl would reduce the backoff wait to 0 and extremely rapidly eat up a of thread stack space with recursive retry calls. This little patch stops the backoff wait kill after 3 retries. Chosen kind of arbitrarily, perhaps 5 is the right number, but I find large retry counts tend to hide things and that has made me default to fairly conservative in all my arbitrary number picking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid
high speed recursion trap in AsyncRequestFutureImpl.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596828273
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 20s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 37s | branch-2 passed |
| +1 :green_heart: | compile | 0m 32s | branch-2 passed |
| +1 :green_heart: | shadedjars | 4m 42s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 30s | hbase-client in branch-2 failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 0s | the patch passed |
| +1 :green_heart: | compile | 0m 31s | the patch passed |
| +1 :green_heart: | javac | 0m 31s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 28s | hbase-client in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | hbase-client in the patch passed. |
| | | 32m 39s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1259 |
| JIRA Issue | HBASE-23951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 91020c7032ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / dacceba0c7 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/testReport/ |
| Max. process+thread count | 468 (vs. ulimit of 10000) |
| modules | C: hbase-client U: hbase-client |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid
high speed recursion trap in AsyncRequestFutureImpl.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596829421
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 36s | branch-2 passed |
| +1 :green_heart: | checkstyle | 0m 34s | branch-2 passed |
| +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 5s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 31s | hbase-client: The patch generated 0 new + 0 unchanged - 21 fixed = 0 total (was 21) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 15m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | findbugs | 1m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 37m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1259 |
| JIRA Issue | HBASE-23951 |
| Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 70f04b541d95 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / dacceba0c7 |
| Max. process+thread count | 93 (vs. ulimit of 10000) |
| modules | C: hbase-client U: hbase-client |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1259:
HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#discussion_r396662327
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
##########
@@ -747,9 +771,9 @@ private void resubmit(ServerName oldServer, List<Action> toReplay,
// It should be possible to have some heuristics to take the right decision. Short term,
// we go for one.
boolean retryImmediately = throwable instanceof RetryImmediatelyException;
- int nextAttemptNumber = retryImmediately ? numAttempt : numAttempt + 1;
+ int nextAttemptNumber = numAttempt + 1;
Review comment:
I read the JIRA. That helped. I think you need a comment here on why the arbitrary '3'.... even it is just referring back to the issue. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid
high speed recursion trap in AsyncRequestFutureImpl.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596826184
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ branch-2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 30s | branch-2 passed |
| +1 :green_heart: | compile | 0m 26s | branch-2 passed |
| +1 :green_heart: | shadedjars | 4m 20s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | branch-2 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 2s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 17s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 58s | hbase-client in the patch passed. |
| | | 24m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1259 |
| JIRA Issue | HBASE-23951 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a7f729cb3dd2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2 / dacceba0c7 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/testReport/ |
| Max. process+thread count | 623 (vs. ulimit of 10000) |
| modules | C: hbase-client U: hbase-client |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1259:
HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#discussion_r396661168
##########
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
##########
@@ -747,9 +771,9 @@ private void resubmit(ServerName oldServer, List<Action> toReplay,
// It should be possible to have some heuristics to take the right decision. Short term,
// we go for one.
boolean retryImmediately = throwable instanceof RetryImmediatelyException;
- int nextAttemptNumber = retryImmediately ? numAttempt : numAttempt + 1;
+ int nextAttemptNumber = numAttempt + 1;
Review comment:
How does this change our behavior @markrmiller
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services