You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Umeshkumar9414 (via GitHub)" <gi...@apache.org> on 2025/11/28 12:50:54 UTC
[PR] @HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Umeshkumar9414 opened a new pull request, #7490:
URL: https://github.com/apache/hbase/pull/7490
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell merged PR #7490:
URL: https://github.com/apache/hbase/pull/7490
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "d-c-manning (via GitHub)" <gi...@apache.org>.
d-c-manning commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2575758014
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRWQueueRpcExecutor.java:
##########
@@ -54,15 +55,16 @@ public class TestRWQueueRpcExecutor {
public void setUp() {
conf = HBaseConfiguration.create();
conf.setFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY, 1.0f);
+ conf.setInt(IPC_SERVER_MAX_CALLQUEUE_LENGTH, 100);
conf.setFloat(CALL_QUEUE_SCAN_SHARE_CONF_KEY, 0.5f);
conf.setFloat(CALL_QUEUE_READ_SHARE_CONF_KEY, 0.5f);
}
@Test
public void itProvidesCorrectQueuesToBalancers() throws InterruptedException {
PriorityFunction qosFunction = mock(PriorityFunction.class);
- RWQueueRpcExecutor executor =
- new RWQueueRpcExecutor(testName.getMethodName(), 100, 100, qosFunction, conf, null);
+ RWQueueRpcExecutor executor = new RWQueueRpcExecutor(testName.getMethodName(), 100,
Review Comment:
Could we have a test that validates the queue length limits? Could we validate `RpcExecutor#currentQueueLimit` directly? Or some other validation that tries to fill the queue, but that may require more work and complexity.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java:
##########
@@ -37,16 +37,16 @@ public class BalancedQueueRpcExecutor extends RpcExecutor {
private final QueueBalancer balancer;
public BalancedQueueRpcExecutor(final String name, final int handlerCount,
- final int maxQueueLength, final PriorityFunction priority, final Configuration conf,
+ final String maxQueueLengthConfKey, final PriorityFunction priority, final Configuration conf,
Review Comment:
can we make the logic change without changing the API signature which is used by `HBaseInterfaceAudience.COPROC` and `PHOENIX`?
What if we pass `-1` for `maxQueueLength`, and if it is found to be `-1`, then determine it using the calculation you have added in this PR?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -225,7 +229,9 @@ public Map<String, Long> getCallQueueSizeSummary() {
protected void initializeQueues(final int numQueues) {
if (queueInitArgs.length > 0) {
currentQueueLimit = (int) queueInitArgs[0];
- queueInitArgs[0] = Math.max((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
+ queueInitArgs[0] = Math.min((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
Review Comment:
This doesn't seem like a safe change, considering people may be choosing larger queue sizes than 250 today. But this change would enforce no queue size larger than 250.
The pre-existing logic confuses me anyway. It talks about hard limits and soft limits, and suggests that we will initialize queues with hard limits (for initial capacity concerns?) but then enforce soft limits. It was added with [HBASE-15306](https://issues.apache.org/jira/browse/HBASE-15306) which talks about dynamic reconfiguration of queues.
But it seems that with RWQueueRpcExecutor, when we call `initializeQueues` three times, we would be setting the "soft" (current) limit to be this "hard" `250` limit, even if it was greater than the provided/calculated `maxQueueLength`. Probably the code does not expect the case of calling `initializeQueues` multiple times.
So perhaps the logic does need to change here, but I don't think we can simply change to `Math.min` either.
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -225,7 +229,9 @@ public Map<String, Long> getCallQueueSizeSummary() {
protected void initializeQueues(final int numQueues) {
if (queueInitArgs.length > 0) {
currentQueueLimit = (int) queueInitArgs[0];
- queueInitArgs[0] = Math.max((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
+ queueInitArgs[0] = Math.min((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
+ // queue should neven be initialised with 0 or less length
Review Comment:
nit typo
```suggestion
// queue should never be initialised with 0 or less length
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2687841540
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -296,7 +309,7 @@ protected void startHandlers(final String nameSuffix, final int numHandlers,
*/
private static final QueueBalancer ONE_QUEUE = val -> 0;
- public static QueueBalancer getBalancer(final String executorName, final Configuration conf,
+ protected static QueueBalancer getBalancer(final String executorName, final Configuration conf,
Review Comment:
This method creates the balancer for RPC queues, so I don't think it should be visible outside of the protected visibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3736100607
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 14s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 16s | | master passed |
| +1 :green_heart: | compile | 1m 0s | | master passed |
| +1 :green_heart: | javadoc | 0m 29s | | master passed |
| +1 :green_heart: | shadedjars | 6m 42s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 56s | | the patch passed |
| +1 :green_heart: | compile | 1m 0s | | the patch passed |
| +1 :green_heart: | javac | 1m 0s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 36s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 251m 10s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 278m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux da16e0f27314 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a826f1e736c0d13a14b8c013aaa2402c1eea67d5 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/testReport/ |
| Max. process+thread count | 4149 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3779772814
In order to get clean cherry picks back to branch-2 based code, https://github.com/apache/hbase/pull/7620 should be merged first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "apurtell (via GitHub)" <gi...@apache.org>.
apurtell commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2687754994
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -222,6 +230,11 @@ public Map<String, Long> getCallQueueSizeSummary() {
.collect(Collectors.groupingBy(Pair::getFirst, Collectors.summingLong(Pair::getSecond)));
}
+ // IMPORTANT: Call this method only ONCE per executor instance.
Review Comment:
Let's do this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3747155709
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 13s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 51s | | master passed |
| +1 :green_heart: | compile | 1m 0s | | master passed |
| +1 :green_heart: | javadoc | 0m 28s | | master passed |
| +1 :green_heart: | shadedjars | 6m 23s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 58s | | the patch passed |
| +1 :green_heart: | compile | 1m 2s | | the patch passed |
| +1 :green_heart: | javac | 1m 2s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 19s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 248m 17s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 274m 35s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux fa24ffe25adf 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 394ad474e462ab61ca4c9b140db23c0bbe3e861a |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/testReport/ |
| Max. process+thread count | 4287 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2673796101
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java:
##########
@@ -37,16 +37,16 @@ public class BalancedQueueRpcExecutor extends RpcExecutor {
private final QueueBalancer balancer;
public BalancedQueueRpcExecutor(final String name, final int handlerCount,
- final int maxQueueLength, final PriorityFunction priority, final Configuration conf,
+ final String maxQueueLengthConfKey, final PriorityFunction priority, final Configuration conf,
Review Comment:
Used your idea of passing -1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3591788673
@Apache9 , @apurtell , @virajjasani need help in review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2673783416
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRWQueueRpcExecutor.java:
##########
@@ -54,15 +55,16 @@ public class TestRWQueueRpcExecutor {
public void setUp() {
conf = HBaseConfiguration.create();
conf.setFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY, 1.0f);
+ conf.setInt(IPC_SERVER_MAX_CALLQUEUE_LENGTH, 100);
conf.setFloat(CALL_QUEUE_SCAN_SHARE_CONF_KEY, 0.5f);
conf.setFloat(CALL_QUEUE_READ_SHARE_CONF_KEY, 0.5f);
}
@Test
public void itProvidesCorrectQueuesToBalancers() throws InterruptedException {
PriorityFunction qosFunction = mock(PriorityFunction.class);
- RWQueueRpcExecutor executor =
- new RWQueueRpcExecutor(testName.getMethodName(), 100, 100, qosFunction, conf, null);
+ RWQueueRpcExecutor executor = new RWQueueRpcExecutor(testName.getMethodName(), 100,
Review Comment:
added a test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3767855935
Both test - TestAcidGuaranteesWithBasicPolicy,TestAcidGuaranteesWithEagerPolicy, are working in local. Seems like flacky or ongoing issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3776512197
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 22s | | master passed |
| +1 :green_heart: | compile | 3m 26s | | master passed |
| +1 :green_heart: | checkstyle | 0m 59s | | master passed |
| +1 :green_heart: | spotbugs | 1m 41s | | master passed |
| +1 :green_heart: | spotless | 1m 4s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 59s | | the patch passed |
| +1 :green_heart: | compile | 3m 28s | | the patch passed |
| +1 :green_heart: | javac | 3m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | spotbugs | 1m 45s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 24s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | | The patch does not generate ASF License warnings. |
| | | 40m 8s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux e98b6ae26245 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d18bb6598dbef2ef0601a94551027f132c8f75c |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3726525633
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 20s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 38s | | master passed |
| +1 :green_heart: | compile | 0m 44s | | master passed |
| +1 :green_heart: | javadoc | 0m 22s | | master passed |
| +1 :green_heart: | shadedjars | 4m 30s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed |
| +1 :green_heart: | shadedjars | 4m 27s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 220m 26s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 241m 37s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux b1aed102a57d 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2835b51bd08f537ec8e5cf709c643aef9bb6fcea |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/testReport/ |
| Max. process+thread count | 6851 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3735294715
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 16s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 59s | | master passed |
| +1 :green_heart: | compile | 5m 8s | | master passed |
| +1 :green_heart: | checkstyle | 1m 2s | | master passed |
| +1 :green_heart: | spotbugs | 1m 42s | | master passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 58s | | the patch passed |
| +1 :green_heart: | compile | 5m 8s | | the patch passed |
| +1 :green_heart: | javac | 5m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | spotbugs | 1m 50s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 34s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 43m 0s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 472c3ae6b16a 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / a826f1e736c0d13a14b8c013aaa2402c1eea67d5 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/4/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3725772643
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 49s | | master passed |
| +1 :green_heart: | compile | 2m 42s | | master passed |
| +1 :green_heart: | checkstyle | 0m 47s | | master passed |
| +1 :green_heart: | spotbugs | 1m 17s | | master passed |
| +1 :green_heart: | spotless | 0m 41s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 17s | | the patch passed |
| +1 :green_heart: | compile | 2m 37s | | the patch passed |
| +1 :green_heart: | javac | 2m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 46s | | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | spotbugs | 1m 21s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 8m 39s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 35s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | | The patch does not generate ASF License warnings. |
| | | 31m 32s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 966f2d12f4e1 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2835b51bd08f537ec8e5cf709c643aef9bb6fcea |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2582330822
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java:
##########
@@ -37,16 +37,16 @@ public class BalancedQueueRpcExecutor extends RpcExecutor {
private final QueueBalancer balancer;
public BalancedQueueRpcExecutor(final String name, final int handlerCount,
- final int maxQueueLength, final PriorityFunction priority, final Configuration conf,
+ final String maxQueueLengthConfKey, final PriorityFunction priority, final Configuration conf,
Review Comment:
Yes, I agree changing API signature might not work here.
Let me try this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3746497652
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 16s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 1s | | master passed |
| +1 :green_heart: | compile | 5m 8s | | master passed |
| +1 :green_heart: | checkstyle | 1m 1s | | master passed |
| +1 :green_heart: | spotbugs | 1m 42s | | master passed |
| +1 :green_heart: | spotless | 0m 47s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 55s | | the patch passed |
| +1 :green_heart: | compile | 5m 7s | | the patch passed |
| +1 :green_heart: | javac | 5m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | spotbugs | 1m 51s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 32s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 46s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 42m 54s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 5901cb989c05 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 394ad474e462ab61ca4c9b140db23c0bbe3e861a |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/5/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "d-c-manning (via GitHub)" <gi...@apache.org>.
d-c-manning commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2686681302
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -222,6 +230,11 @@ public Map<String, Long> getCallQueueSizeSummary() {
.collect(Collectors.groupingBy(Pair::getFirst, Collectors.summingLong(Pair::getSecond)));
}
+ // IMPORTANT: Call this method only ONCE per executor instance.
Review Comment:
I wonder if we could just enforce this in the code? Only allow it to be called once, and if it is called a second time then throw an "already initialized" error?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -296,7 +309,7 @@ protected void startHandlers(final String nameSuffix, final int numHandlers,
*/
private static final QueueBalancer ONE_QUEUE = val -> 0;
- public static QueueBalancer getBalancer(final String executorName, final Configuration conf,
+ protected static QueueBalancer getBalancer(final String executorName, final Configuration conf,
Review Comment:
this does seem like a useful visibility change, but must it be included here? Changing a public method of a "public" class (LimitedPrivate for COPROC, PHOENIX) is a breaking change. That being said, it's hard to imagine how anyone is using it outside of hbase... and the return class is not LimitedPrivate for COPROC... and I don't see it used in Phoenix... so 🤷♂️
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java:
##########
@@ -67,15 +67,13 @@ public SimpleRpcScheduler(Configuration conf, int handlerCount, int priorityHand
Abortable server, int highPriorityLevel) {
int bulkLoadHandlerCount = conf.getInt(HConstants.REGION_SERVER_BULKLOAD_HANDLER_COUNT,
HConstants.DEFAULT_REGION_SERVER_BULKLOAD_HANDLER_COUNT);
- int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH,
- handlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
- int maxPriorityQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_PRIORITY_MAX_CALLQUEUE_LENGTH,
- priorityHandlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
+ int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH, -1);
Review Comment:
can we make the `-1` a constant too? something like `MAX_CALLQUEUE_LENGTH_UNDEFINED`?
```suggestion
int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH, MAX_CALLQUEUE_LENGTH_UNDEFINED);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3727465727
I tried both the tests (TestAcidGuaranteesWithBasicPolicy and TestAcidGuaranteesWithEagerPolicy) on my local and both are working fine.
@Apache9 , @d-c-manning , @apurtell can you please review this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3727610871
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 14s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 18s | | master passed |
| +1 :green_heart: | compile | 5m 9s | | master passed |
| +1 :green_heart: | checkstyle | 1m 2s | | master passed |
| +1 :green_heart: | spotbugs | 1m 42s | | master passed |
| +1 :green_heart: | spotless | 0m 52s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 55s | | the patch passed |
| +1 :green_heart: | compile | 5m 11s | | the patch passed |
| +1 :green_heart: | javac | 5m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 3s | | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) |
| +1 :green_heart: | spotbugs | 1m 50s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 39s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 47s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | | The patch does not generate ASF License warnings. |
| | | 43m 42s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux aeb54909820d 6.14.0-1018-aws #18~24.04.1-Ubuntu SMP Mon Nov 24 19:46:27 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 82b8ce4d01b823c695bf0228a477770d30ee66fb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/3/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3777347713
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 40s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 29s | | master passed |
| +1 :green_heart: | compile | 0m 45s | | master passed |
| +1 :green_heart: | javadoc | 0m 22s | | master passed |
| +1 :green_heart: | shadedjars | 4m 16s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed |
| +1 :green_heart: | shadedjars | 4m 15s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 222m 27s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 243m 25s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 6f26402413b9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5d18bb6598dbef2ef0601a94551027f132c8f75c |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/testReport/ |
| Max. process+thread count | 5772 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/6/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3589411609
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 27s | | master passed |
| +1 :green_heart: | compile | 3m 22s | | master passed |
| +1 :green_heart: | checkstyle | 0m 59s | | master passed |
| +1 :green_heart: | spotbugs | 1m 36s | | master passed |
| +1 :green_heart: | spotless | 0m 49s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 53s | | the patch passed |
| +1 :green_heart: | compile | 3m 50s | | the patch passed |
| +1 :green_heart: | javac | 3m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 14s | | the patch passed |
| +1 :green_heart: | spotbugs | 2m 7s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 13m 4s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. |
| +1 :green_heart: | spotless | 0m 50s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | | The patch does not generate ASF License warnings. |
| | | 42m 37s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux c04b42d20785 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 86285d4f18353fb96d5ab946e8f1c56918cb5c42 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3782536476
> In order to get clean cherry picks back to branch-2 based code, #7620 should be merged first.
Sorry I made it confusing. typo in this msg https://github.com/apache/hbase/pull/7652 shoud be merged first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #7490:
URL: https://github.com/apache/hbase/pull/7490#issuecomment-3589984139
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | | master passed |
| +1 :green_heart: | compile | 1m 24s | | master passed |
| +1 :green_heart: | javadoc | 0m 37s | | master passed |
| +1 :green_heart: | shadedjars | 8m 49s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed |
| +1 :green_heart: | shadedjars | 8m 36s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 218m 45s | | hbase-server in the patch passed. |
| | | 253m 49s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7490 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux d1ae4d1b2cd7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 86285d4f18353fb96d5ab946e8f1c56918cb5c42 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/1/testReport/ |
| Max. process+thread count | 4493 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7490/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2617376740
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -225,7 +229,9 @@ public Map<String, Long> getCallQueueSizeSummary() {
protected void initializeQueues(final int numQueues) {
if (queueInitArgs.length > 0) {
currentQueueLimit = (int) queueInitArgs[0];
- queueInitArgs[0] = Math.max((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
+ queueInitArgs[0] = Math.min((int) queueInitArgs[0], DEFAULT_CALL_QUEUE_SIZE_HARD_LIMIT);
Review Comment:
Than @d-c-manning it helped me understand hard limit and soft limit in a better way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Resolution for high default maxQueueLength for call queues [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2582330822
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java:
##########
@@ -37,16 +37,16 @@ public class BalancedQueueRpcExecutor extends RpcExecutor {
private final QueueBalancer balancer;
public BalancedQueueRpcExecutor(final String name, final int handlerCount,
- final int maxQueueLength, final PriorityFunction priority, final Configuration conf,
+ final String maxQueueLengthConfKey, final PriorityFunction priority, final Configuration conf,
Review Comment:
Yes, I agree changing API signature might not work here.
But somehow we have to pass the config value or config key from outside only. As this same constructor is being used for meta priority handler and normal handler.
I think adding an new constructor might be a way forword
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29141 Calculate default maxQueueLength call queues correctly [hbase]
Posted by "Umeshkumar9414 (via GitHub)" <gi...@apache.org>.
Umeshkumar9414 commented on code in PR #7490:
URL: https://github.com/apache/hbase/pull/7490#discussion_r2687869978
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java:
##########
@@ -222,6 +230,11 @@ public Map<String, Long> getCallQueueSizeSummary() {
.collect(Collectors.groupingBy(Pair::getFirst, Collectors.summingLong(Pair::getSecond)));
}
+ // IMPORTANT: Call this method only ONCE per executor instance.
Review Comment:
done
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java:
##########
@@ -67,15 +67,13 @@ public SimpleRpcScheduler(Configuration conf, int handlerCount, int priorityHand
Abortable server, int highPriorityLevel) {
int bulkLoadHandlerCount = conf.getInt(HConstants.REGION_SERVER_BULKLOAD_HANDLER_COUNT,
HConstants.DEFAULT_REGION_SERVER_BULKLOAD_HANDLER_COUNT);
- int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH,
- handlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
- int maxPriorityQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_PRIORITY_MAX_CALLQUEUE_LENGTH,
- priorityHandlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
+ int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH, -1);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org