You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/08/12 06:26:22 UTC
[GitHub] [hbase] infraio opened a new pull request #2247: HBASE-24847 Backport HBASE-23956: Use less resources running tests to…
infraio opened a new pull request #2247:
URL: https://github.com/apache/hbase/pull/2247
… branch-2.2
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
Rename the threads we create in here so they are NOT named same was
threads created by Hadoop RPC.
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
Allow configuring eventloopgroup thread count (so can override for
tests)
hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
Enable socket resuse.
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
Enable socket resuse and config for how many threads to use.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
Thread name edit; drop the redundant 'Thread' suffix.
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
Make closeable and shutdown executor when called.
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
Call close on HFileReplicator
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
HDFS creates lots of threads. Use less of it so less threads overall.
hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
Constrain resources when running in test context.
hbase-server/src/test/resources/log4j.properties
Enable debug on netty to see netty configs in our log
pom.xml
Add system properties when we launch JVMs to constrain thread counts in
tests
Addendum to cleanup pom warning.
Addendum: This test was depending on hard-coded number for
log cleaners; the parent patch made them less in test context.
Addendum: Two more tests that had issues because parent cut
down the number of default RegionServers launched.
Addendum: TestMultiParallel thread counts were off after
parent change.
Addendum: TestNamespaceCommands was using up the limit on
nioeventloopgroup.... so create them per run of client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2247: HBASE-24847 Backport HBASE-23956: Use less resources running tests to…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2247:
URL: https://github.com/apache/hbase/pull/2247#issuecomment-673501324
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 24s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 26 new or modified test files. |
||| _ branch-2.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 46s | branch-2.2 passed |
| +1 :green_heart: | compile | 2m 57s | branch-2.2 passed |
| +1 :green_heart: | checkstyle | 6m 17s | branch-2.2 passed |
| +1 :green_heart: | shadedjars | 4m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 6m 10s | branch-2.2 passed |
| +0 :ok: | spotbugs | 1m 18s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 21m 16s | branch-2.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 43s | the patch passed |
| +1 :green_heart: | compile | 2m 57s | the patch passed |
| +1 :green_heart: | javac | 2m 57s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 36s | hbase-client: The patch generated 0 new + 4 unchanged - 6 fixed = 4 total (was 10) |
| +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-procedure |
| +1 :green_heart: | checkstyle | 1m 24s | hbase-server: The patch generated 0 new + 279 unchanged - 8 fixed = 279 total (was 287) |
| +1 :green_heart: | checkstyle | 0m 20s | The patch passed checkstyle in hbase-mapreduce |
| +1 :green_heart: | checkstyle | 0m 38s | The patch passed checkstyle in hbase-thrift |
| +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-rsgroup |
| +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-shell |
| +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-endpoint |
| +1 :green_heart: | checkstyle | 0m 18s | The patch passed checkstyle in hbase-rest |
| +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-examples |
| +1 :green_heart: | checkstyle | 1m 55s | root: The patch generated 0 new + 283 unchanged - 14 fixed = 283 total (was 297) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 26s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 25m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. |
| +1 :green_heart: | javadoc | 6m 22s | the patch passed |
| +1 :green_heart: | findbugs | 24m 40s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 184m 29s | root in the patch failed. |
| +1 :green_heart: | asflicense | 5m 37s | The patch does not generate ASF License warnings. |
| | | 323m 19s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | TEST-classes |
| | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2247 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 6e6fff42bcdc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2247/out/precommit/personality/provided.sh |
| git revision | branch-2.2 / a838d727fa |
| Default Java | 1.8.0_181 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/4/artifact/out/patch-unit-root.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/4/testReport/ |
| Max. process+thread count | 3818 (vs. ulimit of 10000) |
| modules | C: hbase-client hbase-procedure hbase-server hbase-mapreduce hbase-thrift hbase-rsgroup hbase-shell hbase-endpoint hbase-rest hbase-examples . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/4/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2247: HBASE-24847 Backport HBASE-23956: Use less resources running tests to…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2247:
URL: https://github.com/apache/hbase/pull/2247#issuecomment-673720378
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 26 new or modified test files. |
||| _ branch-2.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 8s | branch-2.2 passed |
| +1 :green_heart: | compile | 2m 50s | branch-2.2 passed |
| +1 :green_heart: | checkstyle | 6m 11s | branch-2.2 passed |
| +1 :green_heart: | shadedjars | 4m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 6m 8s | branch-2.2 passed |
| +0 :ok: | spotbugs | 1m 22s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 21m 36s | branch-2.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 43s | the patch passed |
| +1 :green_heart: | compile | 2m 49s | the patch passed |
| +1 :green_heart: | javac | 2m 49s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 34s | hbase-client: The patch generated 0 new + 4 unchanged - 6 fixed = 4 total (was 10) |
| +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-procedure |
| +1 :green_heart: | checkstyle | 1m 24s | hbase-server: The patch generated 0 new + 279 unchanged - 8 fixed = 279 total (was 287) |
| +1 :green_heart: | checkstyle | 0m 20s | The patch passed checkstyle in hbase-mapreduce |
| +1 :green_heart: | checkstyle | 0m 39s | The patch passed checkstyle in hbase-thrift |
| +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-rsgroup |
| +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-shell |
| +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-endpoint |
| +1 :green_heart: | checkstyle | 0m 17s | The patch passed checkstyle in hbase-rest |
| +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-examples |
| +1 :green_heart: | checkstyle | 2m 1s | root: The patch generated 0 new + 283 unchanged - 14 fixed = 283 total (was 297) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 27s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 25m 8s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. |
| +1 :green_heart: | javadoc | 6m 21s | the patch passed |
| +1 :green_heart: | findbugs | 23m 0s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 178m 27s | root in the patch failed. |
| +1 :green_heart: | asflicense | 6m 16s | The patch does not generate ASF License warnings. |
| | | 313m 44s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | TEST-classes |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2247 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 47f0fbe44e4d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2247/out/precommit/personality/provided.sh |
| git revision | branch-2.2 / 058569c621 |
| Default Java | 1.8.0_181 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/5/artifact/out/patch-unit-root.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/5/testReport/ |
| Max. process+thread count | 3560 (vs. ulimit of 10000) |
| modules | C: hbase-client hbase-procedure hbase-server hbase-mapreduce hbase-thrift hbase-rsgroup hbase-shell hbase-endpoint hbase-rest hbase-examples . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/5/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] infraio merged pull request #2247: HBASE-24847 Backport HBASE-23956: Use less resources running tests to…
Posted by GitBox <gi...@apache.org>.
infraio merged pull request #2247:
URL: https://github.com/apache/hbase/pull/2247
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2247: HBASE-24847 Backport HBASE-23956: Use less resources running tests to…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2247:
URL: https://github.com/apache/hbase/pull/2247#issuecomment-673342498
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | patch | 0m 8s | https://github.com/apache/hbase/pull/2247 does not apply to branch-2.2. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hbase/pull/2247 |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2247/3/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org