You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/09/02 09:26:54 UTC
[GitHub] [hbase] cuibo01 opened a new pull request #2343: HBASE-24962 Optimize BufferNode Lock
cuibo01 opened a new pull request #2343:
URL: https://github.com/apache/hbase/pull/2343
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-686958716
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 43s | master passed |
| +1 :green_heart: | checkstyle | 0m 18s | master passed |
| +1 :green_heart: | spotbugs | 0m 34s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | the patch passed |
| -0 :warning: | checkstyle | 0m 14s | hbase-procedure: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 9s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 0m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 27m 53s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 3ec281a3a0a1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 19d0140562 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685589107
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 56s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 34s | master passed |
| +1 :green_heart: | compile | 0m 22s | master passed |
| +1 :green_heart: | shadedjars | 7m 35s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 19s | hbase-procedure in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 36s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 31s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 18s | hbase-procedure in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 50s | hbase-procedure in the patch passed. |
| | | 29m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 2b0a59f5629d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4dc08144f3 |
| Default Java | 2020-01-14 |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/testReport/ |
| Max. process+thread count | 229 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-686960154
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 50s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 48s | master passed |
| +1 :green_heart: | compile | 0m 21s | master passed |
| +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 18s | hbase-procedure in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 48s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 32s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 16s | hbase-procedure in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 2s | hbase-procedure in the patch passed. |
| | | 30m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux bb4647d90e65 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 19d0140562 |
| Default Java | 2020-01-14 |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/testReport/ |
| Max. process+thread count | 219 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685599756
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 14s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 31s | master passed |
| +1 :green_heart: | checkstyle | 0m 15s | master passed |
| +1 :green_heart: | spotbugs | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | the patch passed |
| -0 :warning: | checkstyle | 0m 15s | hbase-procedure: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 14m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 0m 49s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 36m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 9be20097b1a0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4dc08144f3 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-686957344
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 46s | master passed |
| +1 :green_heart: | compile | 0m 20s | master passed |
| +1 :green_heart: | shadedjars | 6m 34s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 45s | hbase-procedure in the patch passed. |
| | | 25m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7cd48fcbc9a3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 19d0140562 |
| Default Java | 1.8.0_232 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/testReport/ |
| Max. process+thread count | 373 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/3/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-1026585551
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 4s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 37s | master passed |
| +1 :green_heart: | compile | 0m 20s | master passed |
| +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 28s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 14s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 4s | hbase-procedure in the patch passed. |
| | | 32m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5b54a93b160a 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 351caa5bcc |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/testReport/ |
| Max. process+thread count | 247 (vs. ulimit of 30000) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685532581
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 52s | master passed |
| +1 :green_heart: | compile | 0m 25s | master passed |
| +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 18s | hbase-procedure in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 20s | the patch passed |
| +1 :green_heart: | javac | 0m 20s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 18s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 16s | hbase-procedure in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 51s | hbase-procedure in the patch passed. |
| | | 29m 30s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 53d18e4ac92d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04de594138 |
| Default Java | 2020-01-14 |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
| javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/testReport/ |
| Max. process+thread count | 218 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r482817164
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -298,6 +299,8 @@ default boolean storeInDispatchedQueue() {
// ============================================================================================
private final class TimeoutExecutorThread extends Thread {
private final DelayQueue<DelayedWithTimeout> queue = new DelayQueue<DelayedWithTimeout>();
+ private final ConcurrentHashMap<DelayedWithTimeout, DelayedWithTimeout> pendingBufferNode =
Review comment:
nit: please use ConcurrentMap
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -314,17 +317,26 @@ public void run() {
if (task instanceof DelayedTask) {
threadPool.execute(((DelayedTask) task).getObject());
} else {
+ pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
}
}
+ public void putIfAbsent(BufferNode bufferNode) {
Review comment:
nit: keep it private
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -370,39 +383,47 @@ public TRemote getKey() {
}
@Override
- public synchronized void add(final RemoteProcedure operation) {
- if (this.operations == null) {
- this.operations = new HashSet<>();
- setTimeout(EnvironmentEdgeManager.currentTime() + operationDelay);
- timeoutExecutor.add(this);
- }
+ public void add(final RemoteProcedure operation) {
this.operations.add(operation);
if (this.operations.size() > queueMaxSize) {
- timeoutExecutor.remove(this);
- dispatch();
+ synchronized (lock) {
+ if (this.operations.size() > queueMaxSize) {
+ timeoutExecutor.remove(this);
+ dispatch();
+ }
+ //all procedure have been scheduled by the current thread or another thread.
+ return;
+ }
}
+ timeoutExecutor.putIfAbsent(this);
}
@Override
- public synchronized void dispatch() {
- if (operations != null) {
- remoteDispatch(getKey(), operations);
- operations.stream().filter(operation -> operation.storeInDispatchedQueue())
- .forEach(operation -> dispatchedOperations.add(operation));
- this.operations = null;
+ public void dispatch() {
+ Set<RemoteProcedure> operationsTmp = operations;
+ operations = Sets.newConcurrentHashSet();
+ if (operationsTmp.isEmpty()) {
+ return;
}
+ remoteDispatch(getKey(), operations);
+ operations.stream().filter(operation -> operation.storeInDispatchedQueue())
+ .forEach(operation -> dispatchedOperations.add(operation));
Review comment:
This can be replaced with method references:
```
operations.stream().filter(RemoteProcedure::storeInDispatchedQueue)
.forEach(dispatchedOperations::add);
```
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -370,39 +383,47 @@ public TRemote getKey() {
}
@Override
- public synchronized void add(final RemoteProcedure operation) {
- if (this.operations == null) {
Review comment:
Are you sure we are matching this condition with `timeoutExecutor.putIfAbsent(this);`?
Based on `putIfAbsent()` method, it seems only first time when entry is added for new key in map i.e when concurrent map's thread-safe version of `putIfAbsent()` returns null, we execute this block but is there any scenario that might not adhere to this `if (this.operations == null) ` condition by any chance?
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -314,17 +317,26 @@ public void run() {
if (task instanceof DelayedTask) {
threadPool.execute(((DelayedTask) task).getObject());
} else {
+ pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
}
}
+ public void putIfAbsent(BufferNode bufferNode) {
+ if (pendingBufferNode.putIfAbsent(bufferNode, bufferNode) == null) {
Review comment:
If we want to add same object as key and value, why are we using `ConcurrentHashMap` instead of `Sets.newConcurrentHashSet()`?
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -357,8 +369,9 @@ public void awaitTermination() {
*/
protected final class BufferNode extends DelayedContainerWithTimestamp<TRemote>
implements RemoteNode<TEnv, TRemote> {
- private Set<RemoteProcedure> operations;
- private final Set<RemoteProcedure> dispatchedOperations = new HashSet<>();
+ private Set<RemoteProcedure> operations = Sets.newConcurrentHashSet();
Review comment:
`RemoteProcedure` is parameterized class and it is used in raw format. Although not for this PR, but we should really change this notion of using the class as parameterized in this entire class.
e.g
```
private Set<RemoteProcedure<TEnv, TRemote>> operations ...
public void add(final RemoteProcedure<TEnv, TRemote> operation) ...
protected abstract void remoteDispatch(TRemote key, Set<RemoteProcedure<TEnv, TRemote>> operations);
```
Nothing urgent, can be taken up in separate patch to well distinguish the commit purpose.
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -357,8 +369,9 @@ public void awaitTermination() {
*/
protected final class BufferNode extends DelayedContainerWithTimestamp<TRemote>
implements RemoteNode<TEnv, TRemote> {
- private Set<RemoteProcedure> operations;
- private final Set<RemoteProcedure> dispatchedOperations = new HashSet<>();
+ private Set<RemoteProcedure> operations = Sets.newConcurrentHashSet();
+ private final Set<RemoteProcedure> dispatchedOperations = Sets.newConcurrentHashSet();
+ private final Object lock = new Object();
Review comment:
While this looks promising but `add(RemoteProcedure operation)` and `abortOperationsInQueue()` will now contend with each other. Do we expect both to synchronize on the same object?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-1026594878
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 53s | master passed |
| +1 :green_heart: | compile | 0m 36s | master passed |
| +1 :green_heart: | checkstyle | 0m 18s | master passed |
| +1 :green_heart: | spotbugs | 0m 38s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 22s | the patch passed |
| +1 :green_heart: | compile | 0m 32s | the patch passed |
| +1 :green_heart: | javac | 0m 32s | the patch passed |
| -0 :warning: | checkstyle | 0m 14s | hbase-procedure: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 22m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 0m 57s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 47m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux bfcd021796c5 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 351caa5bcc |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483420844
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -370,39 +383,47 @@ public TRemote getKey() {
}
@Override
- public synchronized void add(final RemoteProcedure operation) {
- if (this.operations == null) {
Review comment:
TimeoutExecutorThread#run will remove task from pendingBufferNode， so after run， task will not be included in the pendingBufferNode
else {
pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-1026585302
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | master passed |
| +1 :green_heart: | compile | 0m 22s | master passed |
| +1 :green_heart: | shadedjars | 8m 33s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | the patch passed |
| +1 :green_heart: | compile | 0m 23s | the patch passed |
| +1 :green_heart: | javac | 0m 23s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 35s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 50s | hbase-procedure in the patch passed. |
| | | 31m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c07764b5e244 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 351caa5bcc |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 30000) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685539551
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 4s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | master passed |
| +1 :green_heart: | compile | 0m 23s | master passed |
| +1 :green_heart: | shadedjars | 8m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 56s | the patch passed |
| +1 :green_heart: | compile | 0m 24s | the patch passed |
| +1 :green_heart: | javac | 0m 24s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 25s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 29s | hbase-procedure in the patch passed. |
| | | 34m 20s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 390d7659b2c5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04de594138 |
| Default Java | 1.8.0_232 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/testReport/ |
| Max. process+thread count | 250 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483414511
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -298,6 +299,8 @@ default boolean storeInDispatchedQueue() {
// ============================================================================================
private final class TimeoutExecutorThread extends Thread {
private final DelayQueue<DelayedWithTimeout> queue = new DelayQueue<DelayedWithTimeout>();
+ private final ConcurrentHashMap<DelayedWithTimeout, DelayedWithTimeout> pendingBufferNode =
Review comment:
?why use concurrentMap
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685594036
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 4s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | master passed |
| +1 :green_heart: | compile | 0m 22s | master passed |
| +1 :green_heart: | shadedjars | 8m 47s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 42s | the patch passed |
| +1 :green_heart: | compile | 0m 22s | the patch passed |
| +1 :green_heart: | javac | 0m 22s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 31s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 10s | hbase-procedure in the patch passed. |
| | | 32m 54s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 0d6642aadeb3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4dc08144f3 |
| Default Java | 1.8.0_232 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/testReport/ |
| Max. process+thread count | 245 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/2/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483418381
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -314,17 +317,26 @@ public void run() {
if (task instanceof DelayedTask) {
threadPool.execute(((DelayedTask) task).getObject());
} else {
+ pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
}
}
+ public void putIfAbsent(BufferNode bufferNode) {
+ if (pendingBufferNode.putIfAbsent(bufferNode, bufferNode) == null) {
Review comment:
use atomically putIfAbsent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#issuecomment-685545934
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 45s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 19s | master passed |
| +1 :green_heart: | checkstyle | 0m 17s | master passed |
| +1 :green_heart: | spotbugs | 0m 41s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 38s | the patch passed |
| -0 :warning: | checkstyle | 0m 17s | hbase-procedure: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 15m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 38m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2343 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 52763e6e3a5a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04de594138 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt |
| Max. process+thread count | 84 (vs. ulimit of 12500) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2343/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483794886
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -298,6 +299,8 @@ default boolean storeInDispatchedQueue() {
// ============================================================================================
private final class TimeoutExecutorThread extends Thread {
private final DelayQueue<DelayedWithTimeout> queue = new DelayQueue<DelayedWithTimeout>();
+ private final ConcurrentHashMap<DelayedWithTimeout, DelayedWithTimeout> pendingBufferNode =
Review comment:
As a general rule, it's good to handle implementation by using interface to avoid the pain of changing all references if we wish to change the implementation. I understand, we don't want to change implementation here, but it's always great to refer to implementation using interface unless we want to deliberately use implementor methods which are not overridden from interface.
Hence, just like why we would want to use `Map<Key,Val> map=new HashMap<>`, let's also use:
```
private final ConcurrentMap<DelayedWithTimeout, DelayedWithTimeout> pendingBufferNode = new ConcurrentHashMap<>();
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483417256
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -314,17 +317,26 @@ public void run() {
if (task instanceof DelayedTask) {
threadPool.execute(((DelayedTask) task).getObject());
} else {
+ pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
}
}
+ public void putIfAbsent(BufferNode bufferNode) {
Review comment:
ok thx @virajjasani
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483420844
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -370,39 +383,47 @@ public TRemote getKey() {
}
@Override
- public synchronized void add(final RemoteProcedure operation) {
- if (this.operations == null) {
Review comment:
TimeoutExecutorThread#run will remove task from pendingBufferNode， so after run， task will not be included in the pendingBufferNode
else {
pendingBufferNode.remove(task);
((BufferNode) task).dispatch();
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483418959
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -357,8 +369,9 @@ public void awaitTermination() {
*/
protected final class BufferNode extends DelayedContainerWithTimestamp<TRemote>
implements RemoteNode<TEnv, TRemote> {
- private Set<RemoteProcedure> operations;
- private final Set<RemoteProcedure> dispatchedOperations = new HashSet<>();
+ private Set<RemoteProcedure> operations = Sets.newConcurrentHashSet();
+ private final Set<RemoteProcedure> dispatchedOperations = Sets.newConcurrentHashSet();
+ private final Object lock = new Object();
Review comment:
yeah, but abortOperationsInQueue does't happen very often
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2343: HBASE-24962 Optimize BufferNode Lock
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #2343:
URL: https://github.com/apache/hbase/pull/2343#discussion_r483893690
##########
File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
##########
@@ -298,6 +299,8 @@ default boolean storeInDispatchedQueue() {
// ============================================================================================
private final class TimeoutExecutorThread extends Thread {
private final DelayQueue<DelayedWithTimeout> queue = new DelayQueue<DelayedWithTimeout>();
+ private final ConcurrentHashMap<DelayedWithTimeout, DelayedWithTimeout> pendingBufferNode =
Review comment:
thx @virajjasani
yes, but i think it is okay to use either, RemoteProcedureDispatcher doesn't change often
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org