You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/04/28 23:52:02 UTC
[GitHub] [hbase] bharathv opened a new pull request #1601: [DRAFT] Make HedgedRpcChannel real async
bharathv opened a new pull request #1601:
URL: https://github.com/apache/hbase/pull/1601
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-620924335
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 35s | master passed |
| +1 :green_heart: | checkstyle | 1m 37s | master passed |
| +1 :green_heart: | spotbugs | 2m 56s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 21s | the patch passed |
| -0 :warning: | checkstyle | 1m 5s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 15s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| -1 :x: | spotbugs | 1m 9s | hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 36m 46s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hbase-client |
| | Unread field:HedgedRpcChannel.java:[line 177] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1601 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux d4a60e9f470f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8fceec8cb1 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-general-check/output/new-spotbugs-hbase-client.html |
| Max. process+thread count | 94 (vs. ulimit of 12500) |
| modules | C: hbase-client hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-622940693
In general, for the implementation for HedgedRpcChannel, I think a better solution is to wrap several RpcChannels as the callMethod method is asynchronous.
> // ** WARN ** This is a blocking call if the underlying connection for the rpc client is
// a blocking implementation (ex: BlockingRpcConnection). That essentially serializes all
// the write calls. Handling blocking connection means that this should be run in a separate
// thread and hence more code complexity. Is it ok to handle only non-blocking connections?
The comment in code is incorrect, even for BlockingRpcConnection, the callMethod is still asynchronous...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
bharathv commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-623162755
>The comment in code is incorrect, even for BlockingRpcConnection, the callMethod is still asynchronous...
> But for the protobuf stub, it also has the ability to do asynchronous call(BlockingInterface vs. Interface, newBlockingStub vs. newStub), so wrapping on the stub can also solve the hedged read problem, and this is exactly what we have done for implementing the read replicas feature.
Ah that makes sense. I missed this part, thanks for pointing it out. I was under the impression that the BlockingRpcClient was always defaulted to a blockingRpcChannel instead of async RpcChannel(). In that case, I think it is much easier to wrap multiple RpcChannels and support both connection types.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-620973652
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 56s | Docker mode activated. |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 59s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 6m 30s | master passed |
| +1 :green_heart: | compile | 2m 33s | master passed |
| +1 :green_heart: | shadedjars | 8m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 13s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 54s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 13s | the patch passed |
| +1 :green_heart: | compile | 1m 24s | the patch passed |
| +1 :green_heart: | javac | 1m 24s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 26s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 4s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 186m 5s | hbase-server in the patch passed. |
| | | 227m 22s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1601 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 8465e2840997 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8fceec8cb1 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/testReport/ |
| Max. process+thread count | 3207 (vs. ulimit of 12500) |
| modules | C: hbase-client hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-622943925
BTW, implementing hedged read in rpc framework, it to use a special channel(HedgedRpcChannel) here to wrap multiple RpcChannels, as the RpcChannel provide a callMethod method to do asynchronous call.
But for the protobuf stub, it also has the ability to do asynchronous call(BlockingInterface vs. Interface, newBlockingStub vs. newStub), so wrapping on the stub can also solve the hedged read problem, and this is exactly what we have done for implementing the read replicas feature.
So to be honest, I do not see the necessity to change the wrapping layer from stub to rpc channel...
Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
bharathv commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-623618250
It was decided that we will move the logic to MasterRegistry itself. #1593 does that, it has a very similar logic around how it does the group calls. Closing this for now. See the other PR for more context.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1601: [DRAFT] Make HedgedRpcChannel real async
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #1601:
URL: https://github.com/apache/hbase/pull/1601#issuecomment-620961292
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 1s | master passed |
| +1 :green_heart: | compile | 1m 35s | master passed |
| +1 :green_heart: | shadedjars | 5m 22s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 26s | hbase-client in master failed. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 1s | the patch passed |
| +1 :green_heart: | compile | 1m 34s | the patch passed |
| +1 :green_heart: | javac | 1m 34s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 16s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 28s | hbase-client in the patch failed. |
| -0 :warning: | javadoc | 0m 42s | hbase-server in the patch failed. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 139m 36s | hbase-server in the patch passed. |
| | | 168m 15s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1601 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7556071f1af6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 8fceec8cb1 |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/testReport/ |
| Max. process+thread count | 3680 (vs. ulimit of 12500) |
| modules | C: hbase-client hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1601/1/console |
| versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org