You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/03/23 23:41:54 UTC
[GitHub] [hbase] jojochuang opened a new pull request #1334: HBASE-8868. add
metric to report client shortcircuit reads.
jojochuang opened a new pull request #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334
Expose file system level read metrics for RegionServer.
If the HBase RS runs on top of HDFS, calculate the aggregation of
ReadStatistics of each HdfsFileInputStream. These metrics include:
(1) total number of bytes read from HDFS.
(2) total number of bytes read from local DataNode.
(3) total number of bytes read locally through short-circuit read.
(4) total number of bytes read locally through zero-copy read.
Because HDFS ReadStatistics is calculated per input stream, it is not
feasible to update the aggregated number in real time. Instead, the
metrics are updated when an input stream is closed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] jojochuang commented on issue #1334: HBASE-8868. add metric
to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-602943550
The test TestVerifyBucketCacheFile#testRetrieveFromFile fails consistently. However, it doesn't look like it was introduced by this patch. HBASE-23976 Nick reported the same issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] jojochuang commented on issue #1334: HBASE-8868. add metric
to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-603001591
Believe the test failure is due to HBASE-23829.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] jojochuang commented on issue #1334: HBASE-8868. add metric
to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-603988363
Thanks Stack!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack merged pull request #1334: HBASE-8868. add
metric to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1334: HBASE-8868. add
metric to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-602935549
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 6m 6s | master passed |
| +1 :green_heart: | compile | 3m 27s | master passed |
| +1 :green_heart: | shadedjars | 5m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 41s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 42s | the patch passed |
| +1 :green_heart: | compile | 3m 26s | the patch passed |
| +1 :green_heart: | javac | 3m 26s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 24s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 3m 46s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 11m 15s | root in the patch failed. |
| | | 53m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1334 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a3e7333cf7b9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / afc1746757 |
| Default Java | 1.8.0_232 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/testReport/ |
| Max. process+thread count | 566 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1334: HBASE-8868. add
metric to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-602952730
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 6m 28s | master passed |
| +1 :green_heart: | compile | 3m 56s | master passed |
| -1 :x: | shadedjars | 0m 10s | branch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 18s | hbase-hadoop-compat in master failed. |
| -0 :warning: | javadoc | 0m 17s | hbase-hadoop2-compat in master failed. |
| -0 :warning: | javadoc | 0m 20s | root in master failed. |
| -0 :warning: | javadoc | 0m 38s | hbase-server in master failed. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 8s | the patch passed |
| +1 :green_heart: | compile | 3m 55s | the patch passed |
| +1 :green_heart: | javac | 3m 55s | the patch passed |
| -1 :x: | shadedjars | 0m 9s | patch has 7 errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 16s | hbase-hadoop-compat in the patch failed. |
| -0 :warning: | javadoc | 0m 17s | hbase-hadoop2-compat in the patch failed. |
| -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. |
| -0 :warning: | javadoc | 0m 19s | root in the patch failed. |
||| _ Other Tests _ |
| -0 :warning: | unit | 75m 28s | root in the patch failed. |
| | | 103m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1334 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux fbc2355cea06 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / afc1746757 |
| Default Java | 2020-01-14 |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop2-compat.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
| shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop2-compat.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/testReport/ |
| Max. process+thread count | 6178 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1334: HBASE-8868. add
metric to report client shortcircuit reads.
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1334: HBASE-8868. add metric to report client shortcircuit reads.
URL: https://github.com/apache/hbase/pull/1334#issuecomment-602948899
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 48s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 8m 6s | master passed |
| +1 :green_heart: | checkstyle | 3m 7s | master passed |
| +0 :ok: | refguide | 6m 33s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| -1 :x: | spotbugs | 15m 47s | root in master has 1 extant spotbugs warnings. |
| -1 :x: | spotbugs | 2m 45s | hbase-server in master has 1 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 28s | the patch passed |
| -0 :warning: | checkstyle | 2m 41s | root: The patch generated 2 new + 10 unchanged - 0 fixed = 12 total (was 10) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +0 :ok: | refguide | 5m 23s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 12m 14s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. |
| +1 :green_heart: | spotbugs | 16m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 46s | The patch does not generate ASF License warnings. |
| | | 94m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1334 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide |
| uname | Linux d0566709b869 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / afc1746757 |
| refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/branch-site/book.html |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/branch-spotbugs-root-warnings.html |
| spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt |
| refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/artifact/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 84 (vs. ulimit of 10000) |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1334/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services