You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/11/20 03:45:24 UTC
[GitHub] [hbase] saintstack opened a new pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
saintstack opened a new pull request #2682:
URL: https://github.com/apache/hbase/pull/2682
…formance. (#2644)
Signed-off-by: Huaxiang Sun <hu...@apache.com>
Signed-off-by: stack <st...@apache.com>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-730842327
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | master passed |
| +1 :green_heart: | checkstyle | 0m 23s | master passed |
| +1 :green_heart: | spotbugs | 0m 59s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 26s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 23s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 23m 9s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 1m 7s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 46m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux 030cecc8d8bf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1528aacacd |
| Max. process+thread count | 94 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
saintstack commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-731369745
I force pushed Clara's fixes for @Apache9 comments. I also fixed 'author'. It was me because I'd pushed this. Set it back to Clara.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-731385659
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 29s | master passed |
| +1 :green_heart: | checkstyle | 0m 21s | master passed |
| +1 :green_heart: | spotbugs | 0m 49s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 36s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 19s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 17m 53s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 52s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| | | 36m 21s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
| uname | Linux d50c3e672ffd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / eca904e0fb |
| Max. process+thread count | 94 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#discussion_r527491930
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1481,6 +1489,31 @@ void onTakedown() throws IOException {
}
}
+ /*
+ Parent class for all meta tests: MetaWriteTest, MetaRandomReadTest and CleanMetaTest
+ */
+ static abstract class MetaTest extends TableTest {
+ protected int keyLength;
+
+ MetaTest(Connection con, TestOptions options, Status status) {
+ super(con, options, status);
+ keyLength = Integer.toString(opts.perClientRunRows).length();
+ }
+
+ @Override
+ void onTakedown() throws IOException {
+ // No clean up
+ }
+
+ /*
+ Generates Lexicographically ascending strings
Review comment:
Ditto.
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1481,6 +1489,31 @@ void onTakedown() throws IOException {
}
}
+ /*
+ Parent class for all meta tests: MetaWriteTest, MetaRandomReadTest and CleanMetaTest
Review comment:
nits: add a * before the line.
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1999,6 +2032,47 @@ protected void testTakedown() throws IOException {
}
}
+ /*
+ Send random reads against fake regions inserted by MetaWriteTest
+ */
+ static class MetaRandomReadTest extends MetaTest {
+ private Random rd = new Random();
+ private RegionLocator regionLocator;
+
+ MetaRandomReadTest(Connection con, TestOptions options, Status status) {
+ super(con, options, status);
+ LOG.info("call getRegionLocation");
+ }
+
+ @Override
+ void onStartup() throws IOException {
+ super.onStartup();
+ this.regionLocator = connection.getRegionLocator(table.getName());
+ }
+
+ @Override
+ boolean testRow(final int i, final long startTime) throws IOException, InterruptedException {
+ if (opts.randomSleep > 0) {
+ Thread.sleep(rd.nextInt(opts.randomSleep));
+ }
+ HRegionLocation hRegionLocation = regionLocator.getRegionLocation(
+ getSplitKey(rd.nextInt(opts.perClientRunRows)), true);
+ LOG.debug("get location for region: " + hRegionLocation);
+ return true;
+ }
+
+ @Override
+ protected int getReportingPeriod() {
+ int period = opts.perClientRunRows / 10;
Review comment:
Out of interest, we have several pattern of the implementations for this method. Some just returns opts.perClientRunRows, some divide by 10, some divide by 100. What's the difference here? Why we choose 10 here? Because the random read test uses 10? Not a big concern, just asking.
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1999,6 +2032,47 @@ protected void testTakedown() throws IOException {
}
}
+ /*
+ Send random reads against fake regions inserted by MetaWriteTest
+ */
+ static class MetaRandomReadTest extends MetaTest {
+ private Random rd = new Random();
Review comment:
Since we do not pre set a seed here, please just use ThreadLocalRandom instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #2682:
URL: https://github.com/apache/hbase/pull/2682
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-735339100
Let's get this done? You are free to fix the above nits or not before merging. Not big problems. I think we need this in 2.4.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-730835584
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | master passed |
| +1 :green_heart: | compile | 0m 26s | master passed |
| +1 :green_heart: | shadedjars | 6m 32s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 25s | the patch passed |
| +1 :green_heart: | javac | 0m 25s | the patch passed |
| +1 :green_heart: | shadedjars | 7m 31s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 11m 2s | hbase-mapreduce in the patch passed. |
| | | 35m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9e6cd72fc7b9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1528aacacd |
| Default Java | AdoptOpenJDK-1.8.0_232-b09 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/testReport/ |
| Max. process+thread count | 4874 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-731385420
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | master passed |
| +1 :green_heart: | compile | 0m 27s | master passed |
| +1 :green_heart: | shadedjars | 6m 45s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 5s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 44s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 26s | hbase-mapreduce in the patch passed. |
| | | 35m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ad4483e6a6cd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / eca904e0fb |
| Default Java | AdoptOpenJDK-11.0.6+10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/testReport/ |
| Max. process+thread count | 3959 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-731385308
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 9s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | master passed |
| +1 :green_heart: | compile | 0m 25s | master passed |
| +1 :green_heart: | shadedjars | 6m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 29s | the patch passed |
| +1 :green_heart: | compile | 0m 25s | the patch passed |
| +1 :green_heart: | javac | 0m 25s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 11m 7s | hbase-mapreduce in the patch passed. |
| | | 35m 31s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b8a22287b64a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / eca904e0fb |
| Default Java | AdoptOpenJDK-1.8.0_232-b09 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/testReport/ |
| Max. process+thread count | 3675 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#issuecomment-730835238
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | master passed |
| +1 :green_heart: | compile | 0m 29s | master passed |
| +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 18s | hbase-mapreduce in the patch passed. |
| | | 34m 19s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/2682 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 80067e1b12b8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1528aacacd |
| Default Java | AdoptOpenJDK-11.0.6+10 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/testReport/ |
| Max. process+thread count | 5028 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2682/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#discussion_r527885165
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1999,6 +2032,47 @@ protected void testTakedown() throws IOException {
}
}
+ /*
+ Send random reads against fake regions inserted by MetaWriteTest
+ */
+ static class MetaRandomReadTest extends MetaTest {
+ private Random rd = new Random();
+ private RegionLocator regionLocator;
+
+ MetaRandomReadTest(Connection con, TestOptions options, Status status) {
+ super(con, options, status);
+ LOG.info("call getRegionLocation");
+ }
+
+ @Override
+ void onStartup() throws IOException {
+ super.onStartup();
+ this.regionLocator = connection.getRegionLocator(table.getName());
+ }
+
+ @Override
+ boolean testRow(final int i, final long startTime) throws IOException, InterruptedException {
+ if (opts.randomSleep > 0) {
+ Thread.sleep(rd.nextInt(opts.randomSleep));
+ }
+ HRegionLocation hRegionLocation = regionLocator.getRegionLocation(
+ getSplitKey(rd.nextInt(opts.perClientRunRows)), true);
+ LOG.debug("get location for region: " + hRegionLocation);
+ return true;
+ }
+
+ @Override
+ protected int getReportingPeriod() {
+ int period = opts.perClientRunRows / 10;
Review comment:
I didn't implement the others but my understanding is it depends on the balance of getting more frequent reports and the overhead of IO. I chose 10 because random read test uses 10. And it works fine in my test. We might want to make it configurable in the future if we run into issues with too many IO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] clarax commented on a change in pull request #2682: HBASE-25127 Enhance PerformanceEvaluation to profile meta replica per…
Posted by GitBox <gi...@apache.org>.
clarax commented on a change in pull request #2682:
URL: https://github.com/apache/hbase/pull/2682#discussion_r527930766
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
##########
@@ -1999,6 +2032,47 @@ protected void testTakedown() throws IOException {
}
}
+ /*
+ Send random reads against fake regions inserted by MetaWriteTest
+ */
+ static class MetaRandomReadTest extends MetaTest {
+ private Random rd = new Random();
Review comment:
Changed to existing random instance that is initialized by current system time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org