You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/07/12 21:01:26 UTC
[GitHub] [hbase] apurtell opened a new pull request, #4616: HBASE-27194 Add test coverage for SimpleRpcServer
apurtell opened a new pull request, #4616:
URL: https://github.com/apache/hbase/pull/4616
Add test coverage for `SimpleRpcServer`.
Improve the way we test both `SimpleRpcServer` and `NettyRpcServer`: Use `LoadTestKVGenerator` to generate random values with varying sizes between 1000 bytes and 1M bytes, and also to verify them when reading the values back.
Add secure test coverage for both `SimpleRpcServer` and `NettyRpcServer.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4616:
URL: https://github.com/apache/hbase/pull/4616#issuecomment-1182652718
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 36s | master passed |
| +1 :green_heart: | compile | 0m 40s | master passed |
| +1 :green_heart: | shadedjars | 3m 50s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 18s | the patch passed |
| +1 :green_heart: | compile | 0m 41s | the patch passed |
| +1 :green_heart: | javac | 0m 41s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 220m 12s | hbase-server in the patch passed. |
| | | 237m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4616 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux c47617eea61d 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7cb7100de5 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/testReport/ |
| Max. process+thread count | 2495 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4616:
URL: https://github.com/apache/hbase/pull/4616#issuecomment-1182646383
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 57s | master passed |
| +1 :green_heart: | compile | 0m 47s | master passed |
| +1 :green_heart: | shadedjars | 3m 45s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 28s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 39s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed |
| +1 :green_heart: | javac | 0m 46s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 46s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 206m 24s | hbase-server in the patch passed. |
| | | 224m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4616 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 62328c59a636 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7cb7100de5 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/testReport/ |
| Max. process+thread count | 2719 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
apurtell commented on PR #4616:
URL: https://github.com/apache/hbase/pull/4616#issuecomment-1182534101
Local test results
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.ipc.TestSimpleRpcServer
[INFO] Running org.apache.hadoop.hbase.ipc.TestSecureNettyRpcServer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.248 s - in org.apache.hadoop.hbase.ipc.TestSimpleRpcServer
[INFO] Running org.apache.hadoop.hbase.ipc.TestSecureSimpleRpcServer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.826 s - in org.apache.hadoop.hbase.ipc.TestSecureSimpleRpcServer
[INFO] Running org.apache.hadoop.hbase.ipc.TestNettyRpcServer
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.923 s - in org.apache.hadoop.hbase.ipc.TestSecureNettyRpcServer
[INFO] Running org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.578 s - in org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.031 s - in org.apache.hadoop.hbase.ipc.TestNettyRpcServer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4616:
URL: https://github.com/apache/hbase/pull/4616#issuecomment-1182530429
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 33s | master passed |
| +1 :green_heart: | compile | 2m 16s | master passed |
| +1 :green_heart: | checkstyle | 0m 28s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 16s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 4s | the patch passed |
| +1 :green_heart: | compile | 2m 12s | the patch passed |
| +1 :green_heart: | javac | 2m 12s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 27s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 11m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. |
| | | 31m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4616 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux ce18e62f7f69 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7cb7100de5 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 69 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4616/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4616:
URL: https://github.com/apache/hbase/pull/4616#discussion_r919558096
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcServer.java:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.testclassification.RPCTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.LoadTestKVGenerator;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+@Category({ RPCTests.class, MediumTests.class })
+public class TestSimpleRpcServer {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestSimpleRpcServer.class);
+
+ private static final byte[] FAMILY = Bytes.toBytes("f");
+ private static final byte[] QUALIFIER = Bytes.toBytes("q");
+ private static final int NUM_ROWS = 100;
+ private static final int MIN_LEN = 1000;
+ private static final int MAX_LEN = 1000000;
+ protected static final LoadTestKVGenerator GENERATOR = new LoadTestKVGenerator(MIN_LEN, MAX_LEN);
+ protected static HBaseTestingUtil TEST_UTIL;
+
+ @Rule
+ public TestName name = new TestName();
+
+ @SuppressWarnings("deprecation")
+ @BeforeClass
+ public static void setupClass() throws Exception {
+ // A subclass may have already created TEST_UTIL and is now upcalling to us
+ if (TEST_UTIL == null) {
+ TEST_UTIL = new HBaseTestingUtil();
+ }
+ // Set RPC server impl to SimpleRpcServer
+ TEST_UTIL.getConfiguration().set(RpcServerFactory.CUSTOM_RPC_SERVER_IMPL_CONF_KEY,
+ SimpleRpcServer.class.getName());
+ TEST_UTIL.startMiniCluster();
+ }
+
+ @AfterClass
+ public static void tearDownClass() throws Exception {
+ TEST_UTIL.shutdownMiniCluster();
+ }
+
+ @Test
+ public void testSimpleRpcServer() throws Exception {
+ doTest(TableName.valueOf(name.getMethodName()));
+ }
+
+ protected void doTest(TableName tableName) throws Exception {
+ // Splitting just complicates the test scenario
+ final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tableName)
+ .setRegionSplitPolicyClassName(DisabledRegionSplitPolicy.class.getName()).build();
+ final Table table =
+ TEST_UTIL.createTable(desc, new byte[][] { FAMILY }, TEST_UTIL.getConfiguration());
+ try {
+ // put some test data
+ for (int i = 0; i < NUM_ROWS; i++) {
+ final byte[] rowKey = Bytes.toBytes(LoadTestKVGenerator.md5PrefixedKey(i));
+ final byte[] v = GENERATOR.generateRandomSizeValue(rowKey, QUALIFIER);
+ table.put(new Put(rowKey).addColumn(FAMILY, QUALIFIER, v));
+ }
+ // read to verify it.
+ for (int i = 0; i < NUM_ROWS; i++) {
+ final byte[] rowKey = Bytes.toBytes(LoadTestKVGenerator.md5PrefixedKey(i));
+ final Result r = table.get(new Get(rowKey).addColumn(FAMILY, QUALIFIER));
+ assertNotNull("Result was empty", r);
+ final byte[] v = r.getValue(FAMILY, QUALIFIER);
+ assertNotNull("Result did not contain expected value", v);
+ assertTrue("Value was not verified", LoadTestKVGenerator.verify(v, rowKey, QUALIFIER));
+ }
+ } finally {
+ table.close();
Review Comment:
Use try-with-resources
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcServer.java:
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.testclassification.RPCTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.LoadTestKVGenerator;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+@Category({ RPCTests.class, MediumTests.class })
+public class TestSimpleRpcServer {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestSimpleRpcServer.class);
+
+ private static final byte[] FAMILY = Bytes.toBytes("f");
+ private static final byte[] QUALIFIER = Bytes.toBytes("q");
+ private static final int NUM_ROWS = 100;
+ private static final int MIN_LEN = 1000;
+ private static final int MAX_LEN = 1000000;
+ protected static final LoadTestKVGenerator GENERATOR = new LoadTestKVGenerator(MIN_LEN, MAX_LEN);
+ protected static HBaseTestingUtil TEST_UTIL;
+
+ @Rule
+ public TestName name = new TestName();
Review Comment:
Use TableNameTestRule?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #4616: HBASE-27194 Add test coverage for SimpleRpcServer
Posted by GitBox <gi...@apache.org>.
apurtell merged PR #4616:
URL: https://github.com/apache/hbase/pull/4616
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org