You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/01/27 00:11:26 UTC
[GitHub] [hbase] bharathv opened a new pull request #1091: HBASE-23731:
De-flake TestFromClientSide
bharathv opened a new pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091
There were a couple of issues.
- There was a leak of a file descriptor for hbck lock file. This
was contributing to all the "ConnectionRefused" stack traces since
it was trying to renew lease for an already expired mini dfs cluster.
This issue was there for a while, just that we noticed it now.
- After upgrade to JUnit 4.13, it looks like the behavior for test
timeouts has changed. Earlier the timeout seems to have applied for
each parameterized run, but now it looks like it is applied across
all the runs.
This patch fixes both the issues.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1091: HBASE-23731:
De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578974055
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 0s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 17s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | compile | 1m 40s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | checkstyle | 1m 47s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | shadedjars | 5m 42s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +0 :ok: | spotbugs | 5m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 6m 0s | HBASE-18095/client-locate-meta-no-zookeeper passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 32s | the patch passed |
| +1 :green_heart: | compile | 1m 27s | the patch passed |
| +1 :green_heart: | javac | 1m 27s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 48s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 6m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 22m 37s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | javadoc | 1m 11s | the patch passed |
| +1 :green_heart: | findbugs | 6m 44s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 43s | hbase-common in the patch passed. |
| -1 :x: | unit | 180m 15s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 1m 15s | The patch does not generate ASF License warnings. |
| | | 267m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1091 |
| JIRA Issue | HBASE-23731 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 046df39cb8f1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1091/out/precommit/personality/provided.sh |
| git revision | HBASE-18095/client-locate-meta-no-zookeeper / 39ff6f82d9 |
| Default Java | 1.8.0_181 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/testReport/ |
| Max. process+thread count | 5244 (vs. ulimit of 10000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1091: HBASE-23731:
De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578642660
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 52s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 53s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | compile | 1m 20s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | checkstyle | 1m 41s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | shadedjars | 5m 6s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +0 :ok: | spotbugs | 4m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 31s | HBASE-18095/client-locate-meta-no-zookeeper passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 38s | the patch passed |
| +1 :green_heart: | compile | 1m 22s | the patch passed |
| +1 :green_heart: | javac | 1m 22s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 38s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 5m 6s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 17m 36s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
| +1 :green_heart: | findbugs | 5m 32s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 9s | hbase-common in the patch passed. |
| -1 :x: | unit | 169m 16s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. |
| | | 243m 24s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.master.TestMasterShutdown |
| | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1091 |
| JIRA Issue | HBASE-23731 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0e4d97a88c02 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1091/out/precommit/personality/provided.sh |
| git revision | HBASE-18095/client-locate-meta-no-zookeeper / 39ff6f82d9 |
| Default Java | 1.8.0_181 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/2/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/2/testReport/ |
| Max. process+thread count | 5529 (vs. ulimit of 10000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/2/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell merged pull request #1091: HBASE-23731: De-flake
TestFromClientSide
Posted by GitBox <gi...@apache.org>.
apurtell merged pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371072816
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
+ RunWith[] runWiths = clazz.getAnnotationsByType(RunWith.class);
+ boolean isTestParameterized = runWiths != null && Arrays.stream(runWiths).anyMatch(
Review comment:
renamed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371374701
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseClassTestRule.java
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import static junit.framework.TestCase.assertEquals;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+import org.apache.hbase.thirdparty.com.google.common.collect.Iterables;
+
+/**
+ * Tests HBaseClassTestRule.
+ */
+@Category(SmallTests.class)
+public class TestHBaseClassTestRule {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE = HBaseClassTestRule.forClass(
+ TestHBaseClassTestRule.class);
+
+ // Test input classes of various kinds.
+ private static class NonParameterizedClass {
+ void dummy() {
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ParameterizedClassWithNoParametersMethod {
+ void dummy() {
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class InValidParameterizedClass {
+ // Not valid because parameters method is private.
+ @Parameters
+ private static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3, 4);
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass1 {
+ @Parameters
+ public static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5);
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass2 {
+ @Parameters
+ public static Object[] parameters() {
+ return new Integer[] {1, 2, 3, 4, 5, 6};
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass3 {
+ @Parameters
+ public static Iterable<Integer> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5, 6, 7);
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass4 {
+ @Parameters
+ public static Collection<Integer> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9);
+ }
+ }
+
+
+ @RunWith(Parameterized.class)
+ private static class ExtendedParameterizedClass1 extends ValidParameterizedClass1 {
+ // Should be inferred from the parent class.
+ int dummy2(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ExtendedParameterizedClass2 extends ValidParameterizedClass1 {
+ // Should override the parent parameters class.
+ @Parameters
+ public static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3);
+ }
+ }
+
+ @Test public void testNumParameters() {
Review comment:
Better put the Test annotation on a separate line. - Done.
Would it be better to split this one into three separate tests according to their categorization? - TBH, I don't see much value, I feel it is better to group them together because all of them belong to a single logical test (which is testing the number of parameters).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1091: HBASE-23731:
De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578570239
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 51s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | compile | 1m 22s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | checkstyle | 1m 40s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +0 :ok: | spotbugs | 4m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 5m 34s | HBASE-18095/client-locate-meta-no-zookeeper passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 35s | the patch passed |
| +1 :green_heart: | compile | 1m 23s | the patch passed |
| +1 :green_heart: | javac | 1m 23s | the patch passed |
| -1 :x: | checkstyle | 1m 13s | hbase-server: The patch generated 1 new + 103 unchanged - 0 fixed = 104 total (was 103) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 17m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
| +1 :green_heart: | findbugs | 5m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 11s | hbase-common in the patch passed. |
| -1 :x: | unit | 34m 17s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. |
| | | 109m 15s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.client.TestAsyncAccessControlAdminApi |
| | hadoop.hbase.io.hfile.TestHFileScannerImplReferenceCount |
| | hadoop.hbase.io.hfile.TestHFilePrettyPrinter |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1091 |
| JIRA Issue | HBASE-23731 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cd804e956199 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1091/out/precommit/personality/provided.sh |
| git revision | HBASE-18095/client-locate-meta-no-zookeeper / 39ff6f82d9 |
| Default Java | 1.8.0_181 |
| checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/1/artifact/out/diff-checkstyle-hbase-server.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/1/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/1/testReport/ |
| Max. process+thread count | 903 (vs. ulimit of 10000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/1/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1091: HBASE-23731: De-flake
TestFromClientSide
Posted by GitBox <gi...@apache.org>.
saintstack commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578885806
+1'd it. Check out the @HorizonNet comment. This goes to the branch only?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] HorizonNet commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
HorizonNet commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371323900
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/TestHBaseClassTestRule.java
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import static junit.framework.TestCase.assertEquals;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+import org.apache.hbase.thirdparty.com.google.common.collect.Iterables;
+
+/**
+ * Tests HBaseClassTestRule.
+ */
+@Category(SmallTests.class)
+public class TestHBaseClassTestRule {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE = HBaseClassTestRule.forClass(
+ TestHBaseClassTestRule.class);
+
+ // Test input classes of various kinds.
+ private static class NonParameterizedClass {
+ void dummy() {
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ParameterizedClassWithNoParametersMethod {
+ void dummy() {
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class InValidParameterizedClass {
+ // Not valid because parameters method is private.
+ @Parameters
+ private static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3, 4);
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass1 {
+ @Parameters
+ public static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5);
+ }
+ int dummy(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass2 {
+ @Parameters
+ public static Object[] parameters() {
+ return new Integer[] {1, 2, 3, 4, 5, 6};
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass3 {
+ @Parameters
+ public static Iterable<Integer> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5, 6, 7);
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ValidParameterizedClass4 {
+ @Parameters
+ public static Collection<Integer> parameters() {
+ return Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9);
+ }
+ }
+
+
+ @RunWith(Parameterized.class)
+ private static class ExtendedParameterizedClass1 extends ValidParameterizedClass1 {
+ // Should be inferred from the parent class.
+ int dummy2(int a) {
+ return 0;
+ }
+ }
+
+ @RunWith(Parameterized.class)
+ private static class ExtendedParameterizedClass2 extends ValidParameterizedClass1 {
+ // Should override the parent parameters class.
+ @Parameters
+ public static List<Object> parameters() {
+ return Arrays.asList(1, 2, 3);
+ }
+ }
+
+ @Test public void testNumParameters() {
Review comment:
NIT: Would it be better to split this one into three separate tests according to their categorization?
NIT: Better put the `Test` annotation on a separate line.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371083459
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on issue #1091: HBASE-23731: De-flake
TestFromClientSide
Posted by GitBox <gi...@apache.org>.
bharathv commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578890470
Thanks for the quick review @saintstack.
> Check out the @HorizonNet comment. - Already addressed his comments in the last push.
> This goes to the branch only? - Yes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1091: HBASE-23731:
De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578745261
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 49s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ |
| +0 :ok: | mvndep | 0m 49s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 7m 41s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | compile | 1m 45s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | checkstyle | 1m 53s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +1 :green_heart: | shadedjars | 6m 26s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 14s | HBASE-18095/client-locate-meta-no-zookeeper passed |
| +0 :ok: | spotbugs | 5m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 6m 32s | HBASE-18095/client-locate-meta-no-zookeeper passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 32s | the patch passed |
| +1 :green_heart: | compile | 1m 51s | the patch passed |
| +1 :green_heart: | javac | 1m 51s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 52s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 6m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 23m 7s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| +1 :green_heart: | javadoc | 1m 26s | the patch passed |
| +1 :green_heart: | findbugs | 7m 5s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 42s | hbase-common in the patch passed. |
| -1 :x: | unit | 184m 41s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. |
| | | 275m 57s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1091 |
| JIRA Issue | HBASE-23731 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 36c645efe5b3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1091/out/precommit/personality/provided.sh |
| git revision | HBASE-18095/client-locate-meta-no-zookeeper / 39ff6f82d9 |
| Default Java | 1.8.0_181 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/3/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/3/testReport/ |
| Max. process+thread count | 5224 (vs. ulimit of 10000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/3/console |
| versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371068889
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
+ RunWith[] runWiths = clazz.getAnnotationsByType(RunWith.class);
+ boolean isTestParameterized = runWiths != null && Arrays.stream(runWiths).anyMatch(
Review comment:
FYI, isTestParameterized is name of a method, not a variable. testParameterized would be the variable name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371072861
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
+ RunWith[] runWiths = clazz.getAnnotationsByType(RunWith.class);
+ boolean isTestParameterized = runWiths != null && Arrays.stream(runWiths).anyMatch(
+ (r) -> r.value().equals(Parameterized.class));
+ if (isTestParameterized) {
+ for (Method method : clazz.getMethods()) {
Review comment:
Refactored to make more sense. Agree with your comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371069322
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
+ RunWith[] runWiths = clazz.getAnnotationsByType(RunWith.class);
+ boolean isTestParameterized = runWiths != null && Arrays.stream(runWiths).anyMatch(
+ (r) -> r.value().equals(Parameterized.class));
+ if (isTestParameterized) {
+ for (Method method : clazz.getMethods()) {
Review comment:
Style nit, rather than have an if with a big indented clause, I'd avoid the indent by testing for the opposite case and just doing an early return if succeeds -- especially when near the opening of a method. No biggie. Just a consideration for future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1091:
HBASE-23731: De-flake TestFromClientSide
Posted by GitBox <gi...@apache.org>.
saintstack commented on a change in pull request #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#discussion_r371069032
##########
File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##########
@@ -82,6 +99,49 @@ private static long getTimeoutInSeconds(Class<?> clazz) {
clazz.getName() + " does not have SmallTests/MediumTests/LargeTests in @Category");
}
+ /**
+ * @param clazz Test class that is running.
+ * @return the number of parameters for this given test class. If the test is not parameterized or
+ * if there is any issue determining the number of parameters, returns 1.
+ */
+ private static int getNumParameters(Class<?> clazz) {
Review comment:
Can you make a test for this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services