You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/09/01 12:56:24 UTC
[GitHub] [hive] ayushtkn commented on a diff in pull request #3556: HIVE-26500: Improve TestHiveMetastore
ayushtkn commented on code in PR #3556:
URL: https://github.com/apache/hive/pull/3556#discussion_r960610224
##########
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
##########
@@ -168,16 +169,16 @@ public void testNameMethods() {
try {
List<String> testVals = client.partitionNameToVals(partName);
- assertTrue("Values from name are incorrect", vals.equals(testVals));
+ assertTrue("Values from name are incorrect: " + testVals, vals.equals(testVals));
Map<String, String> testSpec = client.partitionNameToSpec(partName);
- assertTrue("Spec from name is incorrect", spec.equals(testSpec));
+ assertTrue("Spec from name is incorrect: " + testSpec, spec.equals(testSpec));
List<String> emptyVals = client.partitionNameToVals("");
- assertTrue("Values should be empty", emptyVals.size() == 0);
+ assertTrue("Values should be empty, instead its size is: " + emptyVals.size(), emptyVals.size() == 0);
Map<String, String> emptySpec = client.partitionNameToSpec("");
- assertTrue("Spec should be empty", emptySpec.size() == 0);
+ assertTrue("Spec should be empty, instead its size is: " + emptySpec.size(), emptySpec.size() == 0);
Review Comment:
Can be changed to ``assertEquals``
##########
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
##########
@@ -3306,22 +3314,38 @@ public void testJDOPersistanceManagerCleanup() throws Exception {
return;
}
- int numObjectsBeforeClose = getJDOPersistanceManagerCacheSize();
+ int numObjectsBeforeUse = getJDOPersistanceManagerCacheSize();
HiveMetaStoreClient closingClient = new HiveMetaStoreClient(conf);
closingClient.getAllDatabases();
closingClient.close();
- Thread.sleep(5 * 1000); // give HMS time to handle close request
- int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
- assertTrue(numObjectsBeforeClose == numObjectsAfterClose);
+ MetastoreTestUtils.waitForAssertion("Checking pm cachesize after client close", new Runnable() {
+ @Override
+ public void run() {
+ int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
+ assertTrue(String.format("numObjectsBeforeUse: %d != numObjectsAfterClose: %d", numObjectsBeforeUse,
+ numObjectsAfterClose), numObjectsBeforeUse == numObjectsAfterClose);
+ }
+ }, 500, 30000);
Review Comment:
Can use Lambda & assertTrue can be changed to ``assertEqual``?
Something like:
```
MetastoreTestUtils.waitForAssertion("Checking pm cachesize after client close", () -> {
int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
assertEquals(String
.format("numObjectsBeforeUse: %d != numObjectsAfterClose: %d", numObjectsBeforeUse, numObjectsAfterClose),
numObjectsBeforeUse, numObjectsAfterClose);
}, 500, 30000);
```
##########
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
##########
@@ -3306,22 +3314,38 @@ public void testJDOPersistanceManagerCleanup() throws Exception {
return;
}
- int numObjectsBeforeClose = getJDOPersistanceManagerCacheSize();
+ int numObjectsBeforeUse = getJDOPersistanceManagerCacheSize();
HiveMetaStoreClient closingClient = new HiveMetaStoreClient(conf);
closingClient.getAllDatabases();
closingClient.close();
- Thread.sleep(5 * 1000); // give HMS time to handle close request
- int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
- assertTrue(numObjectsBeforeClose == numObjectsAfterClose);
+ MetastoreTestUtils.waitForAssertion("Checking pm cachesize after client close", new Runnable() {
+ @Override
+ public void run() {
+ int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
+ assertTrue(String.format("numObjectsBeforeUse: %d != numObjectsAfterClose: %d", numObjectsBeforeUse,
+ numObjectsAfterClose), numObjectsBeforeUse == numObjectsAfterClose);
+ }
+ }, 500, 30000);
+
+ int numObjectsAfterClose = getJDOPersistanceManagerCacheSize();
HiveMetaStoreClient nonClosingClient = new HiveMetaStoreClient(conf);
nonClosingClient.getAllDatabases();
// Drop connection without calling close. HMS thread deleteContext
// will trigger cleanup
nonClosingClient.getTTransport().close();
- Thread.sleep(5 * 1000);
- int numObjectsAfterDroppedConnection = getJDOPersistanceManagerCacheSize();
- assertTrue(numObjectsAfterClose == numObjectsAfterDroppedConnection);
+
+ MetastoreTestUtils.waitForAssertion("Checking pm cachesize after transport close", new Runnable() {
+ @Override
+ public void run() {
+ int numObjectsAfterDroppedConnection = getJDOPersistanceManagerCacheSize();
+ assertTrue(String.format("numObjectsAfterClose: %d != numObjectsAfterDroppedConnection: %d",
+ numObjectsAfterClose, numObjectsAfterDroppedConnection),
+ numObjectsAfterClose == numObjectsAfterDroppedConnection);
+ }
+ }, 500, 30000);
Review Comment:
Same as above can use Lambda & change to ``assertEquals``
```
MetastoreTestUtils.waitForAssertion("Checking pm cachesize after transport close", () -> {
int numObjectsAfterDroppedConnection = getJDOPersistanceManagerCacheSize();
assertEquals(String
.format("numObjectsAfterClose: %d != numObjectsAfterDroppedConnection: %d", numObjectsAfterClose,
numObjectsAfterDroppedConnection), numObjectsAfterClose, numObjectsAfterDroppedConnection);
}, 500, 30000);
```
##########
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/utils/MetastoreTestUtils.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.metastore.utils;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Common utils for metastore tests.
+ */
+public class MetastoreTestUtils {
+
+ private static final Logger LOG = LoggerFactory.getLogger(MetastoreTestUtils.class);
+
+ /**
+ * This method can run an assertion wrapped in to a runnable, and keep retrying it for a certain amount of time.
+ * It can be useful when the assertion doesn't necessarily pass immediately, and it would be hard
+ * to mock into production code in order to wait for some conditions properly. Instead, it just
+ * waits, but not like a constant time before a single attempt (which is easy, but errorprone).
+ * @param assertionContext
+ * @param runnable
+ * @param msBetweenAssertionAttempts
+ * @param msOverallTimeout
+ * @throws Exception
+ */
+ public static void waitForAssertion(String assertionContext, Runnable assertionRunnable, int msBetweenAssertionAttempts,
Review Comment:
Hadoop common does provide similar util:
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java#L440
It is used in hive-code at a couple of places, but I think there is no hadoop-common test dependency in this module? if it is we can use that? or just give a check if we can copy that if it has something better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org