[impala] 01/02: IMPALA-10092: Do not skip test vectors of Kudu tests in a custom cluster

[st...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

stakiar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 34668fab878c224632710670618b3f0176cbc78d
Author: Fang-Yu Rao <fa...@cloudera.com>
AuthorDate: Thu Aug 20 14:51:15 2020 -0700

    IMPALA-10092: Do not skip test vectors of Kudu tests in a custom cluster
    
    We found that the following 4 tests do not run even we remove all the
    decorators like "@SkipIfKudu.no_hybrid_clock" or
    "@SkipIfHive3.kudu_hms_notifications_not_supported" to skip the tests.
    This is due to the fact that those 3 classes inherit the class of
    CustomClusterTestSuite, which adds a constraint that only allows test
    vectors with 'file_format' and 'compression_codec' being "text" and
    "none", respectively, to be run.
    
    1. TestKuduOperations::test_local_tz_conversion_ops
    2. TestKuduClientTimeout::test_impalad_timeout
    3. TestKuduHMSIntegration::test_create_managed_kudu_tables
    4. TestKuduHMSIntegration::test_kudu_alter_table
    
    To address this issue, in this patch we create a parent class for those
    3 classes above and override the method of
    add_custom_cluster_constraints() for this newly created parent class so
    that we do not skip test vectors with 'file_format' and
    'compression_codec' being "kudu" and "none", respectively.
    
    On the other hand, this patch also removes a redundant method call to
    super(CustomClusterTestSuite, cls).add_test_dimensions() in
    CustomClusterTestSuite.add_custom_cluster_constraints() since
    super(CustomClusterTestSuite, cls).add_test_dimensions() had
    already been called immediately before the call to
    add_custom_cluster_constraints() in
    CustomClusterTestSuite.add_test_dimensions().
    
    Testing:
     - Manually verified that after removing the decorators to skip those
       tests, those tests could be run.
    
    Change-Id: I60a4bd4ac5a9026629fb840ab9cc7b5f9948290c
    Reviewed-on: http://gerrit.cloudera.org:8080/16348
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 tests/common/custom_cluster_test_suite.py |  1 -
 tests/custom_cluster/test_kudu.py         | 26 +++++++++++++++++++++++---
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/tests/common/custom_cluster_test_suite.py b/tests/common/custom_cluster_test_suite.py
index a3f2fd6..6e83046 100644
--- a/tests/common/custom_cluster_test_suite.py
+++ b/tests/common/custom_cluster_test_suite.py
@@ -74,7 +74,6 @@ class CustomClusterTestSuite(ImpalaTestSuite):
     # Defines constraints for custom cluster tests, called by add_test_dimensions.
     # By default, custom cluster tests only run on text/none and with a limited set of
     # exec options. Subclasses may override this to relax these default constraints.
-    super(CustomClusterTestSuite, cls).add_test_dimensions()
     cls.ImpalaTestMatrix.add_constraint(lambda v:
         v.get_value('table_format').file_format == 'text' and
         v.get_value('table_format').compression_codec == 'none')
diff --git a/tests/custom_cluster/test_kudu.py b/tests/custom_cluster/test_kudu.py
index 96c8717..fb38114 100644
--- a/tests/custom_cluster/test_kudu.py
+++ b/tests/custom_cluster/test_kudu.py
@@ -28,7 +28,26 @@ from tests.common.test_dimensions import add_exec_option_dimension
 KUDU_MASTER_HOSTS = pytest.config.option.kudu_master_hosts
 LOG = logging.getLogger(__name__)
 
-class TestKuduOperations(CustomClusterTestSuite, KuduTestSuite):
+
+class CustomKuduTest(CustomClusterTestSuite, KuduTestSuite):
+
+  @classmethod
+  def get_workload(cls):
+    return 'functional-query'
+
+  @classmethod
+  def add_custom_cluster_constraints(cls):
+    # Override this method to relax the set of constraints added in
+    # CustomClusterTestSuite.add_custom_cluster_constraints() so that a test vector with
+    # 'file_format' and 'compression_codec' being "kudu" and "none" respectively will not
+    # be skipped.
+    cls.ImpalaTestMatrix.add_constraint(lambda v:
+        v.get_value('exec_option')['batch_size'] == 0 and
+        v.get_value('exec_option')['disable_codegen'] is False and
+        v.get_value('exec_option')['num_nodes'] == 0)
+
+
+class TestKuduOperations(CustomKuduTest):
 
   @classmethod
   def get_workload(cls):
@@ -95,7 +114,8 @@ class TestKuduOperations(CustomClusterTestSuite, KuduTestSuite):
     except Exception as e:
       assert "Error overflow in Kudu session." in str(e)
 
-class TestKuduClientTimeout(CustomClusterTestSuite, KuduTestSuite):
+
+class TestKuduClientTimeout(CustomKuduTest):
   """Kudu tests that set the Kudu client operation timeout to 1ms and expect
      specific timeout exceptions. While we expect all exercised operations to take at
      least 1ms, it is possible that some may not and thus the test could be flaky. If
@@ -114,7 +134,7 @@ class TestKuduClientTimeout(CustomClusterTestSuite, KuduTestSuite):
     self.run_test_case('QueryTest/kudu-timeouts-impalad', vector)
 
 
-class TestKuduHMSIntegration(CustomClusterTestSuite, KuduTestSuite):
+class TestKuduHMSIntegration(CustomKuduTest):
   # TODO(IMPALA-8614): parameterize the common tests in query_test/test_kudu.py
   # to run with HMS integration enabled. Also avoid restarting Impala to reduce
   # tests time.