[impala] 01/02: IMPALA-9755: Flaky test: test_global_exchange_counters

[jo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 95ee26354dc0ce61e5844430d1eaf553fd13d154
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Tue May 19 14:58:23 2020 -0700

    IMPALA-9755: Flaky test: test_global_exchange_counters
    
    De-flake TestObservability.test_global_exchange_counters in
    test_observability.py.
    
    IMPALA-6984 added a feature to send a Cancel RPC to running fragments
    when the coordinator fragment fetches all rows defined by a limit. This
    causes fragments to terminate early (which is a good thing). However,
    test_global_exchange_counters expects each fragment to produce some
    rows, which is why it recently became flaky.
    
    This patch modifies test_global_exchange_counters so that it allows for
    some fragments to produce 0 rows.
    
    Testing:
    * Ran test_observability.py locally
    * Looped 8 concurrent streams of test_global_exchange_counters for an
      hour, no failures (previously I was able to reproduce the test issue
      within 5 minutes)
    
    Change-Id: Icb3a1b5ccb5695eb71343e96cc830f12d5c72f1e
    Reviewed-on: http://gerrit.cloudera.org:8080/15960
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 tests/query_test/test_observability.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tests/query_test/test_observability.py b/tests/query_test/test_observability.py
index 101f086..8f27c72 100644
--- a/tests/query_test/test_observability.py
+++ b/tests/query_test/test_observability.py
@@ -404,7 +404,6 @@ class TestObservability(ImpalaTestSuite):
   def __verify_profile_event_sequence(self, event_regexes, runtime_profile):
     """Check that 'event_regexes' appear in a consecutive series of lines in
        'runtime_profile'"""
-    lines = runtime_profile.splitlines()
     event_regex_index = 0
 
     # Check that the strings appear in the above order with no gaps in the profile.
@@ -501,9 +500,13 @@ class TestObservability(ImpalaTestSuite):
         if key in line:
           # Match byte count within parentheses
           m = re.search("\(([0-9]+)\)", line)
-          assert m, "Cannot match pattern for key %s in line '%s'" % (key, line)
-          # Only keep first (query-level) counter
-          if counters[key] == 0:
+
+          # If a match was not found, then the value of the key should be 0
+          if not m:
+            assert key + ": 0" in line, "Invalid format for key %s" % key
+            assert counters[key] != 0, "Query level counter for key %s cannot be 0" % key
+          elif counters[key] == 0:
+            # Only keep first (query-level) counter
             counters[key] = int(m.group(1))
 
     # All counters have values