[impala] 01/03: IMPALA-8907: TestResultSpooling.test_slow_query is flaky

[ta...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 1915a3e7f49f753cf18b919a087e249fe6769127
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Thu Aug 29 09:28:05 2019 -0700

    IMPALA-8907: TestResultSpooling.test_slow_query is flaky
    
    De-flake test_result_spooling.py::TestResultsSpooling::test_slow_query
    by increasing the delay in RowBatch production. This patch makes two
    fixes to ensure that RowBatchGetWaitTime is a non-zero value:
    
    * Add the DELAY DEBUG_ACTION to the SCAN_NODE rather than the
    EXCHANGE_NODE. Since the EXCHANGE_NODE only processes a few rows, adding
    the delay to the SCAN_NODE decreases the rate at which results are
    produced.
    
    * Wait for all rows to be fetched before checking if RowBatchGetWaitTime
    is in the profile. This fixes a possible race condition where the fetch
    thread was not able to issue the fetch RPC before the test checked if
    RowBatchGetWaitTime was in the runtime profile.
    
    Testing:
    * Looped test_slow_query for several hours with 0 failures
    
    Change-Id: Idcb4a6b38f85a9497f80e2674e1c1fa512be5940
    Reviewed-on: http://gerrit.cloudera.org:8080/14170
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 tests/query_test/test_result_spooling.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/query_test/test_result_spooling.py b/tests/query_test/test_result_spooling.py
index feb0041..7ac93ac 100644
--- a/tests/query_test/test_result_spooling.py
+++ b/tests/query_test/test_result_spooling.py
@@ -154,12 +154,12 @@ class TestResultSpooling(ImpalaTestSuite):
     """Tests results spooling when the client is blocked waiting for Impala to add more
     results to the queue. Validates that RowBatchGetWaitTime (amount of time the client
     spends waiting for Impala to buffer query results) is updated properly."""
-    query = "select id from functional.alltypes order by id limit 10"
+    query = "select id from functional.alltypes"
 
-    # Add a delay to the EXCHANGE_NODE in the query above to simulate a "slow" query. The
+    # Add a delay to the SCAN_NODE in the query above to simulate a "slow" query. The
     # delay should give the client enough time to issue a fetch request and block until
     # Impala produces results.
-    vector.get_value('exec_option')['debug_action'] = '2:GETNEXT:DELAY'
+    vector.get_value('exec_option')['debug_action'] = '0:GETNEXT:DELAY'
     vector.get_value('exec_option')['spool_query_results'] = 'true'
 
     # Regex to look for in the runtime profile.
@@ -173,6 +173,7 @@ class TestResultSpooling(ImpalaTestSuite):
           self.create_impala_client().fetch(query, handle))
       thread.start()
       self.wait_for_state(handle, self.client.QUERY_STATES['FINISHED'], 10)
+      thread.join()
       assert re.search(get_wait_time_regex, self.client.get_runtime_profile(handle)) \
           is not None
     finally: