[impala] 02/04: IMPALA-8064: Fix intermittent test failures from test_min_max_filters

[mi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

mikeb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit e5af7f92926f94fd4c91a8c30089de142ea5f3ac
Author: Janaki Lahorani <ja...@cloudera.com>
AuthorDate: Thu Jan 24 09:57:46 2019 -0800

    IMPALA-8064: Fix intermittent test failures from test_min_max_filters
    
    test_min_max_filters and test_decimal_min_max_filters records the aggregated probe rows to
    check whether min-max filter was exercised.  In the case of ASAN builds, the probe side
    started processing before the filters reached the probe side, because ASAN builds are a
    little slower.  The resolution was to increase RUNTIME_FILTER_WAIT_TIME_MS to accommodate ASAN.
    After this patch was merged, https://gerrit.cloudera.org/#/c/12224/ the issue is still seen.
    
    After further analysis, it was found from the profile that though RUNTIME_FILTER_WAIT_TIME_MS
    was set to 100000, the probe side waited for much less than 100000.
    -- Begin Profile Snippet where probe side waited for 36768 ms instead of 100000
          Instance 2b4a14d9933eb108:7565b86500000005 (host=downstream-jenkins-host:22002):(Total: 1m42s, non-child: 0.000ns, % non-child: 0.00%)
            HASH_JOIN_NODE (id=2):(Total: 39s251ms, non-child: 6.999ms, % non-child: 0.01%)
                Runtime filters: 1 of 1 Runtime Filter Published
            KUDU_SCAN_NODE (id=0):(Total: 36s812ms, non-child: 36s812ms, % non-child: 100.00%)
              Runtime filters: Not all filters arrived (arrived: [], missing [1]), waited for 36s768ms
    -- End Profile Snippet
    
    After analyzing the code we found that in ScanNode::WaitForRuntimeFilters, int32 was used for time.
    But it should be using int64.  This is something that needs to be fixed.  There is some probability
    that using 32 bit caused an overflow resulting in the actual wait time shown in profile to be less
    than 100000.
    
    This patch fixes the time variables in ScanNode::WaitForRuntimeFilters to int64.
    
    Change-Id: I7d560adfb098be455a5a89c6de76ac1f17d1d3e3
    Reviewed-on: http://gerrit.cloudera.org:8080/12269
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 be/src/exec/scan-node.cc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/be/src/exec/scan-node.cc b/be/src/exec/scan-node.cc
index 46f843a..ce04a38 100644
--- a/be/src/exec/scan-node.cc
+++ b/be/src/exec/scan-node.cc
@@ -161,13 +161,13 @@ void ScanNode::Close(RuntimeState* state) {
 }
 
 bool ScanNode::WaitForRuntimeFilters() {
-  int32 wait_time_ms = FLAGS_runtime_filter_wait_time_ms;
+  int32_t wait_time_ms = FLAGS_runtime_filter_wait_time_ms;
   if (runtime_state_->query_options().runtime_filter_wait_time_ms > 0) {
     wait_time_ms = runtime_state_->query_options().runtime_filter_wait_time_ms;
   }
   vector<string> arrived_filter_ids;
   vector<string> missing_filter_ids;
-  int32_t start = MonotonicMillis();
+  int64_t start = MonotonicMillis();
   for (auto& ctx: filter_ctxs_) {
     string filter_id = Substitute("$0", ctx.filter->id());
     if (ctx.filter->WaitForArrival(wait_time_ms)) {
@@ -176,7 +176,7 @@ bool ScanNode::WaitForRuntimeFilters() {
       missing_filter_ids.push_back(filter_id);
     }
   }
-  int32_t end = MonotonicMillis();
+  int64_t end = MonotonicMillis();
   const string& wait_time = PrettyPrinter::Print(end - start, TUnit::TIME_MS);
 
   if (arrived_filter_ids.size() == filter_ctxs_.size()) {