[jira] [Commented] (IMPALA-9856) Enable result spooling by default

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17559216#comment-17559216 ] 

ASF subversion and git services commented on IMPALA-9856:
---------------------------------------------------------

Commit df0a014e364854111368cf5756c4a01b3fb423d6 in impala's branch refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=df0a014e3 ]

IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters

IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
test_rows_sent_counters uses DEBUG_ACTION BPRS_BEFORE_ADD_BATCH when
spool_query_results is on, and uses BPRS_BEFORE_ADD_ROWS when
spool_query_results is off with assumption that result spooling is
disabled by default.

IMPALA-9856 enabled result spooling by default.
Following two issues were introduced for the test when result spooling
was enabled by default.
1) spool_query_results as false is not covered in the test since
extended dimension is added with spool_query_results as true.
2) Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay to be added in BufferedPlanRootSink::Send().

There is another bug in the test. It uses bool() to convert string to
boolean value, but the function returns true for any non empty string.

This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.

Testing:
  - Ran the test more than 10000 times without failure on Jenkins.

Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Reviewed-on: http://gerrit.cloudera.org:8080/18671
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Enable result spooling by default
> ---------------------------------
>
>                 Key: IMPALA-9856
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9856
>             Project: IMPALA
>          Issue Type: Task
>          Components: Backend
>            Reporter: Sahil Takiar
>            Assignee: Riza Suminto
>            Priority: Major
>             Fix For: Impala 4.0.0
>
>
> Result spooling has been relatively stable since it was introduced, and it has several benefits described in IMPALA-8656. It would be good to enable it by default.
> I looked into doing this a while ago, and there are a bunch of tests that rely on the "fetch one row batch at a time" behavior. Those tests fail when result spooling is enabled.
> The remaining linked tasks in IMPALA-8656 should be completed as well before enabling result spooling by default.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org