You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/06/27 15:07:00 UTC
[jira] [Commented] (IMPALA-8957) TestFetchAndSpooling.test_rows_sent_counters is flaky
[ https://issues.apache.org/jira/browse/IMPALA-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17559215#comment-17559215 ]
ASF subversion and git services commented on IMPALA-8957:
---------------------------------------------------------
Commit df0a014e364854111368cf5756c4a01b3fb423d6 in impala's branch refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=df0a014e3 ]
IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
test_rows_sent_counters uses DEBUG_ACTION BPRS_BEFORE_ADD_BATCH when
spool_query_results is on, and uses BPRS_BEFORE_ADD_ROWS when
spool_query_results is off with assumption that result spooling is
disabled by default.
IMPALA-9856 enabled result spooling by default.
Following two issues were introduced for the test when result spooling
was enabled by default.
1) spool_query_results as false is not covered in the test since
extended dimension is added with spool_query_results as true.
2) Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay to be added in BufferedPlanRootSink::Send().
There is another bug in the test. It uses bool() to convert string to
boolean value, but the function returns true for any non empty string.
This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.
Testing:
- Ran the test more than 10000 times without failure on Jenkins.
Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Reviewed-on: http://gerrit.cloudera.org:8080/18671
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestFetchAndSpooling.test_rows_sent_counters is flaky
> -----------------------------------------------------
>
> Key: IMPALA-8957
> URL: https://issues.apache.org/jira/browse/IMPALA-8957
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Fix For: Impala 3.4.0
>
>
> Error Details
> {noformat}
> query_test/test_fetch.py:77: in test_rows_sent_counters assert re.search("RowsSentRate: [1-9]", result.runtime_profile) E assert None E + where None = <function search at 0xb14488>('RowsSentRate: [1-9]', 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil... - OptimizationTime: 59.000ms\n - PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n') E + where <function search at 0xb14488> = re.search E + and 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil... - OptimizationTime: 59.000ms\n - PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n' = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0xbbfa5d0>.runtime_profile{noformat}
> Stack Trace
> {noformat}
> query_test/test_fetch.py:77: in test_rows_sent_counters
> assert re.search("RowsSentRate: [1-9]", result.runtime_profile)
> E assert None
> E + where None = <function search at 0xb14488>('RowsSentRate: [1-9]', 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil... - OptimizationTime: 59.000ms\n - PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n')
> E + where <function search at 0xb14488> = re.search
> E + and 'Query (id=3946b19649af9ce3:7f38be6700000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil... - OptimizationTime: 59.000ms\n - PeakMemoryUsage: 213.50 KB (218624)\n - PrepareTime: 26.000ms\n' = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0xbbfa5d0>.runtime_profile{noformat}
> Standard Error
> {noformat}
> SET client_identifier=query_test/test_fetch.py::TestFetchAndSpooling::()::test_rows_sent_counters[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select id from functional.alltypes limit 10;
> -- 2019-09-18 18:51:20,759 INFO MainThread: Started query 3946b19649af9ce3:7f38be6700000000{noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org