You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/07/02 03:28:00 UTC
[jira] [Commented] (IMPALA-10927) TestFetchAndSpooling.test_rows_sent_counters is flaky in core-s3 based test
[ https://issues.apache.org/jira/browse/IMPALA-10927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561687#comment-17561687 ]
ASF subversion and git services commented on IMPALA-10927:
----------------------------------------------------------
Commit df0a014e364854111368cf5756c4a01b3fb423d6 in impala's branch refs/heads/dependabot/pip/infra/python/deps/urllib3-1.26.5 from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=df0a014e3 ]
IMPALA-10927: Deflaky TestFetchAndSpooling.test_rows_sent_counters
IMPALA-8957 fixed the flakiness for test by adding a delay via
DEBUG_ACTION BPRS_BEFORE_ADD_ROWS in BlockingPlanRootSink::Send().
test_rows_sent_counters uses DEBUG_ACTION BPRS_BEFORE_ADD_BATCH when
spool_query_results is on, and uses BPRS_BEFORE_ADD_ROWS when
spool_query_results is off with assumption that result spooling is
disabled by default.
IMPALA-9856 enabled result spooling by default.
Following two issues were introduced for the test when result spooling
was enabled by default.
1) spool_query_results as false is not covered in the test since
extended dimension is added with spool_query_results as true.
2) Since the test uses BPRS_BEFORE_ADD_ROWS if spool_query_results is
not specified as true, it makes DEBUG_ACTION BPRS_BEFORE_ADD_ROWS to be
used for spool_query_results as true. This causes the test flaky since
no delay to be added in BufferedPlanRootSink::Send().
There is another bug in the test. It uses bool() to convert string to
boolean value, but the function returns true for any non empty string.
This patch changed the extended dimension setting for
spool_query_results as false, and made the test to use the right
DEBUG_ACTION for spool_query_results as true and false.
Also reverted the previous fixing which disabled the test for S3
testing environment.
Testing:
- Ran the test more than 10000 times without failure on Jenkins.
Change-Id: I790bbe1072357caf8ee11bb37644cf29dc8bea0f
Reviewed-on: http://gerrit.cloudera.org:8080/18671
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestFetchAndSpooling.test_rows_sent_counters is flaky in core-s3 based test
> ---------------------------------------------------------------------------
>
> Key: IMPALA-10927
> URL: https://issues.apache.org/jira/browse/IMPALA-10927
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Qifan Chen
> Assignee: Qifan Chen
> Priority: Blocker
> Labels: broken-build, flaky-test
> Fix For: Impala 4.2.0
>
>
> RowsSentRate counter is seen with a value of 0 and the following is the error report. A fix in the similar area was described in IMPALA-8957, where a delay via DEBUG_ACTION BPRS_BEFORE_ADD_BATCH was introduced.
> {code:java}
> query_test.test_fetch.TestFetchAndSpooling.test_rows_sent_counters[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from pytest)
> Failing for the past 1 build (Since Failed#206 )
> Took 0.47 sec.
> Error Message
> query_test/test_fetch.py:101: in test_rows_sent_counters assert float(rows_sent_rate.group(1)) > 0 E assert 0.0 > 0 E + where 0.0 = float('0') E + where '0' = <built-in method group of _sre.SRE_Match object at 0x7f9d22693030>(1) E + where <built-in method group of _sre.SRE_Match object at 0x7f9d22693030> = <_sre.SRE_Match object at 0x7f9d22693030>.group
> Stacktrace
> query_test/test_fetch.py:101: in test_rows_sent_counters
> assert float(rows_sent_rate.group(1)) > 0
> E assert 0.0 > 0
> E + where 0.0 = float('0')
> E + where '0' = <built-in method group of _sre.SRE_Match object at 0x7f9d22693030>(1)
> E + where <built-in method group of _sre.SRE_Match object at 0x7f9d22693030> = <_sre.SRE_Match object at 0x7f9d22693030>.group
> Standard Error
> SET client_identifier=query_test/test_fetch.py::TestFetchAndSpooling::()::test_rows_sent_counters[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table;
> -- connecting to: localhost:21000
> -- connecting to localhost:21050 with impyla
> -- 2021-09-10 04:41:56,852 INFO MainThread: Closing active operation
> -- connecting to localhost:28000 with impyla
> -- 2021-09-10 04:41:56,925 INFO MainThread: Closing active operation
> -- connecting to localhost:11050 with impyla
> -- 2021-09-10 04:41:56,939 INFO MainThread: Could not connect to ('::1', 11050, 0, 0)
> -- 2021-09-10 04:41:56,939 ERROR MainThread: Could not connect to any of [('::1', 11050, 0, 0), ('127.0.0.1', 11050)]
> -- 2021-09-10 04:41:56,939 INFO MainThread: HS2 FENG connection setup failed, continuing...: Could not connect to any of [('::1', 11050, 0, 0), ('127.0.0.1', 11050)]
> SET client_identifier=query_test/test_fetch.py::TestFetchAndSpooling::()::test_rows_sent_counters[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|table;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET debug_action=BPRS_BEFORE_ADD_ROWS:SLEEP@1000;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select id from functional.alltypes limit 10;
> -- 2021-09-10 04:41:56,988 INFO MainThread: Started query b04941a7ffff5e31:1da6c8eb00000000
> Options
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org