You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/01/18 00:49:00 UTC
[jira] [Commented] (IMPALA-11072) TestSpillingDebugActionDimensions.test_spilling is flaky
[ https://issues.apache.org/jira/browse/IMPALA-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477499#comment-17477499 ]
ASF subversion and git services commented on IMPALA-11072:
----------------------------------------------------------
Commit 577fc2ee21a8cb6fc49f9df8367d63ebccc13e61 in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=577fc2e ]
IMPALA-11072: Deflake TestSpillingDebugActionDimensions.test_spilling
The first test case in TestSpillingDebugActionDimensions.test_spilling
has been flaky for not spilling any partitions in its hash join node.
This patch fixes the flakiness by reducing the buffer_pool_limit from
215 MB to 110 MB, which is around double of the query Per Host Min
Memory Reservation.
Testing:
- Manually run the first test case of
TestSpillingDebugActionDimensions.test_spilling. Verify that both of
the hash joins are spilling and the test pass.
Change-Id: Ie8802505e0dcae1be5e855107436805bd10e0077
Reviewed-on: http://gerrit.cloudera.org:8080/18138
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestSpillingDebugActionDimensions.test_spilling is flaky
> --------------------------------------------------------
>
> Key: IMPALA-11072
> URL: https://issues.apache.org/jira/browse/IMPALA-11072
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 4.0.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> We have seen some failure of TestSpillingDebugActionDimensions.test_spilling in GVO jenkins job and downstream nightly tests. Latest one happen in [https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/15503/]
>
>
> {code:java}
> query_test/test_spilling.py:75: in test_spilling
> self.run_test_case('QueryTest/spilling', vector)
> common/impala_test_suite.py:743: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:636: in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*SpilledPartitions: .* \([1-9][0-9]*\)
> E
> E ACTUAL PROFILE:
> E Query (id=8b433ac02c4d3fd2:3c50b7c400000000):
> E - InactiveTotalTime: 0.000ns
> E - TotalTime: 0.000ns
> E Summary:
> E Session ID: 9448bded8acf05c6:428a7a797f6b9483
> E Session Type: BEESWAX
> E Start Time: 2022-01-08 09:37:07.647285000
> E End Time: 2022-01-08 09:37:15.514936000
> E Query Type: QUERY
> E Query State: FINISHED
> E Impala Query State: FINISHED
> E Query Status: OK
> E Impala Version: impalad version 4.1.0-SNAPSHOT RELEASE (build 560ff976d3a08920a08b4ce3325a1dd9dbe81765)
> E User: ubuntu
> E Connected User: ubuntu
> E Delegated User:
> E Network Address: ::ffff:127.0.0.1:44648
> E Default Db: tpch_parquet
> E Sql Statement: select count(l1.l_tax)
> E from
> E lineitem l1,
> E lineitem l2,
> E lineitem l3
> E where
> E l1.l_tax < 0.01 and
> E l2.l_tax < 0.04 and
> E l1.l_orderkey = l2.l_orderkey and
> E l1.l_orderkey = l3.l_orderkey and
> E l1.l_comment = l3.l_comment and
> E l1.l_shipdate = l3.l_shipdate
> E Coordinator: ip-172-31-21-231:27000
> E Query Options (set by configuration): BUFFER_POOL_LIMIT=225443840,MT_DOP=0,DEFAULT_SPILLABLE_BUFFER_SIZE=262144,TIMEZONE=Universal,CLIENT_IDENTIFIER=query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[protocol:beeswax|exec_option:{'mt_dop':0;'debug_action':None;'default_spillable_buffer_size':'256k'}|table_format:parquet/none]
> E Query Options (set by configuration and planner): BUFFER_POOL_LIMIT=225443840,MT_DOP=0,DEFAULT_SPILLABLE_BUFFER_SIZE=262144,TIMEZONE=Universal,CLIENT_IDENTIFIER=query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[protocol:beeswax|exec_option:{'mt_dop':0;'debug_action':None;'default_spillable_buffer_size':'256k'}|table_format:parquet/none],MINMAX_FILTER_THRESHOLD=0.5,MINMAX_FILTERING_LEVEL=PAGE
> ...{code}
>
> We should lower the configured BUFFER_POOL_LIMIT for this test to less than 215MB so that it spill more consistently.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org