You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by bo...@apache.org on 2022/02/24 10:33:46 UTC
[impala] 03/03: IMPALA-11135: Deflake LEFT ANTI JOIN test case in test_spilling.py
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 873fe2e5241c5714dfd94a186d524edc1cbad0ad
Author: Riza Suminto <ri...@cloudera.com>
AuthorDate: Mon Feb 21 11:19:39 2022 -0800
IMPALA-11135: Deflake LEFT ANTI JOIN test case in test_spilling.py
TestSpillingDebugActionDimensions.test_spilling has been flaky because a
test case from IMPALA-9725 sometimes does not spill its hash join
partition. This patch lowers the buffer_pool_limit of this test from
110MB to 105MB, just slightly above its Max Per-Host Resource
Reservation (104.61MB), to ensure consistent spilling behavior.
Testing:
After lowering the buffer pool limit, I loop the test 1000 times, and
all spill consistently in fragment "HASH_JOIN_NODE (id=14)".
To be specific, these are the num of SpilledPartitions of the first
instance (ending with "000d") of "Hash Join Builder (join_node_id=14)"
fragment across 1000 query runs:
+--------------------+----------+
| #SpilledPartitions | #Queries |
+--------------------+----------+
| 2 | 30 |
| 3 | 96 |
| 4 | 674 |
| 5 | 52 |
| 6 | 146 |
| 7 | 2 |
+--------------------+----------+
Change-Id: Idad9fc6ec6a0ba7fc70e0701e567da7165e40e83
Reviewed-on: http://gerrit.cloudera.org:8080/18261
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
testdata/workloads/functional-query/queries/QueryTest/spilling.test | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/testdata/workloads/functional-query/queries/QueryTest/spilling.test b/testdata/workloads/functional-query/queries/QueryTest/spilling.test
index 2632953..5867949 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/spilling.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/spilling.test
@@ -313,7 +313,9 @@ Repartitioning did not reduce the size of a spilled partition
# This test is crafted to always return 0 rows, but with the previous bug
# would sometimes return rows while spilling. The query needs to have a join
# key that is more than 256 bytes wide to trigger the bug.
-set buffer_pool_limit=110m;
+# Need to lower buffer_pool_limit just slightly above the Max Per-Host Resource
+# Reservation (104.61MB) for the join to spill consistently.
+set buffer_pool_limit=105m;
set runtime_filter_mode=0;
SELECT straight_join o_orderkey
FROM (