You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/02/24 10:34:00 UTC
[jira] [Commented] (IMPALA-9725) LEFT ANTI JOIN produces wrong result when PHJ build spills
[ https://issues.apache.org/jira/browse/IMPALA-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497328#comment-17497328 ]
ASF subversion and git services commented on IMPALA-9725:
---------------------------------------------------------
Commit 873fe2e5241c5714dfd94a186d524edc1cbad0ad in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=873fe2e ]
IMPALA-11135: Deflake LEFT ANTI JOIN test case in test_spilling.py
TestSpillingDebugActionDimensions.test_spilling has been flaky because a
test case from IMPALA-9725 sometimes does not spill its hash join
partition. This patch lowers the buffer_pool_limit of this test from
110MB to 105MB, just slightly above its Max Per-Host Resource
Reservation (104.61MB), to ensure consistent spilling behavior.
Testing:
After lowering the buffer pool limit, I loop the test 1000 times, and
all spill consistently in fragment "HASH_JOIN_NODE (id=14)".
To be specific, these are the num of SpilledPartitions of the first
instance (ending with "000d") of "Hash Join Builder (join_node_id=14)"
fragment across 1000 query runs:
+--------------------+----------+
| #SpilledPartitions | #Queries |
+--------------------+----------+
| 2 | 30 |
| 3 | 96 |
| 4 | 674 |
| 5 | 52 |
| 6 | 146 |
| 7 | 2 |
+--------------------+----------+
Change-Id: Idad9fc6ec6a0ba7fc70e0701e567da7165e40e83
Reviewed-on: http://gerrit.cloudera.org:8080/18261
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> LEFT ANTI JOIN produces wrong result when PHJ build spills
> ----------------------------------------------------------
>
> Key: IMPALA-9725
> URL: https://issues.apache.org/jira/browse/IMPALA-9725
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.11.0, Impala 3.0, Impala 2.12.0, Impala 3.1.0, Impala 3.2.0, Impala 3.3.0, Impala 3.4.0
> Reporter: Xiaomin Zhang
> Assignee: Tim Armstrong
> Priority: Blocker
> Labels: correctness
> Fix For: Impala 4.0.0, Impala 3.4.1
>
> Attachments: wide.csv.gz
>
>
> Using the attached data set, below query produced non-zero result when setting a small mem_limt. The expected result should be 0 because we are simply ANTI JOIN the same table.
> set mem_limit=100m;
> select id from wide_test.wide L1 where not exists (
> select 1 from wide_test.wide L2 where L1.id = L2.id
> and L1.col2 = L2.col2
> and L1.col3 = L2.col3
> and L1.col4 = L2.col4
> and L1.col5 = L2.col5
> and L1.col6 = L2.col6
> and L1.col7 = L2.col7
> and L1.col8 = L2.col8
> and L1.col9 = L2.col9
> and L1.col10 = L2.col10
> and L1.col11 = L2.col11
> and L1.col12 = L2.col12
> and L1.col13 = L2.col13
> and L1.col14 = L2.col14
> and L1.col15 = L2.col15
> and L1.col16 = L2.col16
> and L1.col17 = L2.col17
> and L1.col18 = L2.col18
> and L1.col19 = L2.col19
> and L1.col20 = L2.col20
> and L1.col21 = L2.col21
> and L1.col22 = L2.col22
> and L1.col23 = L2.col23
> ) order by id;
>
> With a larger mem_limit (or do not set mem_limit), above query return 0 which is correct.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org