You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Arnab Karmakar (Jira)" <ji...@apache.org> on 2026/10/07 18:26:00 UTC
[jira] [Commented] (IMPALA-15513) TestHBO.test_except_cardinality seems flaky
[ https://issues.apache.org/jira/browse/IMPALA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18124646#comment-18124646 ]
Arnab Karmakar commented on IMPALA-15513:
-----------------------------------------
Hi [~stigahuang], assigned this this ticket to you since you added the failed test in IMPALA-14601 and thus may know much more about it. Please re-assign as you see appropriate. Thanks!
> TestHBO.test_except_cardinality seems flaky
> -------------------------------------------
>
> Key: IMPALA-15513
> URL: https://issues.apache.org/jira/browse/IMPALA-15513
> Project: IMPALA
> Issue Type: Bug
> Reporter: Arnab Karmakar
> Assignee: Quanlong Huang
> Priority: Major
>
> The TestHBO.test_except_cardinality test is failing with the error “AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except”.
> Stack Trace:
> {code:java}
> query_test/test_hbo.py:441: in test_except_cardinality
> self._run_hbo_explains('QueryTest/hbo-except')
> self = <test_hbo.TestHBO object at 0x7f5c41371350>
> query_test/test_hbo.py:71: in _run_hbo_explains
> assert actual_cardinality_lines == expected_cardinality_lines, (
> E AssertionError: EXPLAIN output mismatch for QueryTest/hbo-except.
> E Expected:
> E # Calcite uses UNION+AGG instead of ANTI JOIN for EXCEPT
> E PLAN-ROOT SINK
> E |
> E 06:EXCHANGE [UNPARTITIONED]
> E |
> E 05:AGGREGATE [FINALIZE]
> E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL)) = 0
> E | row-size=20B cardinality=7.29K (from HBO)
> E |
> E 04:EXCHANGE [HASH(ID)]
> E |
> E 03:AGGREGATE [STREAMING]
> E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | row-size=20B cardinality=7.30K (from HBO)
> E |
> E 00:UNION
> E | row-size=5B cardinality=7.31K (from HBO)
> E |
> E |--02:SCAN HDFS [functional.alltypestiny]
> E | HDFS partitions=4/4 files=4 size=460B
> E | predicates: functional.alltypestiny.string_col != 'test_except'
> E | row-size=17B cardinality=8 (from HBO)
> E |
> E 01:SCAN HDFS [functional.alltypes]
> E HDFS partitions=24/24 files=24 size=478.45KB
> E predicates: functional.alltypes.string_col != 'test_except'
> E row-size=17B cardinality=7.30K (from HBO)
> E
> E
> E Actual:
> E PLAN-ROOT SINK
> E |
> E 06:EXCHANGE [UNPARTITIONED]
> E |
> E 05:AGGREGATE [FINALIZE]
> E | output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | having: count(if($f1 = 0, 1, NULL)) > 0, count(if($f1 = 1, 1, NULL)) = 0
> E | row-size=20B cardinality=7.29K (from HBO)
> E |
> E 04:EXCHANGE [HASH(ID)]
> E |
> E 03:AGGREGATE [STREAMING]
> E | output: count(if($f1 = 0, 1, NULL)), count(if($f1 = 1, 1, NULL))
> E | group by: ID
> E | row-size=20B cardinality=7.31K (from HBO)
> E |
> E 00:UNION
> E | row-size=5B cardinality=7.31K (from HBO)
> E |
> E |--02:SCAN HDFS [functional.alltypestiny]
> E | HDFS partitions=4/4 files=4 size=460B
> E | predicates: functional.alltypestiny.string_col != 'test_except'
> E | row-size=17B cardinality=8 (from HBO)
> E |
> E 01:SCAN HDFS [functional.alltypes]
> E HDFS partitions=24/24 files=24 size=478.45KB
> E predicates: functional.alltypes.string_col != 'test_except'
> E row-size=17B cardinality=7.30K (from HBO)
> E assert ['| row-size...K (from HBO)'] == ['| row-size...K (from HBO)']
> E At index 1 diff: '| row-size=20B cardinality=7.31K (from HBO)' != '| row-size=20B cardinality=7.30K (from HBO)'
> E Full diff:
> E [
> E '| row-size=20B cardinality=7.29K (from HBO)',
> E - '| row-size=20B cardinality=7.30K (from HBO)',
> E ? ^
> E + '| row-size=20B cardinality=7.31K (from HBO)',
> E ? ^
> E '| row-size=5B cardinality=7.31K (from HBO)',
> E '| row-size=17B cardinality=8 (from HBO)',
> E ' row-size=17B cardinality=7.30K (from HBO)',
> E ]
> actual_cardinality_lines = ['| row-size=20B cardinality=7.29K (from HBO)', '| row-size=20B cardinality=7.31K (from HBO)', '| row-size=5B cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)', ' row-size=17B cardinality=7.30K (from HBO)']
> actual_plan_lines = ['PLAN-ROOT SINK', '|', '06:EXCHANGE [UNPARTITIONED]', '|', '05:AGGREGATE [FINALIZE]', '| output: count:merge(if($f1 = 0, 1, NULL)), count:merge(if($f1 = 1, 1, NULL))', ...]
> expected_cardinality_lines = ['| row-size=20B cardinality=7.29K (from HBO)', '| row-size=20B cardinality=7.30K (from HBO)', '| row-size=5B cardinality=7.31K (from HBO)', '| row-size=17B cardinality=8 (from HBO)', ' row-size=17B cardinality=7.30K (from HBO)']
> expected_plan_lines = ['# Calcite uses UNION+AGG instead of ANTI JOIN for EXCEPT', 'PLAN-ROOT SINK', '|', '06:EXCHANGE [UNPARTITIONED]', '|', '05:AGGREGATE [FINALIZE]', ...]
> plan_section_name = 'CALCITE_PLANNER_PLAN'
> query = "EXPLAIN select id from functional.alltypes where string_col != 'test_except'\n except\n select id from functional.alltypestiny where string_col != 'test_except'"
> result = <tests.common.impala_connection.ImpylaHS2ResultSet object at 0x7f5c414be7d0>
> section = defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the same query that has been executed\nEXPLAIN select id from function...45KB\n predicates: functional.alltypes.string_col != 'test_except'\n row-size=17B cardinality=7.30K (from HBO)\n"})
> self = <test_hbo.TestHBO object at 0x7f5c41371350>
> test_cases = [defaultdict(<class 'str'>, {'QUERY': "# EXPLAIN the same query that has been executed\nEXPLAIN select id from functio...B\n predicates: functional.alltypes_0.string_col != 'test_except'\n row-size=17B cardinality=7.30K (from HBO)\n"})]
> test_file = 'QueryTest/hbo-except'{code}
> stderr:
> {code:java}
> --------------------------------- Captured Err ---------------------------------
> -- 2026-10-06 19:41:17,897 INFO MainThread: hs2:
> set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,897 INFO MainThread: hs2-http:
> set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2:
> set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2: set_configuration:
> set use_hbo_stats=True;
> set store_hbo_stats=True;
> -- 2026-10-06 19:41:17,899 INFO MainThread: hs2: executing against Impala at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor: True user: None
> select id from functional.alltypes where string_col != 'test_except'
> except
> select id from functional.alltypestiny where string_col != 'test_except'
> ;
> -- 2026-10-06 19:41:17,927 INFO MainThread: af483a6aa5c5b599:4276713500000000: query started
> -- 2026-10-06 19:41:17,935 INFO MainThread: af483a6aa5c5b599:4276713500000000: getting log for operation
> -- 2026-10-06 19:41:17,935 INFO MainThread: af483a6aa5c5b599:4276713500000000: getting runtime profile operation
> -- 2026-10-06 19:41:17,946 INFO MainThread: af483a6aa5c5b599:4276713500000000: closing query for operation
> -- 2026-10-06 19:41:18,947 INFO MainThread: Loading query test file: /data/jenkins/workspace/impala-asf-master-core-calcite/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/hbo-except.test
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2:
> set client_identifier=query_test/test_hbo.py::TestHBO::test_except_cardinality;
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2: set_configuration:
> set use_hbo_stats=True;
> -- 2026-10-06 19:41:18,948 INFO MainThread: hs2: executing against Impala at localhost:21050. session: 5d4a5724266456e8:12e78bdab0d8fc8b main_cursor: True user: None
> EXPLAIN select id from functional.alltypes where string_col != 'test_except'
> except
> select id from functional.alltypestiny where string_col != 'test_except';
> -- 2026-10-06 19:41:18,958 INFO MainThread: 5243e19b35c5c195:dcd6f24900000000: query started
> -- 2026-10-06 19:41:18,958 INFO MainThread: 5243e19b35c5c195:dcd6f24900000000: getting log for operation
> -- 2026-10-06 19:41:18,958 INFO MainThread: 5243e19b35c5c195:dcd6f24900000000: getting runtime profile operation
> -- 2026-10-06 19:41:18,959 INFO MainThread: 5243e19b35c5c195:dcd6f24900000000: closing query for operation{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org