[jira] [Created] (IMPALA-15150) test_runtime_filter_annotation_with_hbo_cardinality is flaky

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-15150:
---------------------------------------

             Summary: test_runtime_filter_annotation_with_hbo_cardinality is flaky
                 Key: IMPALA-15150
                 URL: https://issues.apache.org/jira/browse/IMPALA-15150
             Project: IMPALA
          Issue Type: Bug
            Reporter: Quanlong Huang
            Assignee: Quanlong Huang


test_runtime_filter_annotation_with_hbo_cardinality could fail due to the HBO stats on "01:SCAN HDFS [tpch_parquet.orders]" is missing:
{noformat}
AssertionError: Max Per-Host Resource Reservation: Memory=28.88MB Threads=6
  Per-Host Resource Estimates: Memory=409MB
  
  PLAN-ROOT SINK
  |
  07:AGGREGATE [FINALIZE]
  |  output: count:merge(l_orderkey)
  |  row-size=8B cardinality=1
  |
  06:EXCHANGE [UNPARTITIONED]
  |
  03:AGGREGATE
  |  output: count(l_orderkey)
  |  row-size=8B cardinality=3
  |
  02:HASH JOIN [INNER JOIN, PARTITIONED]
  |  hash predicates: l_orderkey = o_orderkey
  |  runtime filters: RF000 <- o_orderkey
  |  row-size=62B cardinality=10.95K
  |
  |--05:EXCHANGE [HASH(o_orderkey)]
  |  |
  |  01:SCAN HDFS [tpch_parquet.orders]
  |     HDFS partitions=1/1 files=2 size=54.21MB
  |     predicates: o_custkey < 1000
  |     row-size=16B cardinality=150.00K
  |
  04:EXCHANGE [HASH(l_orderkey)]
  |
  00:SCAN HDFS [tpch_parquet.lineitem]
     HDFS partitions=1/1 files=3 size=193.97MB
     predicates: l_discount <= 0.07, l_discount >= 0.05, l_quantity < 24, l_shipdate < '1995-01-01', l_shipdate >= '1994-01-01'
     runtime filters: RF000 -> l_orderkey
     row-size=46B cardinality=114.16K (from HBO)
assert 'cardinality=37.88K(filtered from 114.16K from HBO)' in "Max Per-Host Resource Reservation: Memory=28.88MB Threads=6\nPer-Host Resource Estimates: Memory=409MB\n\nPLAN-ROOT S...', l_shipdate >= '1994-01-01'\n   runtime filters: RF000 -> l_orderkey\n   row-size=46B cardinality=114.16K (from HBO)"
 +  where "Max Per-Host Resource Reservation: Memory=28.88MB Threads=6\nPer-Host Resource Estimates: Memory=409MB\n\nPLAN-ROOT S...', l_shipdate >= '1994-01-01'\n   runtime filters: RF000 -> l_orderkey\n   row-size=46B cardinality=114.16K (from HBO)" = <built-in method join of str object at 0x882a98>(['Max Per-Host Resource Reservation: Memory=28.88MB Threads=6', 'Per-Host Resource Estimates: Memory=409MB', '', 'PLAN-ROOT SINK', '|', '07:AGGREGATE [FINALIZE]', ...])
 +    where <built-in method join of str object at 0x882a98> = '\n'.join
 +    and   ['Max Per-Host Resource Reservation: Memory=28.88MB Threads=6', 'Per-Host Resource Estimates: Memory=409MB', '', 'PLAN-ROOT SINK', '|', '07:AGGREGATE [FINALIZE]', ...] = <tests.common.impala_connection.ImpylaHS2ResultSet object at 0x7f9029583950>.data{noformat}
The cause is that the test doesn't wait for 1s for previous queries to flush HBO stats which is asychronous. The other HBO tests all have such a sleep.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)