You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/08/23 22:25:00 UTC

[jira] [Commented] (IMPALA-11295) TestParquet.test_multiple_blocks_mt_dop failed by unexpected ranges_complete_list

    [ https://issues.apache.org/jira/browse/IMPALA-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583879#comment-17583879 ] 

ASF subversion and git services commented on IMPALA-11295:
----------------------------------------------------------

Commit 6357534926a24c1ec8e95c37e7b23f5ac29571e6 in impala's branch refs/heads/branch-4.1.1 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=635753492 ]

IMPALA-11295: Deflake TestParquet.test_multiple_blocks_mt_dop

TestParquet.test_multiple_blocks_mt_dop runs a query on 6 scan ranges
using mt_dop=2. It then verifies the sum of ranges read on a backend is
2 (6/3). The test assumes that counters of the 2 instances on the same
host are printed consecutively. However, this is not always true. They
could be interleaving.

This patch makes the test more robust by grouping the counters based on
the host.

Test
 - I can't reproduce the issue locally. But I'm able to run the new test
   100 times without any error.

Change-Id: I16c576c41a212f83dda82a83931ab336a78a41e4
Reviewed-on: http://gerrit.cloudera.org:8080/18533
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/18892
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Quanlong Huang <hu...@gmail.com>


> TestParquet.test_multiple_blocks_mt_dop failed by unexpected ranges_complete_list
> ---------------------------------------------------------------------------------
>
>                 Key: IMPALA-11295
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11295
>             Project: IMPALA
>          Issue Type: Bug
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Critical
>              Labels: flaky
>             Fix For: Impala 4.2.0
>
>         Attachments: test_multiple_blocks_mt_dop.profile.txt
>
>
> Saw this in [https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5677]
> {noformat}
> query_test.test_acid.TestAcid.test_acid_compute_stats[protocol: beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none] (from pytest)
> {noformat}
> *Stacktrace*
> {code:python}
> query_test/test_scanners.py:701: in test_multiple_blocks_mt_dop
>     assert int(ranges_complete_list[i]) + int(ranges_complete_list[i + 1]) == 2
> E   assert (1 + 2) == 2
> E    +  where 1 = int('1')
> E    +  and   2 = int('2')
> {code}
> *Standard Error*
> {code:sql}
> SET mt_dop=2;
> -- executing against localhost:21000
> select count(l_orderkey) from functional_parquet.lineitem_sixblocks;
> -- 2022-05-16 22:44:28,068 INFO     MainThread: Started query d04d0645dcd4ee6f:dc3ba79f00000000
> SET client_identifier=query_test/test_scanners.py::TestParquet::()::test_multiple_blocks_mt_dop[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':True;'abort_on_error':1;'debug_action':'HDFS_SCANNER_;
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org