You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2022/05/23 16:20:00 UTC
[jira] [Commented] (IMPALA-11249) Precommits are not running strict_hs2_protocol=True configurations for shell tests
[ https://issues.apache.org/jira/browse/IMPALA-11249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541031#comment-17541031 ]
Joe McDonnell commented on IMPALA-11249:
----------------------------------------
Confirmed that the add_test_dimensions() not calling the superclass is the problem. Here is an example run:
{noformat}
test_bash_cmd_timing[table_format_and_file_extension: ('parquet', '.parq') | strict_hs2_protocol: False | protocol: beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0}]{noformat}
This has "table_format_and_file_extension" as a test dimension, but that is only ever used in test_insert.py::TestInsertFileExtension. Somehow it is getting mixed up about the dimensions. As it happens, test_insert.py::TestInsertFileExtension also does not call the superclass from add_test_dimensions().
We should fix all the locations that we can find.
> Precommits are not running strict_hs2_protocol=True configurations for shell tests
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-11249
> URL: https://issues.apache.org/jira/browse/IMPALA-11249
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Affects Versions: Impala 4.1.0
> Reporter: Joe McDonnell
> Priority: Major
>
> When debugging a shell test failure, I noticed that the set of tests running in precommit is different from what happens when I run in my development environment. In precommit, none of the strict_hs2_protocol=True shell tests run. I can't figure out why it behaves that way from the code, given that we enable this test dimension:
> {noformat}
> def add_test_dimensions(cls):
> # Run with both beeswax and HS2 to ensure that behaviour is the same.
> cls.ImpalaTestMatrix.add_dimension(create_client_protocol_dimension())
> cls.ImpalaTestMatrix.add_dimension(create_client_protocol_strict_dimension())
> cls.ImpalaTestMatrix.add_constraint(lambda v:
> v.get_value('protocol') != 'beeswax' or not v.get_value('strict_hs2_protocol')){noformat}
> And create_client_protocol_strict_dimension() should produce both False and True:
> {noformat}
> def create_client_protocol_strict_dimension():
> # only support strict dimensions if the file system is HDFS, since that is
> # where the hive cluster is run.
> if IS_HDFS:
> return ImpalaTestDimension('strict_hs2_protocol', False, True)
> else:
> return create_client_protocol_no_strict_dimension(){noformat}
> When I run locally, those tests run (and there are some failures to fix).
> Example precommit job: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16296/testReport/shell.test_shell_interactive/TestImpalaShellInteractive/
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org