You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/11/25 05:33:00 UTC
[jira] [Updated] (IMPALA-11747) TestAcid.test_acid_compaction is flaky
[ https://issues.apache.org/jira/browse/IMPALA-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-11747:
------------------------------------
Attachment: catalogd.56643cfb8723.invalid-user.log.INFO.20221122-052015.1.gz
hive-server2.log.gz
hive-metastore.log.gz
impalad_coord_exec-1_impalad.f6d753b6d706.invalid-user.log.INFO.20221122-052017.1.gz
impalad_coord_exec-2_impalad.4d15644b3cc3.invalid-user.log.INFO.20221122-052019.1.gz
> TestAcid.test_acid_compaction is flaky
> --------------------------------------
>
> Key: IMPALA-11747
> URL: https://issues.apache.org/jira/browse/IMPALA-11747
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Priority: Major
> Attachments: catalogd.56643cfb8723.invalid-user.log.INFO.20221122-052015.1.gz, hive-metastore.log.gz, hive-server2.log.gz, impalad_coord_exec-1_impalad.f6d753b6d706.invalid-user.log.INFO.20221122-052017.1.gz, impalad_coord_exec-2_impalad.4d15644b3cc3.invalid-user.log.INFO.20221122-052019.1.gz
>
>
> TestAcid.test_acid_compaction fails on an unrelated patch:
> https://gerrit.cloudera.org/c/19055/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/6664
> {noformat}
> query_test/test_acid.py:62: in test_acid_compaction
> self.run_test_case('QueryTest/acid-compaction', vector, use_db=unique_database)
> common/impala_test_suite.py:742: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:578: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:469: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E row_regex:'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/base_0000003_v\d+/000000_0','\d+B','' != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000001_0000001_0000/000000_0','2B',''
> E None != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000002_0000002_0000/000000_0','2B',''
> E None != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000003_0000003_0000/000000_0','2B',''
> E Number of rows returned (expected vs actual): 1 != 3
> {noformat}
> stderr:
> {noformat}
> SET client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> -- connecting to: localhost:21000
> -- connecting to localhost:21050 with impyla
> -- 2022-11-22 06:06:10,804 INFO MainThread: Closing active operation
> -- connecting to localhost:28000 with impyla
> -- 2022-11-22 06:06:10,839 INFO MainThread: Closing active operation
> SET client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_acid_compaction_5c682e9f` CASCADE;
> -- 2022-11-22 06:06:10,858 INFO MainThread: Started query cc4f4280c721d09e:45d406dc00000000
> SET client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_acid_compaction_5c682e9f`;
> -- 2022-11-22 06:06:14,245 INFO MainThread: Started query c246e80a313b2897:bc7ad62200000000
> -- 2022-11-22 06:06:14,308 INFO MainThread: Created database "test_acid_compaction_5c682e9f" for test ID "query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol: beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]"
> SET client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> -- executing against localhost:21000
> use test_acid_compaction_5c682e9f;
> -- 2022-11-22 06:06:14,313 INFO MainThread: Started query 944cc29ee418013f:3684f69900000000
> SET client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET test_replan=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- 2022-11-22 06:06:14,317 INFO MainThread: Loading query test file: /home/ubuntu/Impala/testdata/workloads/functional-query/queries/QueryTest/acid-compaction.test
> -- connecting to localhost:11050 with impyla
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120e50>
> -- 2022-11-22 06:06:16,810 INFO MainThread: Closing active operation
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:16,851 INFO MainThread: Closing active operation
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:38,018 INFO MainThread: Closing active operation
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:39,238 INFO MainThread: Closing active operation
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:40,595 INFO MainThread: Closing active operation
> -- closing connection to: localhost:11050
> -- executing against localhost:21000
> invalidate metadata tt;
> -- 2022-11-22 06:06:40,652 INFO MainThread: Started query 524a12a9f7e2a52b:cabf7f6800000000
> -- executing against localhost:21000
> select * from tt;
> -- 2022-11-22 06:06:40,738 INFO MainThread: Started query 2c4445f501722473:205c850600000000
> -- connecting to localhost:11050 with impyla
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120550>
> -- 2022-11-22 06:06:40,912 INFO MainThread: Closing active operation
> -- closing query for operation handle: <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120650>
> -- 2022-11-22 06:07:35,112 INFO MainThread: Closing active operation
> -- closing connection to: localhost:11050
> -- executing against localhost:21000
> refresh tt;
> -- 2022-11-22 06:07:35,121 INFO MainThread: Started query e14cbd422da3bc24:c48a680b00000000
> -- executing against localhost:21000
> select * from tt;
> -- 2022-11-22 06:07:35,194 INFO MainThread: Started query 0147f79f0d7cdd37:6cd2423b00000000
> -- executing against localhost:21000
> show files in tt;
> -- 2022-11-22 06:07:35,445 INFO MainThread: Started query a6494ba76809ffa6:e0ccc4f000000000
> -- 2022-11-22 06:07:35,474 ERROR MainThread: Comparing QueryTestResults (expected vs actual):
> row_regex:'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/base_0000003_v\d+/000000_0','\d+B','' != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000001_0000001_0000/000000_0','2B',''
> None != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000002_0000002_0000/000000_0','2B',''
> None != 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000003_0000003_0000/000000_0','2B',''
> Number of rows returned (expected vs actual): 1 != 3
> {noformat}
> It seems the major compaction didn't finish (or fail) so the test still saw the original files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org