You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Zoltán Borók-Nagy (Jira)" <ji...@apache.org> on 2020/11/18 08:54:00 UTC
[jira] [Resolved] (IMPALA-10335) Iceberg tests failing with Failed
to load metadata for table: 'iceberg_partitioned'
[ https://issues.apache.org/jira/browse/IMPALA-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zoltán Borók-Nagy resolved IMPALA-10335.
----------------------------------------
Resolution: Not A Bug
The tests failed because they used old snapshot data.
We recently switched to the Iceberg Input/Output format classes: [https://gerrit.cloudera.org/#/c/16707/]
The snapshot data had the old classes that's why Impala couldn't recognize the tables.
> Iceberg tests failing with Failed to load metadata for table: 'iceberg_partitioned'
> ------------------------------------------------------------------------------------
>
> Key: IMPALA-10335
> URL: https://issues.apache.org/jira/browse/IMPALA-10335
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Tim Armstrong
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: broken-build, flaky, iceberg
>
> Failed to load metadata for table: 'iceberg_partitioned'
> Affected tests
> {noformat}
> query_test.test_iceberg.TestIcebergTable.test_iceberg_negative[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:FAIL@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:FAIL@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:FAIL@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> query_test.test_scanners.TestIceberg.test_iceberg_profile[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:FAIL@0.5', 'exec_single_node_rows_threshold': 0} |
> {noformat}
> Example error:
> {noformat}
> query_test.test_scanners.TestIceberg.test_iceberg_query[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from pytest)
> Failing for the past 1 build (Since Failed#672 )
> Took 0.25 sec.
> add description
> Error Message
> query_test/test_scanners.py:357: in test_iceberg_query self.run_test_case('QueryTest/iceberg-query', vector) common/impala_test_suite.py:662: in run_test_case result = exec_fn(query, user=test_section.get('USER', '').strip() or None) common/impala_test_suite.py:600: in __exec_in_impala result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:920: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:205: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:187: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:363: in __execute_query handle = self.execute_query_async(query_string, user=user) beeswax/impala_beeswax.py:357: in execute_query_async handle = self.__do_rpc(lambda: self.imp_service.query(query,)) beeswax/impala_beeswax.py:520: in __do_rpc raise ImpalaBeeswaxException(self.__build_error_message(b), b) E ImpalaBeeswaxException: ImpalaBeeswaxException: E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> E MESSAGE: AnalysisException: Failed to load metadata for table: 'iceberg_partitioned' E CAUSED BY: TableLoadingException: Unrecognized table type for table: functional_parquet.iceberg_partitioned
> Stacktrace
> query_test/test_scanners.py:357: in test_iceberg_query
> self.run_test_case('QueryTest/iceberg-query', vector)
> common/impala_test_suite.py:662: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:600: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:920: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:363: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:357: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:520: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: AnalysisException: Failed to load metadata for table: 'iceberg_partitioned'
> E CAUSED BY: TableLoadingException: Unrecognized table type for table: functional_parquet.iceberg_partitioned
> Standard Error
> SET client_identifier=query_test/test_scanners.py::TestIceberg::()::test_iceberg_query[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action':None;'exec_single_node_rows_threshold';
> -- executing against localhost:21000
> use functional_parquet;
> -- 2020-11-13 12:52:58,534 INFO MainThread: Started query 6c4f1f4b3e4405f9:22caf6de00000000
> SET client_identifier=query_test/test_scanners.py::TestIceberg::()::test_iceberg_query[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'debug_action':None;'exec_single_node_rows_threshold';
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- 2020-11-13 12:52:58,537 INFO MainThread: Loading query test file: /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
> -- 2020-11-13 12:52:58,540 INFO MainThread: Starting new HTTP connection (1): localhost
> -- executing against localhost:21000
> SELECT count(*) from iceberg_partitioned;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org