You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/12/02 03:45:00 UTC
[jira] [Commented] (IMPALA-14582) test_show_create_table_with_stats is flaky
[ https://issues.apache.org/jira/browse/IMPALA-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18042069#comment-18042069 ]
Quanlong Huang commented on IMPALA-14582:
-----------------------------------------
The coordinator logs of that statement:
{noformat}
I20251201 19:28:07.531167 72684 Frontend.java:2445] ea4e44b926dd9971:e0063c5a00000000] Analyzing query: show create table test_show_create_table_with_stats_5f4ae381.t_part_stats WITH STATS db: default
...
W20251201 19:28:07.548882 72684 CatalogdMetaProvider.java:531] ea4e44b926dd9971:e0063c5a00000000] Catalog object TCatalogObject(type:TABLE, catalog_version:50379, table:TTable(db_name:test_show_create_table_with_stats_5f4ae381, tbl_name:t_part_stats)) changed version from 50379 to 50381 while fetching metadata
I20251201 19:28:07.549014 72684 jni-util.cc:321] ea4e44b926dd9971:e0063c5a00000000] org.apache.impala.catalog.local.InconsistentMetadataFetchException: Catalog object TCatalogObject(type:TABLE, catalog_version:50379, table:TTable(db_name:test_show_create_table_with_stats_5f4ae381, tbl_name:t_part_stats)) changed version between accesses.
at org.apache.impala.catalog.local.CatalogdMetaProvider.sendRequest(CatalogdMetaProvider.java:535)
at org.apache.impala.catalog.local.CatalogdMetaProvider.loadPartitionsFromCatalogd(CatalogdMetaProvider.java:1121)
at org.apache.impala.catalog.local.CatalogdMetaProvider.loadPartitionsByRefs(CatalogdMetaProvider.java:1039)
at org.apache.impala.catalog.local.LocalFsTable.loadPartitions(LocalFsTable.java:473)
at org.apache.impala.analysis.ToSqlUtils.appendPartitionStatements(ToSqlUtils.java:770)
at org.apache.impala.analysis.ToSqlUtils.getCreateTableWithStatsSql(ToSqlUtils.java:678)
at org.apache.impala.service.JniFrontend.showCreateTable(JniFrontend.java:613){noformat}
Coordinator should catch InconsistentMetadataFetchException and retry.
On the other hand, I'm curious why the table changed in catalogd. Checked the catalogd logs to see how the table version changes from 50379 to 50381, I found it's due to HMS event processing:
{noformat}
I20251201 19:28:07.541288 150 CatalogServiceCatalog.java:1382] Could not find version 50361 in in-flight event list of table test_show_create_table_with_stats_5f4ae381.t_part_stats
I20251201 19:28:07.541352 150 MetastoreEvents.java:936] EventId: 40529 EventType: ALTER_TABLE Target: test_show_create_table_with_stats_5f4ae381.t_part_stats. Skipping reloading of file metadata for table test_show_create_table_with_stats_5f4ae381.t_part_stats since SD and whitelistedTblProperties has not changed.
I20251201 19:28:07.541396 150 CatalogServiceCatalog.java:3195] Refreshing table metadata: test_show_create_table_with_stats_5f4ae381.t_part_stats
I20251201 19:28:07.544323 150 HdfsTable.java:1256] Reloading metadata for table definition and all partition(s) of test_show_create_table_with_stats_5f4ae381.t_part_stats (ALTER_TABLE event 40529)
I20251201 19:28:07.544654 150 HdfsTable.java:1923] Loaded 3 columns from HMS. Actual columns: 3
I20251201 19:28:07.548062 150 HdfsTable.java:3003] Load Valid Write Id List Done. Time taken: 1.322us
I20251201 19:28:07.548465 150 HdfsTable.java:1305] Incrementally loaded table metadata for: test_show_create_table_with_stats_5f4ae381.t_part_stats
I20251201 19:28:07.548533 150 HdfsTable.java:3095] Setting the catalog version of HdfsTable@c0afb98d test_show_create_table_with_stats_5f4ae381.t_part_stats to 50381{noformat}
Operations on this table are all done in Impala. I think it's a bug that ALTER_TABLE event 40529 is not treated as a self-event.
[~hemanth619], [~VenuReddy] Do you have time to take a look? We might need to fire a separate JIRA for the self-event detection bug.
> test_show_create_table_with_stats is flaky
> ------------------------------------------
>
> Key: IMPALA-14582
> URL: https://issues.apache.org/jira/browse/IMPALA-14582
> Project: IMPALA
> Issue Type: Bug
> Components: Test
> Affects Versions: Impala 5.0.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Labels: broken-build
> Attachments: catalogd.6f230ec72740.impala.log.INFO.20251201-183755.1.gz
>
>
> test_show_create_table_with_stats is flaky can be flaky due to inconsistent metadata between after table queries in test file.
> It recently failed in [https://jenkins.impala.io/job/ubuntu-20.04-dockerised-tests/4296/testReport/junit/metadata.test_show_create_table/TestShowCreateTable/test_show_create_table_with_stats__table_format__text_none_____protocol__hs2__/]
>
> {code:java}
> Stacktrace
> metadata/test_show_create_table.py:106: in test_show_create_table_with_stats
> self.__run_show_create_table_with_stats_test_case(
> metadata/test_show_create_table.py:249: in __run_show_create_table_with_stats_test_case
> result = self.__exec(test_case.show_create_table_sql + " WITH STATS")
> metadata/test_show_create_table.py:263: in __exec
> return self.execute_query_expect_success(self.client, sql_str)
> common/impala_test_suite.py:1126: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:1136: in execute_query_expect_success
> result = cls.__execute_query(impalad_client, query, query_options, user)
> common/impala_test_suite.py:1299: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:687: in execute
> cursor.execute(sql_stmt, configuration=self.__query_options)
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:391: in execute
> self.execute_async(operation, parameters=parameters,
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:443: in execute_async
> self._execute_async(op)
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:462: in _execute_async
> operation_fn()
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:438: in op
> op = self.session.execute(self._last_operation_string,
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:1324: in execute
> return self._operation('ExecuteStatement', req, False)
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:1244: in _operation
> resp = self._rpc(kind, request, safe_to_retry)
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:1181: in _rpc
> err_if_rpc_not_ok(response)
> ../infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/impala/hiveserver2.py:867: in err_if_rpc_not_ok
> raise HiveServer2Error(resp.status.errorMessage)
> E impala.error.HiveServer2Error: Query ea4e44b926dd9971:e0063c5a00000000 failed:
> E InconsistentMetadataFetchException: Catalog object TCatalogObject(type:TABLE, catalog_version:50379, table:TTable(db_name:test_show_create_table_with_stats_5f4ae381, tbl_name:t_part_stats)) changed version between accesses.
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org