You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/08/13 09:12:00 UTC
[jira] [Created] (IMPALA-14307) invalidateTable could add table to old Db object while concurrent reset is running
Quanlong Huang created IMPALA-14307:
---------------------------------------
Summary: invalidateTable could add table to old Db object while concurrent reset is running
Key: IMPALA-14307
URL: https://issues.apache.org/jira/browse/IMPALA-14307
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Saw a failure in TestConcurrentDdls.test_ddls_with_invalidate_metadata:
{code:python}
custom_cluster/test_concurrent_ddls.py:50: in test_ddls_with_invalidate_metadata
self._run_ddls_with_invalidation(unique_database, sync_ddl=False)
custom_cluster/test_concurrent_ddls.py:179: in _run_ddls_with_invalidation
worker[i].get(timeout=100)
/data/jenkins/workspace/impala-asf-master-core-ubsan-arm/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/multiprocessing/pool.py:572: in get
raise self._value
E AssertionError: Query b148f272b5821772:ace4369900000000 failed:
E AnalysisException: Table does not exist: test_ddls_with_invalidate_metadata_9525e717.test_6
E
E
E assert <bound method type.is_acceptable_error of <class 'test_concurrent_ddls.TestConcurrentDdls'>>('Query b148f272b5821772:ace4369900000000 failed:\nAnalysisException: Table does not exist: test_ddls_with_invalidate_metadata_9525e717.test_6\n\n', False)
E + where <bound method type.is_acceptable_error of <class 'test_concurrent_ddls.TestConcurrentDdls'>> = <test_concurrent_ddls.TestConcurrentDdls object at 0xffff1ff7d810>.is_acceptable_error{code}
The test runs in legacy catalog mode. Coordinator logs show that it receives a deletion update on the table before analying the REFRESH statement on it:
{noformat}
I20250811 16:22:58.429529 3608823 ImpaladCatalog.java:227] Deleting: TABLE:test_ddls_with_invalidate_metadata_9525e717.test_6 version: 8523 size: 97
...
I20250811 16:22:58.540377 3609121 Frontend.java:2398] b148f272b5821772:ace4369900000000] Analyzing query: refresh test_ddls_with_invalidate_metadata_9525e717.test_6 db: default
...
I20250811 16:22:58.542122 3609121 jni-util.cc:321] b148f272b5821772:ace4369900000000] org.apache.impala.common.AnalysisException: Table does not exist: test_ddls_with_invalidate_metadata_9525e717.test_6
at org.apache.impala.analysis.ResetMetadataStmt.analyze(ResetMetadataStmt.java:173)
at org.apache.impala.analysis.AnalysisContext$AnalysisDriverImpl.analyze(AnalysisContext.java:576)
at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:496)
at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2938)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2505)
at org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2374)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2063)
at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:176){noformat}
Catalogd logs show that invalidateTable triggered by alterTableOrViewRename() is running concurrently with a catalog reset (i.e. global INVALIDATE METADATA):
{noformat}
I20250811 16:22:58.129398 3609170 JniUtil.java:167] f548d67b72570890:626780e200000000] execDdl request: ALTER_TABLE test_ddls_with_invalidate_metadata_9525e717.test_6_2 RENAME_TABLE issued by jenkins
I20250811 16:22:58.144670 3608935 JniUtil.java:167] 1d4918bff853c0ec:c5646e4a00000000] resetMetadata request: INVALIDATE ALL issued by jenkins
I20250811 16:22:58.173722 3608935 CatalogServiceCatalog.java:2339] 1d4918bff853c0ec:c5646e4a00000000] Loaded table list (12) and functions (0) for database: test_ddls_with_invalidate_metadata_9525e717. Fetch duration: 2 ms.
I20250811 16:22:58.177870 3609170 CatalogServiceCatalog.java:3283] f548d67b72570890:626780e200000000] Invalidating table metadata: test_ddls_with_invalidate_metadata_9525e717.test_6
I20250811 16:22:58.178402 3608935 JniUtil.java:178] 1d4918bff853c0ec:c5646e4a00000000] Finished resetMetadata request: INVALIDATE ALL issued by jenkins. Time spent: 34ms
I20250811 16:22:58.181602 3609170 Table.java:246] f548d67b72570890:626780e200000000] createEventId_ for table: test_ddls_with_invalidate_metadata_9525e717.test_6 set to: -1
I20250811 16:22:58.181922 3609170 JniUtil.java:178] f548d67b72570890:626780e200000000] Finished execDdl request: ALTER_TABLE test_ddls_with_invalidate_metadata_9525e717.test_6_2 RENAME_TABLE issued by jenkins. Time spent: 52ms
{noformat}
Then the catalog update collector thread saw the table disappears and sent the deletion update:
{noformat}
I20250811 16:22:58.247798 3608258 JniUtil.java:167] getCatalogDelta request: Getting catalog delta from version 8517
I20250811 16:22:58.250487 3608258 catalog-server.cc:1615] Collected deletion: 1:TABLE:test_ddls_with_invalidate_metadata_9525e717.test_6, version=8523, original size=97, compressed size=95
I20250811 16:22:58.250890 3608258 catalog-server.cc:1615] Collected update: 1:CATALOG_SERVICE_ID, version=8532, original size=60, compressed size=58
I20250811 16:22:58.250959 3608258 JniUtil.java:178] Finished getCatalogDelta request: Getting catalog delta from version 8517. Time spent: 3ms{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org