You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Riza Suminto (Jira)" <ji...@apache.org> on 2025/10/13 19:21:00 UTC
[jira] [Resolved] (IMPALA-14447) Metadata loading is not triggered in parallel in local catalog mode
[ https://issues.apache.org/jira/browse/IMPALA-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Riza Suminto resolved IMPALA-14447.
-----------------------------------
Fix Version/s: Impala 5.0.0
Target Version: Impala 5.0.0
Resolution: Fixed
> Metadata loading is not triggered in parallel in local catalog mode
> -------------------------------------------------------------------
>
> Key: IMPALA-14447
> URL: https://issues.apache.org/jira/browse/IMPALA-14447
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Riza Suminto
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> When a query accesses multiple tables that are unloaded, metadata loading of them is triggered sequentially in local catalog mode. The stacktrace of coordinator thread:
> {noformat}
> "Thread-20 [LoadWithCaching for table metadata for default.part_900k_parq1]" #112 prio=5 os_prio=0 tid=0x000000000aa08000 nid=0x27f7 runnable [0x00007fcb9afe5000]
> java.lang.Thread.State: RUNNABLE
> at org.apache.impala.service.FeSupport.NativeGetPartialCatalogObject(Native Method)
> at org.apache.impala.service.FeSupport.GetPartialCatalogObject(FeSupport.java:472)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.sendRequest(CatalogdMetaProvider.java:463)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.access$100(CatalogdMetaProvider.java:209)
> at org.apache.impala.catalog.local.CatalogdMetaProvider$4.call(CatalogdMetaProvider.java:815)
> at org.apache.impala.catalog.local.CatalogdMetaProvider$4.call(CatalogdMetaProvider.java:807)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:601)
> at org.apache.impala.catalog.local.CatalogdMetaProvider.loadTable(CatalogdMetaProvider.java:803)
> at org.apache.impala.catalog.local.LocalTable.loadTableMetadata(LocalTable.java:164)
> at org.apache.impala.catalog.local.LocalTable.load(LocalTable.java:114)
> at org.apache.impala.catalog.local.LocalDb.getTable(LocalDb.java:148)
> at org.apache.impala.analysis.StmtMetadataLoader.getMissingTables(StmtMetadataLoader.java:323)
> at org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:176)
> at org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:145)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2600)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2295)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2032)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:171){noformat}
> For an unloaded table, metadata loading is triggered by the first call on LocalDb.getTable(). The metadata loading of the second unloaded table is triggered after this is done.
> This is a performance regression comparing to the legacy catalog mode where metadata loading on all tables accessed by a query are triggered in parallel.
> CC [~rizaon]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org