You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/02/26 08:52:00 UTC
[jira] [Updated] (IMPALA-13800) Add catalogd logs for the progress of file metadata loading
[ https://issues.apache.org/jira/browse/IMPALA-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-13800:
------------------------------------
Issue Type: Improvement (was: Bug)
> Add catalogd logs for the progress of file metadata loading
> -----------------------------------------------------------
>
> Key: IMPALA-13800
> URL: https://issues.apache.org/jira/browse/IMPALA-13800
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> When loading file metadata of an Hdfs table, we just have logs when the parallel load starts and finishes:
> {code:java}
> I0224 23:59:51.042261 2189 HdfsTable.java:1366] Fetched partition metadata from the Metastore: default.part_6m
> I0225 00:01:46.838979 2189 ParallelFileMetadataLoader.java:217] Loading file and block metadata for 6000000 paths for table default.part_6m using a thread pool of size 16
> I0225 00:05:38.324651 2189 HdfsTable.java:840] Loaded file and block metadata for default.part_6m partitions: p=0/q=0, p=0/q=1, p=0/q=10, and 5999997 others. Time taken: 4m6s
> W0225 00:05:45.111688 2189 HdfsTable.java:1388] Time taken on loading table default.part_6m exceeded warning threshold. Time: 11m43s{code}
> After fetching the partition metadata from HMS, no other logs except the JVM pause logs are shown. It'd be helpful to add more logs to show the progress.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org