You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/02/26 08:52:00 UTC

[jira] [Updated] (IMPALA-13800) Add catalogd logs for the progress of file metadata loading

     [ https://issues.apache.org/jira/browse/IMPALA-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang updated IMPALA-13800:
------------------------------------
    Issue Type: Improvement  (was: Bug)

> Add catalogd logs for the progress of file metadata loading
> -----------------------------------------------------------
>
>                 Key: IMPALA-13800
>                 URL: https://issues.apache.org/jira/browse/IMPALA-13800
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>
> When loading file metadata of an Hdfs table, we just have logs when the parallel load starts and finishes:
> {code:java}
> I0224 23:59:51.042261  2189 HdfsTable.java:1366] Fetched partition metadata from the Metastore: default.part_6m
> I0225 00:01:46.838979  2189 ParallelFileMetadataLoader.java:217] Loading file and block metadata for 6000000 paths for table default.part_6m using a thread pool of size 16
> I0225 00:05:38.324651  2189 HdfsTable.java:840] Loaded file and block metadata for default.part_6m partitions: p=0/q=0, p=0/q=1, p=0/q=10, and 5999997 others. Time taken: 4m6s
> W0225 00:05:45.111688  2189 HdfsTable.java:1388] Time taken on loading table default.part_6m exceeded warning threshold. Time: 11m43s{code}
> After fetching the partition metadata from HMS, no other logs except the JVM pause logs are shown. It'd be helpful to add more logs to show the progress.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org