You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/05/25 03:38:00 UTC
[jira] [Commented] (IMPALA-8822) No metadata loading information in query profile when Catalog V2 enabled
[ https://issues.apache.org/jira/browse/IMPALA-8822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541807#comment-17541807 ]
Quanlong Huang commented on IMPALA-8822:
----------------------------------------
The timeline event of "Metadata of all xxx tables cached" could also be misleading. I've seen a profile like this:
{code}
Query Compilation
Metadata of all 3 tables cached: 138毫秒 (138057174)
Analysis finished: 140毫秒 (140132150)
Authorization finished (ranger): 140毫秒 (140721790)
Value transfer graph computed: 140毫秒 (140997116)
Single node plan created: 18.42秒 (18417767827)
Runtime filters computed: 18.42秒 (18417974098)
Distributed plan created: 18.42秒 (18418001039)
Lineage info computed: 18.42秒 (18418113003)
Planning finished: 18.51秒 (18512506927)
Query Timeline
Query submitted: 54.33微秒 (54330)
Planning finished: 18.55秒 (18555661399)
Submit for admission: 18.57秒 (18567161277)
Completed admission: 18.59秒 (18593506985)
Ready to start on 4 backends: 18.59秒 (18595877367)
All 4 execution backends (12 fragment instances) started: 18.62秒 (18622787680)
Rows available: 19.36秒 (19365809384)
First row fetched: 19.37秒 (19370149673)
Last row fetched: 19.37秒 (19370784550)
Released admission control resources: 19.37秒 (19372909626)
Unregister query: 19.38秒 (19375021381)
Frontend
- CatalogFetch.ColumnStats.Hits: 0 (0)
- CatalogFetch.ColumnStats.Misses: 95 (95)
- CatalogFetch.ColumnStats.Requests: 95 (95)
- CatalogFetch.ColumnStats.Time: 0毫秒 (0)
- CatalogFetch.Config.Hits: 0 (0)
- CatalogFetch.Config.Misses: 1 (1)
- CatalogFetch.Config.Requests: 1 (1)
- CatalogFetch.Config.Time: 1毫秒 (1)
- CatalogFetch.DatabaseList.Hits: 1 (1)
- CatalogFetch.DatabaseList.Misses: 0 (0)
- CatalogFetch.DatabaseList.Requests: 1 (1)
- CatalogFetch.DatabaseList.Time: 0毫秒 (0)
- CatalogFetch.PartitionLists.Hits: 0 (0)
- CatalogFetch.PartitionLists.Misses: 2 (2)
- CatalogFetch.PartitionLists.Requests: 2 (2)
- CatalogFetch.PartitionLists.Time: 58毫秒 (58)
- CatalogFetch.Partitions.Hits: 22,914 (22914)
- CatalogFetch.Partitions.Misses: 11,457 (11457)
- CatalogFetch.Partitions.Requests: 34,371 (34371)
- CatalogFetch.Partitions.Time: 17.81秒 (17811)
- CatalogFetch.RPCs.Bytes: 509.9 兆字节 (534706086)
- CatalogFetch.RPCs.Requests: 10 (10)
- CatalogFetch.RPCs.Time: 12.45秒 (12448)
- CatalogFetch.StorageLoad.Time: 0毫秒 (0)
- CatalogFetch.TableList.Hits: 3 (3)
- CatalogFetch.TableList.Misses: 0 (0)
- CatalogFetch.TableList.Requests: 3 (3)
- CatalogFetch.TableList.Time: 0毫秒 (0)
- CatalogFetch.Tables.Hits: 0 (0)
- CatalogFetch.Tables.Misses: 3 (3)
- CatalogFetch.Tables.Requests: 3 (3)
- CatalogFetch.Tables.Time: 2毫秒 (2)
- InactiveTotalTime: 0纳秒 (0)
- TotalTime: 0纳秒 (0)
{code}
The time of CatalogFetch.Partitions.Time should be added to the metadata loading event instead of "Single node plan created".
> No metadata loading information in query profile when Catalog V2 enabled
> ------------------------------------------------------------------------
>
> Key: IMPALA-8822
> URL: https://issues.apache.org/jira/browse/IMPALA-8822
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.2.0
> Reporter: Yongzhi Chen
> Assignee: Quanlong Huang
> Priority: Major
>
> When local catalog is enabled, we can no longer find table loading information in query profile even just after invalidate metadata for the tables.
> In Catalog V1, you can find the table loading information in query profile like following:
> Query Compilation: 4s401ms
> - Metadata load started: 661.084us (661.084us)
> - Metadata load finished. loaded-tables=1/1 load-requests=1 catalog-updates=3: 3s819ms (3s819ms)
> - Analysis finished: 3s820ms (763.979us)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org