You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/04/12 11:01:00 UTC
[jira] [Commented] (IMPALA-13487) Capture metrics for memory allocation in execution profile
[ https://issues.apache.org/jira/browse/IMPALA-13487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17943792#comment-17943792 ]
Quanlong Huang commented on IMPALA-13487:
-----------------------------------------
Uploaded an off-cpu flamegraph captured for a tcmalloc contention scenario. [^mt_4_con_4-off-cpu.svg]
The query is a simple scan with selective predicates but materializing large arrays. Ran with MT_DOP=4 and 4 concurrent queries are running. On a machine with a 16 cores CPU. All threads of impalad are tracked here. Looking into the fragment instance threads (impala::QueryState::ExecFInstance), most of the off-cpu time is spent in waiting for locks of TCMalloc:
!image-2025-04-12-18-57-46-354.png|width=871,height=461!
The above patch is able to reveal time spent in these (stuck in tcmalloc).
> Capture metrics for memory allocation in execution profile
> ----------------------------------------------------------
>
> Key: IMPALA-13487
> URL: https://issues.apache.org/jira/browse/IMPALA-13487
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Manish Maheshwari
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: image-2025-04-12-18-57-46-354.png, mt_4_con_4-off-cpu.svg
>
>
> Capture metrics for memory allocation in execution profile to identify queries that are bound on memory allocation and allocation wait times esp during running many concurrent queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org