You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2026/02/02 23:34:00 UTC

[jira] [Commented] (IMPALA-12191) Add hardware and OS details to runtime profile

    [ https://issues.apache.org/jira/browse/IMPALA-12191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18056068#comment-18056068 ] 

Joe McDonnell commented on IMPALA-12191:
----------------------------------------

After thinking about this for a while, I want to rework this Jira. Here are my thoughts:
 # We want information for all the backends used by the query, not just one node. That means we need to be collecting and transmitting the information from each node. The information in the profile should be a summary of all the nodes used by this query.
 # The list of desired information could be very long, so I think we should break this into pieces. This Jira should focus on the main code to collect and convey information into the profile, but it should limit itself to a couple interesting things (e.g. CPU type, CPU count, OS). We can add more pieces of information in subsequent patches.
 # The profile information should be succinct. If we are describing CPUs, we can summarize multiple nodes by giving a count of each of the configurations. e.g. "12th Gen Intel(R) Core(TM) i9-12900K (15), 12th Gen Intel(R) Core(TM) i9-12700K (1)" or something like that.
 # To make it easier to summarize across the cluster, it is better to keep the information structured. We don't want to convey a complicated string summary of dozens of things. Instead, each piece can be its own field.

I think a good way to convey this information is via the BackendDescriptorPB protobuf structure that is sent by each backend when registering with the statestore. This already conveys various other information about the backend (e.g. ip address, admission control memory, etc). The fields are currently set in ImpalaServer::BuildLocalBackendDescriptorInternal(), and the information is automatically conveyed via the statestore to the coordinator. The code on the coordinator to combine information about multiple backends would be associated with the ExecutorGroup.

> Add hardware and OS details to runtime profile
> ----------------------------------------------
>
>                 Key: IMPALA-12191
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12191
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: David Rorke
>            Assignee: Arnab Karmakar
>            Priority: Major
>              Labels: ramp-up
>
> The runtime profiles are currently lacking any details about the hardware the query ran on (CPU model and core count, cache sizes, etc) OS versions, etc which may all be relevant when analyzing performance issues, comparing performance metrics across different profiles, etc.
> We should add relevant hardware and OS details to the profile.  The information currently displayed at the root Impalad web UI page (Hardware Info, OS Info) would be a good starting point.
> IMPALA-12118 is also relevant if we want to cover ARM processors.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org