[jira] [Commented] (IMPALA-12118) Consider using pytorch cpuinfo library for CPU detection / logging

["Joe McDonnell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723629#comment-17723629 ] 

Joe McDonnell commented on IMPALA-12118:
----------------------------------------

The existing code does not detect information about ARM processors. This is what we print on Amazon ARM instances:
{noformat}
I0515 22:07:25.198132 207691 init.cc:419] Cpu Info:
  Model: unknown
  Cores: 16
  Max Possible Cores: 16
  L1 Cache: 0 (Line: 64.00 B)
  L2 Cache: 0 (Line: 0)
  L3 Cache: 0 (Line: 0)
  Hardware Supports:
  Numa Nodes: 1
  Numa Nodes of Cores: 0->0 | 1->0 | 2->0 | 3->0 | 4->0 | 5->0 | 6->0 | 7->0 | 8->0 | 9->0 | 10->0 | 11->0 | 12->0 | 13->0 | 14->0 | 15->0 |{noformat}

> Consider using pytorch cpuinfo library for CPU detection / logging
> ------------------------------------------------------------------
>
>                 Key: IMPALA-12118
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12118
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 4.3.0
>            Reporter: Joe McDonnell
>            Priority: Major
>
> The code in be/src/util/cpu-info.h/.cc provides information about the CPU that we run on, along with NUMA configurations, cache sizes, etc. 
> Pytorch's cpuinfo package seems to cover most of what we want to do across a much broader set of processors / architectures. It has a compatible license. See [https://github.com/pytorch/cpuinfo]
> We should see if this is useful for our cpu detection code.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org