[jira] [Commented] (IMPALA-9221) Optimize HashRing's map implementation

["Joe McDonnell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541634#comment-17541634 ] 

Joe McDonnell commented on IMPALA-9221:
---------------------------------------

Boost's flat_map seems like a good fit for what we are doing. So, we can try updating be/src/scheduler/hash-ring* to use that, and then do a performance comparison.

> Optimize HashRing's map implementation
> --------------------------------------
>
>                 Key: IMPALA-9221
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9221
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 3.4.0
>            Reporter: Joe McDonnell
>            Priority: Major
>
> The hash ring used for consistent scheduling currently uses a std::map for the hash-to-IpAddr lookup. HashRing is heavy on reads, with writes only happening when executors come and go. There are some cases where we copy the HashRing.
> The standard map uses a large number of small allocations. This hurts cache performance, adds overhead, and also increases the cost of copying the structure. Something like boost's flat_map or Abseil's btree_map is likely to be more efficient.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org