You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/05/26 23:52:00 UTC
[jira] [Commented] (IMPALA-8737) Patch gperftools to fix O(n)
scaling in PageHeap::AllocLarge()
[ https://issues.apache.org/jira/browse/IMPALA-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17352198#comment-17352198 ]
ASF subversion and git services commented on IMPALA-8737:
---------------------------------------------------------
Commit 822e8373d1f1737865899b80862c2be7b07cc950 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=822e837 ]
IMPALA-8737: Switch to gperftool with O(log n) PageHeap::AllocLarge()
In gperftools 2.5, PageHeap::AllocLarge() has O(n) behavior.
This switches to a patched version of gperftools 2.5 that
changes the behavior to O(log n). This corresponds to
these commits:
https://github.com/gperftools/gperftools/commit/06c9414ec423ffe442c047b2560555f9d5847b1d
https://github.com/gperftools/gperftools/commit/f1d3fe4a21e339a3fd6e4592ee7444484a7b92dc
Testing:
- Core job
Change-Id: I6377f7087111cf10ae35ce102beabcafb505579f
Reviewed-on: http://gerrit.cloudera.org:8080/17484
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Kurt Deschler <kd...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
> Patch gperftools to fix O(n) scaling in PageHeap::AllocLarge()
> --------------------------------------------------------------
>
> Key: IMPALA-8737
> URL: https://issues.apache.org/jira/browse/IMPALA-8737
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Joe McDonnell
> Priority: Major
>
> TCMalloc's PageHeap::AllocLarge() has O( n ) behavior. As the heap gets fragmented, this O( n ) behavior can lead to contention, because the thread executing PageHeap::AllocLarge() is holding a lock. In recent versions of gperftools, this code has been modified to have O(log n) behavior. This could reduce contention significantly in some cases.
> We can get this fix by using a more modern version of gperftools (see https://issues.apache.org/jira/browse/IMPALA-6784 ). However, the patches for the O(log n) behavior are fairly contained. Here are the two patches needed:
> [https://github.com/gperftools/gperftools/commit/06c9414ec423ffe442c047b2560555f9d5847b1d]
> [https://github.com/gperftools/gperftools/commit/f1d3fe4a21e339a3fd6e4592ee7444484a7b92dc]
> These would be easy to port to gperftools-2.5. This Jira tracks that effort (which is separate and would be superseded if we upgrade gperftools).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org