You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Joe McDonnell (JIRA)" <ji...@apache.org> on 2019/07/02 23:19:00 UTC

[jira] [Created] (IMPALA-8737) Patch gperftools to fix O(n) scaling in PageHeap::AllocLarge()

Joe McDonnell created IMPALA-8737:
-------------------------------------

             Summary: Patch gperftools to fix O(n) scaling in PageHeap::AllocLarge()
                 Key: IMPALA-8737
                 URL: https://issues.apache.org/jira/browse/IMPALA-8737
             Project: IMPALA
          Issue Type: Bug
    Affects Versions: Impala 3.3.0
            Reporter: Joe McDonnell


TCMalloc's PageHeap::AllocLarge() has O(n) behavior. As the heap gets fragmented, this O(n) behavior can lead to contention, because the thread executing PageHeap::AllocLarge() is holding a lock. In recent versions of gperftools, this code has been modified to have O(log n) behavior. This could reduce contention significantly in some cases.

We can get this fix by using a more modern version of gperftools (see https://issues.apache.org/jira/browse/IMPALA-6784 ). However, the patches for the O(log n) behavior are fairly contained. Here are the two patches needed:

[https://github.com/gperftools/gperftools/commit/06c9414ec423ffe442c047b2560555f9d5847b1d]

[https://github.com/gperftools/gperftools/commit/f1d3fe4a21e339a3fd6e4592ee7444484a7b92dc]

These would be easy to port to gperftools-2.5. This Jira tracks that effort (which is separate and would be superseded if we upgrade gperftools). 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)