You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2026/06/16 21:35:00 UTC
[jira] [Commented] (IMPALA-14702) Add support for building against other malloc implementations (e.g. jemalloc)
[ https://issues.apache.org/jira/browse/IMPALA-14702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18089506#comment-18089506 ]
ASF subversion and git services commented on IMPALA-14702:
----------------------------------------------------------
Commit 0b8294b1a5b0ad4a817dee13b7fbb2ee53f534e2 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0b8294b1a ]
IMPALA-14702 (prep): Add an abstraction around malloc implementation details
This adds a MallocUtil abstraction around the malloc implementation
to make it easier to try other mallocs. This has different
MallocUtil libraries for different malloc implementations:
- gperftools tcmalloc
- sanitizer builds (which have their own malloc implementation)
- libc malloc (used only for certain tests).
Programs should set any relevant startup flags, then call
MallocUtil::Init() to apply the settings.
This introduces a tcmalloc_aggressive_memory_decommit startup
flag, but limits the configurability and retain existing behavior.
Impalad is hard coded to use tcmalloc_aggressive_memory_decommit=true
and does not respect a user's value. Other binaries default to
tcmalloc_aggressive_memory_decommit=false but will respect a
custom value.
Impalad's memory tracker has an entry for the malloc overhead.
This converts it to a regular metric so that it is available
for other daemons.
Testing:
- Ran core job
- Ran pprof locally against the heap profiler endpoint
Change-Id: I3ac7841cf279ac40141147d4be9192894d6dc908
Reviewed-on: http://gerrit.cloudera.org:8080/24401
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
> Add support for building against other malloc implementations (e.g. jemalloc)
> -----------------------------------------------------------------------------
>
> Key: IMPALA-14702
> URL: https://issues.apache.org/jira/browse/IMPALA-14702
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Impala builds against gperftools tcmalloc. Other malloc implementations like jemalloc and Google's tcmalloc have different designs that would be interesting to test. We know that there are performance improvements related to avoiding gperftool tcmalloc's aggressive decommit setting. It would be nice to test that against other malloc implementations (or forgo overriding malloc altogether).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org