[jira] [Assigned] (IMPALA-14527) Switch to mold as the default linker

["Michael Smith (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Smith reassigned IMPALA-14527:
--------------------------------------

    Assignee: Michael Smith

> Switch to mold as the default linker
> ------------------------------------
>
>                 Key: IMPALA-14527
>                 URL: https://issues.apache.org/jira/browse/IMPALA-14527
>             Project: IMPALA
>          Issue Type: Task
>          Components: Infrastructure
>            Reporter: Michael Smith
>            Assignee: Michael Smith
>            Priority: Major
>
> The mold linker better utilizes concurrency to speed up linking a single binary. This has two benefits
> 1. Development when relinking e.g. impalad will be significantly faster (
> 2. During a full build that links many test binaries, we can use MOLD_LINKER=1 to link a single binary at a time - using significantly less memory - while still fully utilizing all CPU cores.
> Full builds are also a bit faster. After priming ccache with a full build; with mold rebuilds take ~2m33s and memory use remains under 10GiB (24 cores)
> {code}
> $ env MOLD_JOBS=1 IMPALA_LINKER=mold time ./buildall.sh -skiptests
> ...
> 313.23user 67.68system 2:33.65elapsed 247%CPU (0avgtext+0avgdata 5664100maxresident)k
> 16inputs+34793584outputs (216major+12078088minor)pagefaults 0swaps
> {code}
> With gold, it takes ~3m15s and maxes out my memory (64GiB) requiring additional swap
> {code}
> $ env IMPALA_LINKER=gold time ./buildall.sh -skiptests
> ...
> 261.36user 71.43system 3:15.41elapsed 170%CPU (0avgtext+0avgdata 3655132maxresident)k
> 7485904inputs+34794136outputs (88605major+11836509minor)pagefaults 0swaps
> {code}
> Performance of the resulting binaries looks essentially the same: https://jenkins.impala.io/job/perf-AB-test-ub2004/336/artifact/Impala/perf_results/latest/performance_result.txt.
> Let's switch to using IMPALA_LINKER=mold by default.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org