You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2022/06/17 17:12:00 UTC

[jira] [Created] (IMPALA-11370) Run make with --load-average for large toolchain components

Joe McDonnell created IMPALA-11370:
--------------------------------------

             Summary: Run make with --load-average for large toolchain components
                 Key: IMPALA-11370
                 URL: https://issues.apache.org/jira/browse/IMPALA-11370
             Project: IMPALA
          Issue Type: Improvement
          Components: Infrastructure
    Affects Versions: Impala 4.2.0
            Reporter: Joe McDonnell


Several components of the toolchain can use a significant amount of parallelism and resources (e.g. LLVM, Kudu). When using the docker build infrastructures, multiple of these builds can happen simultaneously. This can result in overloading the machine and high memory usage / memory pressure.

One way to deal with it is to run those builds with make's --load-average command so that the total load doesn't go too high. This can result in a sawtooth load pattern as the make commands cut off and the load drifts down, so this may need some tuning of the value.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org