[jira] [Created] (IMPALA-13329) Add a more to bin/single_node_perf_run.py to use cpusets to isolate impalads

["Joe McDonnell (Jira)" <ji...@apache.org>]

Joe McDonnell created IMPALA-13329:
--------------------------------------

             Summary: Add a more to bin/single_node_perf_run.py to use cpusets to isolate impalads
                 Key: IMPALA-13329
                 URL: https://issues.apache.org/jira/browse/IMPALA-13329
             Project: IMPALA
          Issue Type: Task
          Components: Infrastructure
    Affects Versions: Impala 4.5.0
            Reporter: Joe McDonnell


We want the perf-AB-test job to have as little noise and bias as possible to get a clean read on performance differences. One way to reduce the noise would be to use dedicated CPUs for each Impalad while they are running. Keeping the minicluster components and perf script runner separate avoids interactions between the two. Keeping the impalads separate from each other better simulates different physical machines with dedicated CPUs.

One way to do this is to use cpusets via cgroups. Essentially, we can create a separate cpuset cgroup for each impalad and another for everything else. For example, on a 16 CPU machine, one could give 4 CPUs to each of the three impalads and then have the remaining 4 CPUs available for everything else. This aligns with the hyperthreading boundaries, so nothing is sharing anything.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)