Dedicated hardware for tests/benchmarking

[Matthew Benedict de Detrich <ma...@aiven.io.INVALID>]

Letting everyone know that I am currently going through the process of
trying to procure some dedicated hardware for Pekko, this is being done for
two main reasons

* Provide deterministic resources for our nightly tests (i.e.
https://github.com/apache/incubator-pekko/blob/main/.github/workflows/nightly-builds.yml
and
https://github.com/apache/incubator-pekko/blob/main/.github/workflows/nightly-builds-aeron.yml)
so we don’t have to deal with the noisy neighbour problem. With a dedicated
machine we can also use technologies like LXC/Docker with bespoke
CPU/memory resources for specific tests to simulate/test Pekko under
constrained conditions.

* Also provide a machine that can be used for benchmarking which naturally
requires dedicated hardware. I am currently thinking of this as a nice to
have since dedicated machines are only one piece of the problem (i.e. if
you want to have really accurate benchmarks then you also need to do things
like disable turboing of clocks but at least having a dedicated machine is
a good starting point).

Thankfully in today's day and age, such dedicated hardware is not that
expensive and so I am in the process of writing a proposal within the
company where I work (Aiven) to sponsor the hardware.

Naturally such an endeavour raises questions. To start off with for those
who are not aware we are not alone with this, it’s not uncommon for
companies to sponsor hardware for Apache projects (Ververica is an example
of an external company which provides hardware for Apache Flink). The point
of bringing this up is to indicate this has been done before, and
specifically in our case because we are using GitHub actions there is
already a wiki article on the technical details at
https://cwiki.apache.org/confluence/display/INFRA/GitHub+self-hosted+runners.
Do note that while the wiki does raise specific security concerns in our
case the dedicated machine will not run on PR’s, instead it's going to run
nightly on the main branch which only contains code that has been approved
(since main is protected) and wrt possible future benchmarking this would
likely be triggered by a GitHub bot that would only respond to committers
(in the case we want to run a benchmark on specific PR’s to see that there
isn’t a performance regression).

There are other questions such as securing the machine itself and how to
integrate this within Apache’s both structurally and organisationally
(amongst others), these questions will be asked in the upcoming
Infrastructure Round Table this Wednesday.

-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io