You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Paul King <pa...@asert.com.au> on 2026/05/13 12:08:19 UTC
JMH Benchmark Dashboard
Hi folks,
There is now a dashboard for our JMH test results on master:
https://apache.github.io/groovy/dev/bench/jmh/bench/indy/
There are ten groupings covering classic and indy bytecode for 5 groupings
of our benchmarks. The data is from a daily run of the benchmarks. At the
moment, the graphs are measuring as much the load on the day of the CI
container running our tests as they are of our performance. Over time
though, I expect they will show a good trend line.
I was initially thinking of also having an alert if some PR/push commit,
which triggered the performance tests, exceeded the last daily average (or
some running average) by some threshold, e.g. 20% or 50%. But the runner
variance seems quite high, so perhaps there is no real benefit. I have some
code in one of the actions commented out that would do that but if it
doesn't seem useful, I'll delete it.
Cheers, Paul.
Re: JMH Benchmark Dashboard
Posted by Daniel Sun <su...@apache.org>.
Nice work!
Cheers,
Daniel Sun
On 2026/05/14 23:29:27 Paul King wrote:
> I added the compiler performance stats to the dashboard:
>
> https://apache.github.io/groovy/dev/bench/perf/compiler/
>
> Again, runner speed variation causes a lot of noise, but the long-term
> trends will be useful.
>
> The dashboard index is here:
>
> https://apache.github.io/groovy/
>
> Cheers, Paul.
>
>
> On Wed, May 13, 2026 at 10:08 PM Paul King <pa...@asert.com.au> wrote:
> >
> >
> > Hi folks,
> >
> > There is now a dashboard for our JMH test results on master:
> >
> > https://apache.github.io/groovy/dev/bench/jmh/bench/indy/
> >
> > There are ten groupings covering classic and indy bytecode for 5 groupings of our benchmarks. The data is from a daily run of the benchmarks. At the moment, the graphs are measuring as much the load on the day of the CI container running our tests as they are of our performance. Over time though, I expect they will show a good trend line.
> >
> > I was initially thinking of also having an alert if some PR/push commit, which triggered the performance tests, exceeded the last daily average (or some running average) by some threshold, e.g. 20% or 50%. But the runner variance seems quite high, so perhaps there is no real benefit. I have some code in one of the actions commented out that would do that but if it doesn't seem useful, I'll delete it.
> >
> > Cheers, Paul.
> >
>
Re: JMH Benchmark Dashboard
Posted by Paul King <pa...@asert.com.au>.
I added the compiler performance stats to the dashboard:
https://apache.github.io/groovy/dev/bench/perf/compiler/
Again, runner speed variation causes a lot of noise, but the long-term
trends will be useful.
The dashboard index is here:
https://apache.github.io/groovy/
Cheers, Paul.
On Wed, May 13, 2026 at 10:08 PM Paul King <pa...@asert.com.au> wrote:
>
>
> Hi folks,
>
> There is now a dashboard for our JMH test results on master:
>
> https://apache.github.io/groovy/dev/bench/jmh/bench/indy/
>
> There are ten groupings covering classic and indy bytecode for 5 groupings of our benchmarks. The data is from a daily run of the benchmarks. At the moment, the graphs are measuring as much the load on the day of the CI container running our tests as they are of our performance. Over time though, I expect they will show a good trend line.
>
> I was initially thinking of also having an alert if some PR/push commit, which triggered the performance tests, exceeded the last daily average (or some running average) by some threshold, e.g. 20% or 50%. But the runner variance seems quite high, so perhaps there is no real benefit. I have some code in one of the actions commented out that would do that but if it doesn't seem useful, I'll delete it.
>
> Cheers, Paul.
>