You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Jochen Theodorou <bl...@gmx.org> on 2026/05/27 09:40:27 UTC

how to use the github benchmarks for branches

Hi,

Let us assume I just created some code that could improve performance 
and I want to compare this to existing benchmark results. Currently I 
would run the benchmarks for over 2 hours on my laptop, with unreliable 
results, locking me out of work - or maybe the jmh results that did run 
on my branch anyway. For the later I would like to know if I have to 
manually download all the artifacts for master and my branch, write a 
script that compares and then finally get an overview... Or we have 
something already that lets me see if my change improve or degrade 
performance?

bye Jochen

Re: how to use the github benchmarks for branches

Posted by Paul King <pa...@asert.com.au>.
It is still highly subject to GitHub runner performance which fluctuates a lot.

On Wed, May 27, 2026 at 11:40 PM Jochen Theodorou <bl...@gmx.org> wrote:
>
> Hi,
>
> that is something I can work with, thanks. Also I can now see where my
> changes may need improvement
>
> bye Jochen
>
> On 5/27/26 14:30, Paul King wrote:
> > There should now be a summary of performance added to the action
> > result page, e.g.:
> > https://github.com/apache/groovy/actions/runs/26506671271/attempts/4#summary-78075520486
> >
> > On Wed, May 27, 2026 at 8:41 PM Paul King <pa...@asert.com.au> wrote:
> >>
> >> Currently, the comparison work and pretty graphs are done for daily tasks only.
> >>
> >> We have a placeholder for per commit/PR info but it is commented out.
> >> Let me see if I can tweak that ...
> >>
> >> On Wed, May 27, 2026 at 7:40 PM Jochen Theodorou <bl...@gmx.org> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Let us assume I just created some code that could improve performance
> >>> and I want to compare this to existing benchmark results. Currently I
> >>> would run the benchmarks for over 2 hours on my laptop, with unreliable
> >>> results, locking me out of work - or maybe the jmh results that did run
> >>> on my branch anyway. For the later I would like to know if I have to
> >>> manually download all the artifacts for master and my branch, write a
> >>> script that compares and then finally get an overview... Or we have
> >>> something already that lets me see if my change improve or degrade
> >>> performance?
> >>>
> >>> bye Jochen
>

Re: how to use the github benchmarks for branches

Posted by Jochen Theodorou <bl...@gmx.org>.
Hi,

that is something I can work with, thanks. Also I can now see where my 
changes may need improvement

bye Jochen

On 5/27/26 14:30, Paul King wrote:
> There should now be a summary of performance added to the action
> result page, e.g.:
> https://github.com/apache/groovy/actions/runs/26506671271/attempts/4#summary-78075520486
> 
> On Wed, May 27, 2026 at 8:41 PM Paul King <pa...@asert.com.au> wrote:
>>
>> Currently, the comparison work and pretty graphs are done for daily tasks only.
>>
>> We have a placeholder for per commit/PR info but it is commented out.
>> Let me see if I can tweak that ...
>>
>> On Wed, May 27, 2026 at 7:40 PM Jochen Theodorou <bl...@gmx.org> wrote:
>>>
>>> Hi,
>>>
>>> Let us assume I just created some code that could improve performance
>>> and I want to compare this to existing benchmark results. Currently I
>>> would run the benchmarks for over 2 hours on my laptop, with unreliable
>>> results, locking me out of work - or maybe the jmh results that did run
>>> on my branch anyway. For the later I would like to know if I have to
>>> manually download all the artifacts for master and my branch, write a
>>> script that compares and then finally get an overview... Or we have
>>> something already that lets me see if my change improve or degrade
>>> performance?
>>>
>>> bye Jochen


Re: how to use the github benchmarks for branches

Posted by Paul King <pa...@asert.com.au>.
There should now be a summary of performance added to the action
result page, e.g.:
https://github.com/apache/groovy/actions/runs/26506671271/attempts/4#summary-78075520486

On Wed, May 27, 2026 at 8:41 PM Paul King <pa...@asert.com.au> wrote:
>
> Currently, the comparison work and pretty graphs are done for daily tasks only.
>
> We have a placeholder for per commit/PR info but it is commented out.
> Let me see if I can tweak that ...
>
> On Wed, May 27, 2026 at 7:40 PM Jochen Theodorou <bl...@gmx.org> wrote:
> >
> > Hi,
> >
> > Let us assume I just created some code that could improve performance
> > and I want to compare this to existing benchmark results. Currently I
> > would run the benchmarks for over 2 hours on my laptop, with unreliable
> > results, locking me out of work - or maybe the jmh results that did run
> > on my branch anyway. For the later I would like to know if I have to
> > manually download all the artifacts for master and my branch, write a
> > script that compares and then finally get an overview... Or we have
> > something already that lets me see if my change improve or degrade
> > performance?
> >
> > bye Jochen

Re: how to use the github benchmarks for branches

Posted by Paul King <pa...@asert.com.au>.
Currently, the comparison work and pretty graphs are done for daily tasks only.

We have a placeholder for per commit/PR info but it is commented out.
Let me see if I can tweak that ...

On Wed, May 27, 2026 at 7:40 PM Jochen Theodorou <bl...@gmx.org> wrote:
>
> Hi,
>
> Let us assume I just created some code that could improve performance
> and I want to compare this to existing benchmark results. Currently I
> would run the benchmarks for over 2 hours on my laptop, with unreliable
> results, locking me out of work - or maybe the jmh results that did run
> on my branch anyway. For the later I would like to know if I have to
> manually download all the artifacts for master and my branch, write a
> script that compares and then finally get an overview... Or we have
> something already that lets me see if my change improve or degrade
> performance?
>
> bye Jochen