You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Paul King <pa...@asert.com.au> on 2022/08/10 12:30:15 UTC
performance subproject directions
Hi folks,
The performance subproject in our build was designed to check on
(among other things) compiler performance. The 'performanceTests' task
compiles a set of source files across numerous Groovy versions. Over
time, the number of source files it checks against has been diluted.
The exact history isn't too important, but the summary is something
like this:
* We have tried to keep the tests to just relying on the core subproject.
* We have tried to not rely on too many external dependencies.
* We have tried not to include any classes which have been impacted by
package changes, e.g. using XmlParser would require
groovy.util.XmlParser in early versions and groovy.xml.XmlParser in
later versions.
* We haven't included functionality included in later versions.
* Our approach when dropping further impacted files has been to just
remove them from the list of files compiled.
The end result is that the current performance test reveals very
little information. I think we need to add in numerous additional
files. I have also recently pruned down the versions we try to check
against. I don't think there is a lot of value in knowing performance
compared to Groovy 1.8.x.
I have just created a PR which adds a bunch of additional files - I
used a whole bunch of examples I created some years ago for PLEAC. I
believe the licensing is all legit. There are around 10000 lines of
Groovy source in those files. The alternative would be to pick our way
through the current source base and add selected files which meet all
of the criteria we have above.
Incidentally, results on Windows were:
> Task :performance:performanceTests
Groovy 2.4.21 Average 437.965ms ± 16.07ms
Groovy 4.0.5-SNAPSHOT Average 649.8199999999999ms ± 89.79ms (48.37% slower)
Groovy 4.0.4 Average 654.635ms ± 85.86ms (49.47% slower)
Groovy 2.5.18 Average 676.224ms ± 8.38ms (54.4% slower)
Groovy 3.0.12 Average 792.625ms ± 192.19ms (80.98% slower)
On Linux, the results were:
> Task :performance:performanceTests
Groovy 2.4.21 Average 473.17699999999996ms ± 24.47ms
Groovy 4.0.4 Average 648.592ms ± 129.15ms (37.07% slower)
Groovy current Average 649.4449999999999ms ± 126.79ms (37.25% slower)
Groovy 2.5.18 Average 683.147ms ± 12.6ms (44.37% slower)
Groovy 3.0.12 Average 796.938ms ± 225.73ms (68.42% slower)
Here, "current" is the version of the branch you are running on
(GROOVY_4_0_X for me). I also published the current branch (on
Windows) to the local Maven repo in case that made any difference.
The usual disclaimer about microbenchmarks applies. Also, remember,
this is testing oldish code - so not code rich in AST transforms or
using CompileStatic.
Thoughts about whether the Pleac examples are good to add or should we
move all of this to another repo?
Cheers, Paul.
Re: performance subproject directions
Posted by Daniel Sun <su...@apache.org>.
As the proposal received no objection for a long time, so I merged the PR.
Cheers,
Daniel Sun
On 2022/08/10 12:42:20 Paul King wrote:
> I should've included a link to the PR:
> https://github.com/apache/groovy/pull/1761
>
> On Wed, Aug 10, 2022 at 10:30 PM Paul King <pa...@asert.com.au> wrote:
> >
> > Hi folks,
> >
> > The performance subproject in our build was designed to check on
> > (among other things) compiler performance. The 'performanceTests' task
> > compiles a set of source files across numerous Groovy versions. Over
> > time, the number of source files it checks against has been diluted.
> > The exact history isn't too important, but the summary is something
> > like this:
> >
> > * We have tried to keep the tests to just relying on the core subproject.
> > * We have tried to not rely on too many external dependencies.
> > * We have tried not to include any classes which have been impacted by
> > package changes, e.g. using XmlParser would require
> > groovy.util.XmlParser in early versions and groovy.xml.XmlParser in
> > later versions.
> > * We haven't included functionality included in later versions.
> > * Our approach when dropping further impacted files has been to just
> > remove them from the list of files compiled.
> >
> > The end result is that the current performance test reveals very
> > little information. I think we need to add in numerous additional
> > files. I have also recently pruned down the versions we try to check
> > against. I don't think there is a lot of value in knowing performance
> > compared to Groovy 1.8.x.
> >
> > I have just created a PR which adds a bunch of additional files - I
> > used a whole bunch of examples I created some years ago for PLEAC. I
> > believe the licensing is all legit. There are around 10000 lines of
> > Groovy source in those files. The alternative would be to pick our way
> > through the current source base and add selected files which meet all
> > of the criteria we have above.
> >
> > Incidentally, results on Windows were:
> > > Task :performance:performanceTests
> > Groovy 2.4.21 Average 437.965ms ± 16.07ms
> > Groovy 4.0.5-SNAPSHOT Average 649.8199999999999ms ± 89.79ms (48.37% slower)
> > Groovy 4.0.4 Average 654.635ms ± 85.86ms (49.47% slower)
> > Groovy 2.5.18 Average 676.224ms ± 8.38ms (54.4% slower)
> > Groovy 3.0.12 Average 792.625ms ± 192.19ms (80.98% slower)
> >
> > On Linux, the results were:
> > > Task :performance:performanceTests
> > Groovy 2.4.21 Average 473.17699999999996ms ± 24.47ms
> > Groovy 4.0.4 Average 648.592ms ± 129.15ms (37.07% slower)
> > Groovy current Average 649.4449999999999ms ± 126.79ms (37.25% slower)
> > Groovy 2.5.18 Average 683.147ms ± 12.6ms (44.37% slower)
> > Groovy 3.0.12 Average 796.938ms ± 225.73ms (68.42% slower)
> >
> > Here, "current" is the version of the branch you are running on
> > (GROOVY_4_0_X for me). I also published the current branch (on
> > Windows) to the local Maven repo in case that made any difference.
> >
> > The usual disclaimer about microbenchmarks applies. Also, remember,
> > this is testing oldish code - so not code rich in AST transforms or
> > using CompileStatic.
> >
> > Thoughts about whether the Pleac examples are good to add or should we
> > move all of this to another repo?
> >
> > Cheers, Paul.
>
Re: performance subproject directions
Posted by Paul King <pa...@asert.com.au>.
I should've included a link to the PR:
https://github.com/apache/groovy/pull/1761
On Wed, Aug 10, 2022 at 10:30 PM Paul King <pa...@asert.com.au> wrote:
>
> Hi folks,
>
> The performance subproject in our build was designed to check on
> (among other things) compiler performance. The 'performanceTests' task
> compiles a set of source files across numerous Groovy versions. Over
> time, the number of source files it checks against has been diluted.
> The exact history isn't too important, but the summary is something
> like this:
>
> * We have tried to keep the tests to just relying on the core subproject.
> * We have tried to not rely on too many external dependencies.
> * We have tried not to include any classes which have been impacted by
> package changes, e.g. using XmlParser would require
> groovy.util.XmlParser in early versions and groovy.xml.XmlParser in
> later versions.
> * We haven't included functionality included in later versions.
> * Our approach when dropping further impacted files has been to just
> remove them from the list of files compiled.
>
> The end result is that the current performance test reveals very
> little information. I think we need to add in numerous additional
> files. I have also recently pruned down the versions we try to check
> against. I don't think there is a lot of value in knowing performance
> compared to Groovy 1.8.x.
>
> I have just created a PR which adds a bunch of additional files - I
> used a whole bunch of examples I created some years ago for PLEAC. I
> believe the licensing is all legit. There are around 10000 lines of
> Groovy source in those files. The alternative would be to pick our way
> through the current source base and add selected files which meet all
> of the criteria we have above.
>
> Incidentally, results on Windows were:
> > Task :performance:performanceTests
> Groovy 2.4.21 Average 437.965ms ± 16.07ms
> Groovy 4.0.5-SNAPSHOT Average 649.8199999999999ms ± 89.79ms (48.37% slower)
> Groovy 4.0.4 Average 654.635ms ± 85.86ms (49.47% slower)
> Groovy 2.5.18 Average 676.224ms ± 8.38ms (54.4% slower)
> Groovy 3.0.12 Average 792.625ms ± 192.19ms (80.98% slower)
>
> On Linux, the results were:
> > Task :performance:performanceTests
> Groovy 2.4.21 Average 473.17699999999996ms ± 24.47ms
> Groovy 4.0.4 Average 648.592ms ± 129.15ms (37.07% slower)
> Groovy current Average 649.4449999999999ms ± 126.79ms (37.25% slower)
> Groovy 2.5.18 Average 683.147ms ± 12.6ms (44.37% slower)
> Groovy 3.0.12 Average 796.938ms ± 225.73ms (68.42% slower)
>
> Here, "current" is the version of the branch you are running on
> (GROOVY_4_0_X for me). I also published the current branch (on
> Windows) to the local Maven repo in case that made any difference.
>
> The usual disclaimer about microbenchmarks applies. Also, remember,
> this is testing oldish code - so not code rich in AST transforms or
> using CompileStatic.
>
> Thoughts about whether the Pleac examples are good to add or should we
> move all of this to another repo?
>
> Cheers, Paul.