You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Cédric Champeau <cc...@apache.org> on 2018/05/24 06:30:13 UTC
Performance of the compiler
Hi folks,
I just wanted to share the result of profiling the performance of the
compiler on "real world" classes, namely compiling the tests of Gradle. We
have a lot of tests, so compilation times becomes really a pain point, so I
have checked where we spend time. I have attached the export of hotspots
from a real compilation session.
It's no surprise to me, most of the time (70%) is spent in the resolve
visitor, and most of this time itself is spent in loading classes. We made
some improvements in the past, by not initializing those classes, but it's
still a crazy amount of time.
Similarly, we spend around 10% of our time in filling stack traces which
are used for flow control. Unfortunately we don't have the opportunity to
change this because it's either ClassNotFoundException (during resolution)
sent by the classloader, or ANTLR recognition exception, used for flow
control (duh!).
I remember that for Gradle I had implemented a custom ResolveVisitor that
adds some assumptions for Gradle scripts to avoid too many lookups for
classes which would obvisouly not exist, and it significantly improved the
performance of compiling scripts, but that was because there were lots of
implicit imports. For regular classes I'm not sure it's as simple.
Resolution is also very easy to break... Anyway, any change in this area
would probably make the lives of our users better!
Re: Performance of the compiler
Posted by Cédric Champeau <cc...@apache.org>.
I have no idea!
Le ven. 25 mai 2018 à 11:05, Jesper Steen Møller <je...@selskabet.org> a
écrit :
> Oh - I see now that this is implemented already (optimizationOptions
> w/asmResolvingin:true).
> Is there any reason this is not the default? Is it slower? Incorrect?
>
> -Jesper
>
> > On 24 May 2018, at 10.08, Jesper Steen Møller <je...@selskabet.org>
> wrote:
> >
> > Interesting! So let me get this straight: Are we using an actual
> "in-JVM" classloader to load classes examined by the Groovy Compiler itself?
> > In the Eclipse Java compiler, we don't actually load the classes into
> the JVM, instead we have our own implementation of classpath traversal and
> read it using ClassFileReader. Would a similar approach not work for
> constructing ClassNodes in Groovy? (obviousluy using ASM to do the bytecode
> parsing instead of rolling it ourselves).
> >
> > JPMS would naturally make complicate things further, but again, ECJ has
> cracked this, too.
> >
> > -Jesper
> >
> >> On 24 May 2018, at 08.30, Cédric Champeau <cc...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >> I just wanted to share the result of profiling the performance of the
> compiler on "real world" classes, namely compiling the tests of Gradle. We
> have a lot of tests, so compilation times becomes really a pain point, so I
> have checked where we spend time. I have attached the export of hotspots
> from a real compilation session.
> >>
> >> It's no surprise to me, most of the time (70%) is spent in the resolve
> visitor, and most of this time itself is spent in loading classes. We made
> some improvements in the past, by not initializing those classes, but it's
> still a crazy amount of time.
> >>
> >> Similarly, we spend around 10% of our time in filling stack traces
> which are used for flow control. Unfortunately we don't have the
> opportunity to change this because it's either ClassNotFoundException
> (during resolution) sent by the classloader, or ANTLR recognition
> exception, used for flow control (duh!).
> >>
> >> I remember that for Gradle I had implemented a custom ResolveVisitor
> that adds some assumptions for Gradle scripts to avoid too many lookups for
> classes which would obvisouly not exist, and it significantly improved the
> performance of compiling scripts, but that was because there were lots of
> implicit imports. For regular classes I'm not sure it's as simple.
> >>
> >> Resolution is also very easy to break... Anyway, any change in this
> area would probably make the lives of our users better!
> >>
> >>
> >> <CPU-hot-spots.txt>
> >
>
>
Re: Performance of the compiler
Posted by Jesper Steen Møller <je...@selskabet.org>.
Oh - I see now that this is implemented already (optimizationOptions w/asmResolvingin:true).
Is there any reason this is not the default? Is it slower? Incorrect?
-Jesper
> On 24 May 2018, at 10.08, Jesper Steen Møller <je...@selskabet.org> wrote:
>
> Interesting! So let me get this straight: Are we using an actual "in-JVM" classloader to load classes examined by the Groovy Compiler itself?
> In the Eclipse Java compiler, we don't actually load the classes into the JVM, instead we have our own implementation of classpath traversal and read it using ClassFileReader. Would a similar approach not work for constructing ClassNodes in Groovy? (obviousluy using ASM to do the bytecode parsing instead of rolling it ourselves).
>
> JPMS would naturally make complicate things further, but again, ECJ has cracked this, too.
>
> -Jesper
>
>> On 24 May 2018, at 08.30, Cédric Champeau <cc...@apache.org> wrote:
>>
>> Hi folks,
>>
>> I just wanted to share the result of profiling the performance of the compiler on "real world" classes, namely compiling the tests of Gradle. We have a lot of tests, so compilation times becomes really a pain point, so I have checked where we spend time. I have attached the export of hotspots from a real compilation session.
>>
>> It's no surprise to me, most of the time (70%) is spent in the resolve visitor, and most of this time itself is spent in loading classes. We made some improvements in the past, by not initializing those classes, but it's still a crazy amount of time.
>>
>> Similarly, we spend around 10% of our time in filling stack traces which are used for flow control. Unfortunately we don't have the opportunity to change this because it's either ClassNotFoundException (during resolution) sent by the classloader, or ANTLR recognition exception, used for flow control (duh!).
>>
>> I remember that for Gradle I had implemented a custom ResolveVisitor that adds some assumptions for Gradle scripts to avoid too many lookups for classes which would obvisouly not exist, and it significantly improved the performance of compiling scripts, but that was because there were lots of implicit imports. For regular classes I'm not sure it's as simple.
>>
>> Resolution is also very easy to break... Anyway, any change in this area would probably make the lives of our users better!
>>
>>
>> <CPU-hot-spots.txt>
>
Re: Performance of the compiler
Posted by Jesper Steen Møller <je...@selskabet.org>.
Interesting! So let me get this straight: Are we using an actual "in-JVM" classloader to load classes examined by the Groovy Compiler itself?
In the Eclipse Java compiler, we don't actually load the classes into the JVM, instead we have our own implementation of classpath traversal and read it using ClassFileReader. Would a similar approach not work for constructing ClassNodes in Groovy? (obviousluy using ASM to do the bytecode parsing instead of rolling it ourselves).
JPMS would naturally make complicate things further, but again, ECJ has cracked this, too.
-Jesper
> On 24 May 2018, at 08.30, Cédric Champeau <cc...@apache.org> wrote:
>
> Hi folks,
>
> I just wanted to share the result of profiling the performance of the compiler on "real world" classes, namely compiling the tests of Gradle. We have a lot of tests, so compilation times becomes really a pain point, so I have checked where we spend time. I have attached the export of hotspots from a real compilation session.
>
> It's no surprise to me, most of the time (70%) is spent in the resolve visitor, and most of this time itself is spent in loading classes. We made some improvements in the past, by not initializing those classes, but it's still a crazy amount of time.
>
> Similarly, we spend around 10% of our time in filling stack traces which are used for flow control. Unfortunately we don't have the opportunity to change this because it's either ClassNotFoundException (during resolution) sent by the classloader, or ANTLR recognition exception, used for flow control (duh!).
>
> I remember that for Gradle I had implemented a custom ResolveVisitor that adds some assumptions for Gradle scripts to avoid too many lookups for classes which would obvisouly not exist, and it significantly improved the performance of compiling scripts, but that was because there were lots of implicit imports. For regular classes I'm not sure it's as simple.
>
> Resolution is also very easy to break... Anyway, any change in this area would probably make the lives of our users better!
>
>
> <CPU-hot-spots.txt>
Re: Performance of the compiler
Posted by Cédric Champeau <ce...@gmail.com>.
The problem is not the performance of the test, it's the performance of
_compiling_ the test. @CompileStatic wouldn't help there.
Le ven. 25 mai 2018 à 14:24, Thibault Kruse <ti...@googlemail.com> a
écrit :
> Would the test performance be improved if @CompileStatic were used? I
> think gradle uses Spock, and last time I checked Spock could not be
> used with @CompileStatic. But Spock could also be removed with some
> effort...
>
> On Fri, May 25, 2018 at 8:52 PM, Jochen Theodorou <bl...@gmx.org>
> wrote:
> >
> >
> > Am 25.05.2018 um 12:51 schrieb Daniel.Sun:
> >>
> >> Hi Cédric,
> >>
> >> I am not going to cache ClassNode instance(just cache class names,
> >> which are `String`), but I want to add a check whether the name of the
> >> ClassNode being resolved is possibly in the default imported packages,
> >> e.g.
> >> If a ClassNode instance's name is `Foobar`(apparently it could not be in
> >> any
> >> default imported packages), then we can `return false` immediately and
> the
> >> further resolving can be eliminated.
> >
> >
> > but this means we will have to manually update the list for java.lang,
> > java.util, java.io and java.net
> >
> > Take for example Module. It is new in Java 9 and is in java.lang. If we
> had
> > this logic already in say Groovy 2.0 I am pretty sure the last versions
> till
> > Groovy 2.3 would not be able to resolve this class anymore then.
> >
> > I think there would be no problem with Java10, but think of Java 11...
> we do
> > not know yet.
> >
> > bye Jochen
>
Re: Performance of the compiler
Posted by Thibault Kruse <ti...@googlemail.com>.
Would the test performance be improved if @CompileStatic were used? I
think gradle uses Spock, and last time I checked Spock could not be
used with @CompileStatic. But Spock could also be removed with some
effort...
On Fri, May 25, 2018 at 8:52 PM, Jochen Theodorou <bl...@gmx.org> wrote:
>
>
> Am 25.05.2018 um 12:51 schrieb Daniel.Sun:
>>
>> Hi Cédric,
>>
>> I am not going to cache ClassNode instance(just cache class names,
>> which are `String`), but I want to add a check whether the name of the
>> ClassNode being resolved is possibly in the default imported packages,
>> e.g.
>> If a ClassNode instance's name is `Foobar`(apparently it could not be in
>> any
>> default imported packages), then we can `return false` immediately and the
>> further resolving can be eliminated.
>
>
> but this means we will have to manually update the list for java.lang,
> java.util, java.io and java.net
>
> Take for example Module. It is new in Java 9 and is in java.lang. If we had
> this logic already in say Groovy 2.0 I am pretty sure the last versions till
> Groovy 2.3 would not be able to resolve this class anymore then.
>
> I think there would be no problem with Java10, but think of Java 11... we do
> not know yet.
>
> bye Jochen
Re: Performance of the compiler
Posted by "Daniel.Sun" <su...@apache.org>.
Hi Jochen,
> but this means we will have to manually update the list for java.lang,
> java.util, java.io and java.net
In order to avoid updating the list manually, I tried to find the way
to get the list via reflection or classloader but failed...
However, even if we add this check to return early, the whole resolving
performance will not be improved a lot...
Cheers,
Daniel.Sun
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Re: Performance of the compiler
Posted by Jochen Theodorou <bl...@gmx.org>.
Am 25.05.2018 um 12:51 schrieb Daniel.Sun:
> Hi Cédric,
>
> I am not going to cache ClassNode instance(just cache class names,
> which are `String`), but I want to add a check whether the name of the
> ClassNode being resolved is possibly in the default imported packages, e.g.
> If a ClassNode instance's name is `Foobar`(apparently it could not be in any
> default imported packages), then we can `return false` immediately and the
> further resolving can be eliminated.
but this means we will have to manually update the list for java.lang,
java.util, java.io and java.net
Take for example Module. It is new in Java 9 and is in java.lang. If we
had this logic already in say Groovy 2.0 I am pretty sure the last
versions till Groovy 2.3 would not be able to resolve this class anymore
then.
I think there would be no problem with Java10, but think of Java 11...
we do not know yet.
bye Jochen
Re: Performance of the compiler
Posted by Cédric Champeau <ce...@gmail.com>.
FWIW, I tried with compileOptions.optimizationOptions.put("asmResolving",
true)
and it doesn't improve much the situation. Sometimes it's even slower. I
didn't investigate why.
Le ven. 25 mai 2018 à 12:51, Daniel.Sun <su...@apache.org> a écrit :
> Hi Cédric,
>
> I am not going to cache ClassNode instance(just cache class names,
> which are `String`), but I want to add a check whether the name of the
> ClassNode being resolved is possibly in the default imported packages, e.g.
> If a ClassNode instance's name is `Foobar`(apparently it could not be in
> any
> default imported packages), then we can `return false` immediately and the
> further resolving can be eliminated.
>
> Cheers,
> Daniel.Sun
>
>
>
> --
> Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
>
Re: Performance of the compiler
Posted by "Daniel.Sun" <su...@apache.org>.
Hi Cédric,
I am not going to cache ClassNode instance(just cache class names,
which are `String`), but I want to add a check whether the name of the
ClassNode being resolved is possibly in the default imported packages, e.g.
If a ClassNode instance's name is `Foobar`(apparently it could not be in any
default imported packages), then we can `return false` immediately and the
further resolving can be eliminated.
Cheers,
Daniel.Sun
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Re: Performance of the compiler
Posted by Cédric Champeau <ce...@gmail.com>.
Be warned that caching may prove to be complicated: what if a class in the
same compile unit is named `List`, or a class in the same package? This is
one of the reasons the lookup is not cached for this.
Le ven. 25 mai 2018 à 10:17, Daniel.Sun <su...@apache.org> a écrit :
> I am going to cache the class names of default imported packages, e.g.
> `List`, `ArrayList`, `Map`, `HashMap`, etc.
>
> And try to find the name of ClassNode instance in the class names cache, if
> not found, `return false`(add the check at the line[1]). The polished logic
> can avoid unnecessary resolving and improve the resolving performance
> somehow.
>
> It's a pity that I have not found any approach to get all classes of the
> given package, e.g. `java.lang`, `java.util`
> Any help is appreciated. (Note: We can not use 3rd party library excluding
> antlr, asm, cli)
>
> Cheers,
> Daniel.Sun
> [1]
>
> https://github.com/apache/groovy/blob/master/src/main/java/org/codehaus/groovy/control/ResolveVisitor.java#L513
>
>
>
> --
> Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
>
Re: Performance of the compiler
Posted by "Daniel.Sun" <su...@apache.org>.
I am going to cache the class names of default imported packages, e.g.
`List`, `ArrayList`, `Map`, `HashMap`, etc.
And try to find the name of ClassNode instance in the class names cache, if
not found, `return false`(add the check at the line[1]). The polished logic
can avoid unnecessary resolving and improve the resolving performance
somehow.
It's a pity that I have not found any approach to get all classes of the
given package, e.g. `java.lang`, `java.util`
Any help is appreciated. (Note: We can not use 3rd party library excluding
antlr, asm, cli)
Cheers,
Daniel.Sun
[1]
https://github.com/apache/groovy/blob/master/src/main/java/org/codehaus/groovy/control/ResolveVisitor.java#L513
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Re: Performance of the compiler
Posted by Cédric Champeau <ce...@gmail.com>.
Tens of thousands of tests, and ~4 minutes compile time (with --parallel).
Le ven. 25 mai 2018 à 14:50, mg <mg...@arscreat.com> a écrit :
> Hi Cedric,
>
> to put this in context and to better understand the ongoing discussion:
> How many tests and what absolute compile time are we talking about here ?
>
> Cheers,
> mg
>
>
> -------- Ursprüngliche Nachricht --------
> Von: Cédric Champeau <cc...@apache.org>
> Datum: 24.05.18 08:30 (GMT+01:00)
> An: dev@groovy.apache.org
> Betreff: Performance of the compiler
>
> Hi folks,
>
> I just wanted to share the result of profiling the performance of the
> compiler on "real world" classes, namely compiling the tests of Gradle. We
> have a lot of tests, so compilation times becomes really a pain point, so I
> have checked where we spend time. I have attached the export of hotspots
> from a real compilation session.
>
> It's no surprise to me, most of the time (70%) is spent in the resolve
> visitor, and most of this time itself is spent in loading classes. We made
> some improvements in the past, by not initializing those classes, but it's
> still a crazy amount of time.
>
> Similarly, we spend around 10% of our time in filling stack traces which
> are used for flow control. Unfortunately we don't have the opportunity to
> change this because it's either ClassNotFoundException (during resolution)
> sent by the classloader, or ANTLR recognition exception, used for flow
> control (duh!).
>
> I remember that for Gradle I had implemented a custom ResolveVisitor that
> adds some assumptions for Gradle scripts to avoid too many lookups for
> classes which would obvisouly not exist, and it significantly improved the
> performance of compiling scripts, but that was because there were lots of
> implicit imports. For regular classes I'm not sure it's as simple.
>
> Resolution is also very easy to break... Anyway, any change in this area
> would probably make the lives of our users better!
>
>
>
Re: Performance of the compiler
Posted by mg <mg...@arscreat.com>.
Hi Cedric,
to put this in context and to better understand the ongoing discussion: How many tests and what absolute compile time are we talking about here ?
Cheers,mg
-------- Ursprüngliche Nachricht --------Von: Cédric Champeau <cc...@apache.org> Datum: 24.05.18 08:30 (GMT+01:00) An: dev@groovy.apache.org Betreff: Performance of the compiler
Hi folks,
I just wanted to share the result of profiling the performance of the compiler on "real world" classes, namely compiling the tests of Gradle. We have a lot of tests, so compilation times becomes really a pain point, so I have checked where we spend time. I have attached the export of hotspots from a real compilation session.
It's no surprise to me, most of the time (70%) is spent in the resolve visitor, and most of this time itself is spent in loading classes. We made some improvements in the past, by not initializing those classes, but it's still a crazy amount of time.
Similarly, we spend around 10% of our time in filling stack traces which are used for flow control. Unfortunately we don't have the opportunity to change this because it's either ClassNotFoundException (during resolution) sent by the classloader, or ANTLR recognition exception, used for flow control (duh!).
I remember that for Gradle I had implemented a custom ResolveVisitor that adds some assumptions for Gradle scripts to avoid too many lookups for classes which would obvisouly not exist, and it significantly improved the performance of compiling scripts, but that was because there were lots of implicit imports. For regular classes I'm not sure it's as simple.
Resolution is also very easy to break... Anyway, any change in this area would probably make the lives of our users better!