You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Divij Vaidya <di...@gmail.com> on 2018/02/23 22:24:35 UTC
Thread contention on GroovyClassLoader sourceCache
Hello,
My name is Divij and I am working on an application which requires
concurrent script compilation with a high throughput.
I am using version 2.4.12 and currently, I am not able to achieve high
throughput because the threads are waiting at the synchronized code block
at [1].
I propose removing the synchronized block and changing sourceCache to
ConcurrentHashMap (or maybe some other thread safe cache that can provide
weak references).
I would be happy to make this change and send a pull request if you think
it is valid change to make.
Regards,
Divij
[1]
https://github.com/groovy/groovy-core/blob/master/src/main/groovy/lang/GroovyClassLoader.java#L266
Re: Thread contention on GroovyClassLoader sourceCache
Posted by mg <mg...@arscreat.com>.
Suggestion: GroovyScriptCompiler o.s., and add comment in GCL documentation "If you want concurrency performance, ... to compile Groovy scripts in your application, use GroovyScriptCompiler."Make GCL functionality private, or at least add the same comment there.
Rationale: From what I have read here, Groovy users seem to follow the same path of reasoning, landing at GCL as the place they expect this to work. GCL looks quite low level from the outside, so this makes sense to me.GroovyShell is not where I would look for this, since I would assume it to be too heavyweight a class, with a lot of functionality I do not need/want.
mg10c
-------- Ursprüngliche Nachricht --------Von: Jochen Theodorou <bl...@gmx.org> Datum: 24.02.18 12:06 (GMT+01:00) An: dev@groovy.apache.org Betreff: Re: Thread contention on GroovyClassLoader sourceCache
On 24.02.2018 07:14, Daniel Sun wrote:
> Hi Jochen,
>
> As a Groovy user, I prefer higher level API, e.g. `GroovyShell`,
> `GroovyClassLoader` ;-)
This kind of functionality can be in a higher API of course, but I think
GCL needs to be split instead of having another "mode". I think
GroovyShell would be a better place for this. Or maybe a another GroovyShell
bye Jochen
Re: Thread contention on GroovyClassLoader sourceCache
Posted by Jochen Theodorou <bl...@gmx.org>.
On 24.02.2018 07:14, Daniel Sun wrote:
> Hi Jochen,
>
> As a Groovy user, I prefer higher level API, e.g. `GroovyShell`,
> `GroovyClassLoader` ;-)
This kind of functionality can be in a higher API of course, but I think
GCL needs to be split instead of having another "mode". I think
GroovyShell would be a better place for this. Or maybe a another GroovyShell
bye Jochen
Re: Thread contention on GroovyClassLoader sourceCache
Posted by Daniel Sun <re...@hotmail.com>.
Hi Jochen,
As a Groovy user, I prefer higher level API, e.g. `GroovyShell`,
`GroovyClassLoader` ;-)
Cheers,
Daniel.Sun
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Re: Thread contention on GroovyClassLoader sourceCache
Posted by Jochen Theodorou <bl...@gmx.org>.
On 23.02.2018 23:24, Divij Vaidya wrote:
> Hello,
>
> My name is Divij and I am working on an application which requires
> concurrent script compilation with a high throughput.
>
> I am using version 2.4.12 and currently, I am not able to achieve high
> throughput because the threads are waiting at the synchronized code
> block at [1].
Can we run concurrent compilations? Sure
Can we run concurrent compilations using GCL as entry point... depends.
GroovyClassLoader can perform two tasks:
(1) create the class from a groovy source, if found (as well as
recompilation)
(2) loading a class and if the class does not exist, create it using (1)
I think the API here requires rework. parseClass should, in that simple
form be available through other means. And GCL should have it only
internally. And that is because (2) requires us to think of certain
rules, defined by how a classloader should behave. And why we obviously
do violate some of the rules, one is, that without recompilation a
loadClass call should return the same class.
Now assume the class does not exist and we compile it from source and
there have been two parallel loadClass calls for this. The effect is
that loadClass will return two different classes. That basically makes
us require that synchronized block.
Why not use CompilationUnit directly?
bye Jochen
Re: Thread contention on GroovyClassLoader sourceCache
Posted by Daniel Sun <re...@hotmail.com>.
Another thread for your reference:
http://groovy.329449.n5.nabble.com/Executing-Groovy-DSL-scripts-concurrently-tp5747234.html
Cheers,
Daniel.Sun
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Re: Thread contention on GroovyClassLoader sourceCache
Posted by di...@gmail.com,
di...@gmail.com.
Thanks Daniel.
Would it be possible to backport this particular commit to 2.4.x branch? Can you please point me to the commit where you made the change?
On 2018/02/24 03:02:28, Daniel Sun <re...@hotmail.com> wrote:
> Hi Divij,
>
> `ConcurrentHashMap` does not provide atomic operation(e.g.
> check-and-put) util Java8. So I have achieved a fine-grained access control
> to these caches[1], which is available in 2.5+. 2.5 beta-3[2] is out and RC
> will be released soon, so please give it a try :)
>
> Cheers,
> Daniel.Sun
>
> [1]
> https://github.com/apache/groovy/blob/GROOVY_2_5_X/src/main/groovy/groovy/lang/GroovyClassLoader.java#L105
> [2]
> https://dl.bintray.com/groovy/maven/apache-groovy-binary-2.5.0-beta-3.zip
>
>
>
> --
> Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
>
Re: Thread contention on GroovyClassLoader sourceCache
Posted by Daniel Sun <re...@hotmail.com>.
Hi Divij,
`ConcurrentHashMap` does not provide atomic operation(e.g.
check-and-put) util Java8. So I have achieved a fine-grained access control
to these caches[1], which is available in 2.5+. 2.5 beta-3[2] is out and RC
will be released soon, so please give it a try :)
Cheers,
Daniel.Sun
[1]
https://github.com/apache/groovy/blob/GROOVY_2_5_X/src/main/groovy/groovy/lang/GroovyClassLoader.java#L105
[2]
https://dl.bintray.com/groovy/maven/apache-groovy-binary-2.5.0-beta-3.zip
--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html