You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by Serega Sheypak <se...@gmail.com> on 2016/04/10 19:38:56 UTC
Optimizing groovy performace
I'm working on groovy code performance optimization. I've used jvisualvm to
connect to running application and gather CPU samples. Samples say that
*org.codehaus.groovy.reflection.CachedMethod.inkove* takes the most CPU
time. I don't see any other application methods in samples.
What is the right way to dig into *CachedMethod.invoke* and understand what
code lines really give performance penalties?
Then I switched to groovy-indy version, now I see different results in
jviusalvm.
Probably, my problem a bit similar to this thread: Call site caching faster
than invokedynamic?
<http://groovy.329449.n5.nabble.com/Call-site-caching-faster-than-invokedynamic-tp5721659p5721674.html>
Most of CPU time spent in
*org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet*
I have a code that dynamically composes groovy script. Script template
looks this way:
def evaluateExpression(Map context){
def user = context.user
%s}
where *%s* replaced with
user.attr1 == '1' || user.attr2 == '2' || user.attr3 = '3'
There is a set (20 in total) of replacements have taken from Databases. The
code gets replacements from DB, creates GroovyScript and evaluates it. I
suppose the bottleneck is in the script execution. What is the right way to
fix it?
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
I replaced all Regex operations with StringUtils from commons-lang3.
Really, I don't have full Regex support, so StringUtils is nice replacement.
Nothing helps. Really, I don't see any changes at all.
What else could I try? Ho can I profile that indy stuff?
2016-04-15 20:33 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> I'm using 2.4.4 Indy
> Storm Framework design is thread safe.
> Each thread has its own hash map. This factory is not shared across
> concurrent threads.
>
> пятница, 15 апреля 2016 г. пользователь John Wagenleitner написал:
>
> Which version of Groovy are you using? I believe in 2.4.6 there was a
>> minor change in the invalidateCallSites for synchronization, probably not
>> the issue here but would be curious to know if also a problem on other
>> version.
>>
>> Looking your gist for ScriptFactory I notice you use a HashMap. I think
>> you mentioned that this app is highly concurrent, so I would recommend
>> using a static final ConcurrentHashMap.
>>
>> On Thu, Apr 14, 2016 at 2:58 PM, Serega Sheypak <serega.sheypak@gmail.com
>> > wrote:
>>
>>> Here is the screenshot from VisualVM.
>>> [image: Встроенное изображение 1]
>>>
>>> 2016-04-14 23:49 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>>
>>>> Bump
>>>>
>>>> 2016-04-12 11:19 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>>>
>>>>> Hi, any update here? Could you give me some input for further
>>>>> investigation?
>>>>>
>>>>> 2016-04-11 12:58 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>>>>
>>>>>> *>you wrote there are 20 such replacement cases... executing 20
>>>>>> scripts is hardly a problem, so it must happen in a loop...*
>>>>>> Sorry, didn't understand, what do you mean by loop?
>>>>>>
>>>>>> *>do you reuse the scripts, or do you make a new script each time? *
>>>>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>>>>
>>>>>> *Hi, here is naive caching factory:*
>>>>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>>>>
>>>>>> *>To give an advice I have to understand your setup better first*
>>>>>> I run http://storm.apache.org/ topology.
>>>>>> If you are not familiar with it, assume I have a running thread-safe
>>>>>> java application. Java application keeps 1000 User objects and 20
>>>>>> GroovyScripts in memory.
>>>>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>>>>> executions each second.
>>>>>> If script for User object returns true, I do something special.
>>>>>>
>>>>>> I try to cache script instances (see my gist).
>>>>>> User object is just a bean with some logic incapsulated.
>>>>>>
>>>>>> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>>>>>>
>>>>>>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>>>>>>
>>>>>>>> >
>>>>>>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>>>>>>> I see this method after switching to indy version with invkoedynamic
>>>>>>>>
>>>>>>>> That is a hint, that you have a lot of invalid callsites going on
>>>>>>>>>
>>>>>>>> What does it mean? What can I do in order fix it?
>>>>>>>>
>>>>>>>> Those are all strings?
>>>>>>>>>
>>>>>>>> Script template is:
>>>>>>>>
>>>>>>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>>>>>>
>>>>>>>> dynamically substituted expression could contain anything
>>>>>>>>
>>>>>>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>>>>>>
>>>>>>>> user object holds info about a user doing something online. attr1,
>>>>>>>> attr2, attr3 could returns various types. Sometimes they return
>>>>>>>> object
>>>>>>>> and I override returned object equals method.
>>>>>>>>
>>>>>>>
>>>>>>> you wrote there are 20 such replacement cases... executing 20
>>>>>>> scripts is hardly a problem, so it must happen in a loop... do you reuse
>>>>>>> the scripts, or do you make a new script each time? Because if you do them
>>>>>>> new each time, you have of course never ever an caching going on... but
>>>>>>> then the compiler should appear in your measurement as bad factor as well
>>>>>>> actually.
>>>>>>>
>>>>>>> To give an advice I have to understand your setup better first
>>>>>>>
>>>>>>>
>>>>>>> For example, 'user' object has method:
>>>>>>>> LastPage getLastPage(){
>>>>>>>> def attr = getLatestAttribute(PAGE)
>>>>>>>> if(attr){
>>>>>>>> lastPage = attr.value
>>>>>>>> }
>>>>>>>> new LastPage(lastPage: lastPage)
>>>>>>>> }
>>>>>>>>
>>>>>>>> then you can write expression:
>>>>>>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if
>>>>>>>> user
>>>>>>>> last page mathes provided regex.
>>>>>>>>
>>>>>>>> Here is LastPage impl:
>>>>>>>> class LastPage {
>>>>>>>> String lastPage
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> String toString(){
>>>>>>>> lastPage
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> int hashCode(){
>>>>>>>> 1
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> boolean equals(Object object){
>>>>>>>> if(object == null || !(object instanceof String || object
>>>>>>>> instanceof LastPage)){
>>>>>>>> return false
>>>>>>>> }
>>>>>>>> if(object instanceof LastPage){
>>>>>>>> lastPage.equals(object.lastPage)
>>>>>>>> }
>>>>>>>> else{
>>>>>>>> Pattern pattern = Pattern.compile((String)object)
>>>>>>>> lastPage ==~ pattern
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>
>>>>>>> and pattern compilation here does not have a negative factor in your
>>>>>>> measurement?
>>>>>>>
>>>>>>> bye Jochen
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
I'm using 2.4.4 Indy
Storm Framework design is thread safe.
Each thread has its own hash map. This factory is not shared across
concurrent threads.
пятница, 15 апреля 2016 г. пользователь John Wagenleitner написал:
> Which version of Groovy are you using? I believe in 2.4.6 there was a
> minor change in the invalidateCallSites for synchronization, probably not
> the issue here but would be curious to know if also a problem on other
> version.
>
> Looking your gist for ScriptFactory I notice you use a HashMap. I think
> you mentioned that this app is highly concurrent, so I would recommend
> using a static final ConcurrentHashMap.
>
> On Thu, Apr 14, 2016 at 2:58 PM, Serega Sheypak <serega.sheypak@gmail.com
> <javascript:_e(%7B%7D,'cvml','serega.sheypak@gmail.com');>> wrote:
>
>> Here is the screenshot from VisualVM.
>> [image: Встроенное изображение 1]
>>
>> 2016-04-14 23:49 GMT+02:00 Serega Sheypak <serega.sheypak@gmail.com
>> <javascript:_e(%7B%7D,'cvml','serega.sheypak@gmail.com');>>:
>>
>>> Bump
>>>
>>> 2016-04-12 11:19 GMT+02:00 Serega Sheypak <serega.sheypak@gmail.com
>>> <javascript:_e(%7B%7D,'cvml','serega.sheypak@gmail.com');>>:
>>>
>>>> Hi, any update here? Could you give me some input for further
>>>> investigation?
>>>>
>>>> 2016-04-11 12:58 GMT+02:00 Serega Sheypak <serega.sheypak@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','serega.sheypak@gmail.com');>>:
>>>>
>>>>> *>you wrote there are 20 such replacement cases... executing 20
>>>>> scripts is hardly a problem, so it must happen in a loop...*
>>>>> Sorry, didn't understand, what do you mean by loop?
>>>>>
>>>>> *>do you reuse the scripts, or do you make a new script each time? *
>>>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>>>
>>>>> *Hi, here is naive caching factory:*
>>>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>>>
>>>>> *>To give an advice I have to understand your setup better first*
>>>>> I run http://storm.apache.org/ topology.
>>>>> If you are not familiar with it, assume I have a running thread-safe
>>>>> java application. Java application keeps 1000 User objects and 20
>>>>> GroovyScripts in memory.
>>>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>>>> executions each second.
>>>>> If script for User object returns true, I do something special.
>>>>>
>>>>> I try to cache script instances (see my gist).
>>>>> User object is just a bean with some logic incapsulated.
>>>>>
>>>>> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <blackdrag@gmx.org
>>>>> <javascript:_e(%7B%7D,'cvml','blackdrag@gmx.org');>>:
>>>>>
>>>>>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>>>>>
>>>>>>> >
>>>>>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>>>>>> I see this method after switching to indy version with invkoedynamic
>>>>>>>
>>>>>>> That is a hint, that you have a lot of invalid callsites going on
>>>>>>>>
>>>>>>> What does it mean? What can I do in order fix it?
>>>>>>>
>>>>>>> Those are all strings?
>>>>>>>>
>>>>>>> Script template is:
>>>>>>>
>>>>>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>>>>>
>>>>>>> dynamically substituted expression could contain anything
>>>>>>>
>>>>>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>>>>>
>>>>>>> user object holds info about a user doing something online. attr1,
>>>>>>> attr2, attr3 could returns various types. Sometimes they return
>>>>>>> object
>>>>>>> and I override returned object equals method.
>>>>>>>
>>>>>>
>>>>>> you wrote there are 20 such replacement cases... executing 20 scripts
>>>>>> is hardly a problem, so it must happen in a loop... do you reuse the
>>>>>> scripts, or do you make a new script each time? Because if you do them new
>>>>>> each time, you have of course never ever an caching going on... but then
>>>>>> the compiler should appear in your measurement as bad factor as well
>>>>>> actually.
>>>>>>
>>>>>> To give an advice I have to understand your setup better first
>>>>>>
>>>>>>
>>>>>> For example, 'user' object has method:
>>>>>>> LastPage getLastPage(){
>>>>>>> def attr = getLatestAttribute(PAGE)
>>>>>>> if(attr){
>>>>>>> lastPage = attr.value
>>>>>>> }
>>>>>>> new LastPage(lastPage: lastPage)
>>>>>>> }
>>>>>>>
>>>>>>> then you can write expression:
>>>>>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>>>>>>> last page mathes provided regex.
>>>>>>>
>>>>>>> Here is LastPage impl:
>>>>>>> class LastPage {
>>>>>>> String lastPage
>>>>>>>
>>>>>>> @Override
>>>>>>> String toString(){
>>>>>>> lastPage
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> int hashCode(){
>>>>>>> 1
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> boolean equals(Object object){
>>>>>>> if(object == null || !(object instanceof String || object
>>>>>>> instanceof LastPage)){
>>>>>>> return false
>>>>>>> }
>>>>>>> if(object instanceof LastPage){
>>>>>>> lastPage.equals(object.lastPage)
>>>>>>> }
>>>>>>> else{
>>>>>>> Pattern pattern = Pattern.compile((String)object)
>>>>>>> lastPage ==~ pattern
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>
>>>>>> and pattern compilation here does not have a negative factor in your
>>>>>> measurement?
>>>>>>
>>>>>> bye Jochen
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Optimizing groovy performace
Posted by John Wagenleitner <jo...@gmail.com>.
Which version of Groovy are you using? I believe in 2.4.6 there was a
minor change in the invalidateCallSites for synchronization, probably not
the issue here but would be curious to know if also a problem on other
version.
Looking your gist for ScriptFactory I notice you use a HashMap. I think
you mentioned that this app is highly concurrent, so I would recommend
using a static final ConcurrentHashMap.
On Thu, Apr 14, 2016 at 2:58 PM, Serega Sheypak <se...@gmail.com>
wrote:
> Here is the screenshot from VisualVM.
> [image: Встроенное изображение 1]
>
> 2016-04-14 23:49 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>
>> Bump
>>
>> 2016-04-12 11:19 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>
>>> Hi, any update here? Could you give me some input for further
>>> investigation?
>>>
>>> 2016-04-11 12:58 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>>
>>>> *>you wrote there are 20 such replacement cases... executing 20 scripts
>>>> is hardly a problem, so it must happen in a loop...*
>>>> Sorry, didn't understand, what do you mean by loop?
>>>>
>>>> *>do you reuse the scripts, or do you make a new script each time? *
>>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>>
>>>> *Hi, here is naive caching factory:*
>>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>>
>>>> *>To give an advice I have to understand your setup better first*
>>>> I run http://storm.apache.org/ topology.
>>>> If you are not familiar with it, assume I have a running thread-safe
>>>> java application. Java application keeps 1000 User objects and 20
>>>> GroovyScripts in memory.
>>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>>> executions each second.
>>>> If script for User object returns true, I do something special.
>>>>
>>>> I try to cache script instances (see my gist).
>>>> User object is just a bean with some logic incapsulated.
>>>>
>>>> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>>>>
>>>>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>>>>
>>>>>> >
>>>>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>>>>> I see this method after switching to indy version with invkoedynamic
>>>>>>
>>>>>> That is a hint, that you have a lot of invalid callsites going on
>>>>>>>
>>>>>> What does it mean? What can I do in order fix it?
>>>>>>
>>>>>> Those are all strings?
>>>>>>>
>>>>>> Script template is:
>>>>>>
>>>>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>>>>
>>>>>> dynamically substituted expression could contain anything
>>>>>>
>>>>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>>>>
>>>>>> user object holds info about a user doing something online. attr1,
>>>>>> attr2, attr3 could returns various types. Sometimes they return object
>>>>>> and I override returned object equals method.
>>>>>>
>>>>>
>>>>> you wrote there are 20 such replacement cases... executing 20 scripts
>>>>> is hardly a problem, so it must happen in a loop... do you reuse the
>>>>> scripts, or do you make a new script each time? Because if you do them new
>>>>> each time, you have of course never ever an caching going on... but then
>>>>> the compiler should appear in your measurement as bad factor as well
>>>>> actually.
>>>>>
>>>>> To give an advice I have to understand your setup better first
>>>>>
>>>>>
>>>>> For example, 'user' object has method:
>>>>>> LastPage getLastPage(){
>>>>>> def attr = getLatestAttribute(PAGE)
>>>>>> if(attr){
>>>>>> lastPage = attr.value
>>>>>> }
>>>>>> new LastPage(lastPage: lastPage)
>>>>>> }
>>>>>>
>>>>>> then you can write expression:
>>>>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>>>>>> last page mathes provided regex.
>>>>>>
>>>>>> Here is LastPage impl:
>>>>>> class LastPage {
>>>>>> String lastPage
>>>>>>
>>>>>> @Override
>>>>>> String toString(){
>>>>>> lastPage
>>>>>> }
>>>>>>
>>>>>> @Override
>>>>>> int hashCode(){
>>>>>> 1
>>>>>> }
>>>>>>
>>>>>> @Override
>>>>>> boolean equals(Object object){
>>>>>> if(object == null || !(object instanceof String || object
>>>>>> instanceof LastPage)){
>>>>>> return false
>>>>>> }
>>>>>> if(object instanceof LastPage){
>>>>>> lastPage.equals(object.lastPage)
>>>>>> }
>>>>>> else{
>>>>>> Pattern pattern = Pattern.compile((String)object)
>>>>>> lastPage ==~ pattern
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>
>>>>> and pattern compilation here does not have a negative factor in your
>>>>> measurement?
>>>>>
>>>>> bye Jochen
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
Here is the screenshot from VisualVM.
[image: Встроенное изображение 1]
2016-04-14 23:49 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> Bump
>
> 2016-04-12 11:19 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>
>> Hi, any update here? Could you give me some input for further
>> investigation?
>>
>> 2016-04-11 12:58 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>>
>>> *>you wrote there are 20 such replacement cases... executing 20 scripts
>>> is hardly a problem, so it must happen in a loop...*
>>> Sorry, didn't understand, what do you mean by loop?
>>>
>>> *>do you reuse the scripts, or do you make a new script each time? *
>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>
>>> *Hi, here is naive caching factory:*
>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>
>>> *>To give an advice I have to understand your setup better first*
>>> I run http://storm.apache.org/ topology.
>>> If you are not familiar with it, assume I have a running thread-safe
>>> java application. Java application keeps 1000 User objects and 20
>>> GroovyScripts in memory.
>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>> executions each second.
>>> If script for User object returns true, I do something special.
>>>
>>> I try to cache script instances (see my gist).
>>> User object is just a bean with some logic incapsulated.
>>>
>>> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>>>
>>>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>>>
>>>>> >
>>>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>>>> I see this method after switching to indy version with invkoedynamic
>>>>>
>>>>> That is a hint, that you have a lot of invalid callsites going on
>>>>>>
>>>>> What does it mean? What can I do in order fix it?
>>>>>
>>>>> Those are all strings?
>>>>>>
>>>>> Script template is:
>>>>>
>>>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>>>
>>>>> dynamically substituted expression could contain anything
>>>>>
>>>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>>>
>>>>> user object holds info about a user doing something online. attr1,
>>>>> attr2, attr3 could returns various types. Sometimes they return object
>>>>> and I override returned object equals method.
>>>>>
>>>>
>>>> you wrote there are 20 such replacement cases... executing 20 scripts
>>>> is hardly a problem, so it must happen in a loop... do you reuse the
>>>> scripts, or do you make a new script each time? Because if you do them new
>>>> each time, you have of course never ever an caching going on... but then
>>>> the compiler should appear in your measurement as bad factor as well
>>>> actually.
>>>>
>>>> To give an advice I have to understand your setup better first
>>>>
>>>>
>>>> For example, 'user' object has method:
>>>>> LastPage getLastPage(){
>>>>> def attr = getLatestAttribute(PAGE)
>>>>> if(attr){
>>>>> lastPage = attr.value
>>>>> }
>>>>> new LastPage(lastPage: lastPage)
>>>>> }
>>>>>
>>>>> then you can write expression:
>>>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>>>>> last page mathes provided regex.
>>>>>
>>>>> Here is LastPage impl:
>>>>> class LastPage {
>>>>> String lastPage
>>>>>
>>>>> @Override
>>>>> String toString(){
>>>>> lastPage
>>>>> }
>>>>>
>>>>> @Override
>>>>> int hashCode(){
>>>>> 1
>>>>> }
>>>>>
>>>>> @Override
>>>>> boolean equals(Object object){
>>>>> if(object == null || !(object instanceof String || object
>>>>> instanceof LastPage)){
>>>>> return false
>>>>> }
>>>>> if(object instanceof LastPage){
>>>>> lastPage.equals(object.lastPage)
>>>>> }
>>>>> else{
>>>>> Pattern pattern = Pattern.compile((String)object)
>>>>> lastPage ==~ pattern
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>
>>>> and pattern compilation here does not have a negative factor in your
>>>> measurement?
>>>>
>>>> bye Jochen
>>>>
>>>
>>>
>>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
Bump
2016-04-12 11:19 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> Hi, any update here? Could you give me some input for further
> investigation?
>
> 2016-04-11 12:58 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>
>> *>you wrote there are 20 such replacement cases... executing 20 scripts
>> is hardly a problem, so it must happen in a loop...*
>> Sorry, didn't understand, what do you mean by loop?
>>
>> *>do you reuse the scripts, or do you make a new script each time? *
>> I suppose I do reuse, please have a look at "ScriptFactory"
>>
>> *Hi, here is naive caching factory:*
>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>
>> *>To give an advice I have to understand your setup better first*
>> I run http://storm.apache.org/ topology.
>> If you are not familiar with it, assume I have a running thread-safe java
>> application. Java application keeps 1000 User objects and 20
>> GroovyScripts in memory.
>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>> executions each second.
>> If script for User object returns true, I do something special.
>>
>> I try to cache script instances (see my gist).
>> User object is just a bean with some logic incapsulated.
>>
>> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>>
>>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>>
>>>> >
>>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>>> I see this method after switching to indy version with invkoedynamic
>>>>
>>>> That is a hint, that you have a lot of invalid callsites going on
>>>>>
>>>> What does it mean? What can I do in order fix it?
>>>>
>>>> Those are all strings?
>>>>>
>>>> Script template is:
>>>>
>>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>>
>>>> dynamically substituted expression could contain anything
>>>>
>>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>>
>>>> user object holds info about a user doing something online. attr1,
>>>> attr2, attr3 could returns various types. Sometimes they return object
>>>> and I override returned object equals method.
>>>>
>>>
>>> you wrote there are 20 such replacement cases... executing 20 scripts is
>>> hardly a problem, so it must happen in a loop... do you reuse the scripts,
>>> or do you make a new script each time? Because if you do them new each
>>> time, you have of course never ever an caching going on... but then the
>>> compiler should appear in your measurement as bad factor as well actually.
>>>
>>> To give an advice I have to understand your setup better first
>>>
>>>
>>> For example, 'user' object has method:
>>>> LastPage getLastPage(){
>>>> def attr = getLatestAttribute(PAGE)
>>>> if(attr){
>>>> lastPage = attr.value
>>>> }
>>>> new LastPage(lastPage: lastPage)
>>>> }
>>>>
>>>> then you can write expression:
>>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>>>> last page mathes provided regex.
>>>>
>>>> Here is LastPage impl:
>>>> class LastPage {
>>>> String lastPage
>>>>
>>>> @Override
>>>> String toString(){
>>>> lastPage
>>>> }
>>>>
>>>> @Override
>>>> int hashCode(){
>>>> 1
>>>> }
>>>>
>>>> @Override
>>>> boolean equals(Object object){
>>>> if(object == null || !(object instanceof String || object
>>>> instanceof LastPage)){
>>>> return false
>>>> }
>>>> if(object instanceof LastPage){
>>>> lastPage.equals(object.lastPage)
>>>> }
>>>> else{
>>>> Pattern pattern = Pattern.compile((String)object)
>>>> lastPage ==~ pattern
>>>> }
>>>> }
>>>> }
>>>>
>>>
>>> and pattern compilation here does not have a negative factor in your
>>> measurement?
>>>
>>> bye Jochen
>>>
>>
>>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
Hi, any update here? Could you give me some input for further investigation?
2016-04-11 12:58 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> *>you wrote there are 20 such replacement cases... executing 20 scripts is
> hardly a problem, so it must happen in a loop...*
> Sorry, didn't understand, what do you mean by loop?
>
> *>do you reuse the scripts, or do you make a new script each time? *
> I suppose I do reuse, please have a look at "ScriptFactory"
>
> *Hi, here is naive caching factory:*
> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>
> *>To give an advice I have to understand your setup better first*
> I run http://storm.apache.org/ topology.
> If you are not familiar with it, assume I have a running thread-safe java
> application. Java application keeps 1000 User objects and 20 GroovyScripts in
> memory.
> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
> executions each second.
> If script for User object returns true, I do something special.
>
> I try to cache script instances (see my gist).
> User object is just a bean with some logic incapsulated.
>
> 2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>
>> On 10.04.2016 20:52, Serega Sheypak wrote:
>>
>>> >
>>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>>> I see this method after switching to indy version with invkoedynamic
>>>
>>> That is a hint, that you have a lot of invalid callsites going on
>>>>
>>> What does it mean? What can I do in order fix it?
>>>
>>> Those are all strings?
>>>>
>>> Script template is:
>>>
>>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>>
>>> dynamically substituted expression could contain anything
>>>
>>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>>
>>> user object holds info about a user doing something online. attr1,
>>> attr2, attr3 could returns various types. Sometimes they return object
>>> and I override returned object equals method.
>>>
>>
>> you wrote there are 20 such replacement cases... executing 20 scripts is
>> hardly a problem, so it must happen in a loop... do you reuse the scripts,
>> or do you make a new script each time? Because if you do them new each
>> time, you have of course never ever an caching going on... but then the
>> compiler should appear in your measurement as bad factor as well actually.
>>
>> To give an advice I have to understand your setup better first
>>
>>
>> For example, 'user' object has method:
>>> LastPage getLastPage(){
>>> def attr = getLatestAttribute(PAGE)
>>> if(attr){
>>> lastPage = attr.value
>>> }
>>> new LastPage(lastPage: lastPage)
>>> }
>>>
>>> then you can write expression:
>>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>>> last page mathes provided regex.
>>>
>>> Here is LastPage impl:
>>> class LastPage {
>>> String lastPage
>>>
>>> @Override
>>> String toString(){
>>> lastPage
>>> }
>>>
>>> @Override
>>> int hashCode(){
>>> 1
>>> }
>>>
>>> @Override
>>> boolean equals(Object object){
>>> if(object == null || !(object instanceof String || object
>>> instanceof LastPage)){
>>> return false
>>> }
>>> if(object instanceof LastPage){
>>> lastPage.equals(object.lastPage)
>>> }
>>> else{
>>> Pattern pattern = Pattern.compile((String)object)
>>> lastPage ==~ pattern
>>> }
>>> }
>>> }
>>>
>>
>> and pattern compilation here does not have a negative factor in your
>> measurement?
>>
>> bye Jochen
>>
>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
Ok, finally the solution is:
count = 139724
mean rate = 8950.43 calls/second
1-minute rate = 2011.54 calls/second
5-minute rate = 426.96 calls/second
15-minute rate = 143.76 calls/second
min = 0.02 milliseconds
max = 24.18 milliseconds
mean = 0.08 milliseconds
stddev = 0.72 milliseconds
median = 0.06 milliseconds
75% <= 0.08 milliseconds
95% <= 0.11 milliseconds
98% <= 0.15 milliseconds
99% <= 0.20 milliseconds
99.9% <= 1.27 milliseconds
Mean rate for indy+InvokeDynamic is about 300-400 ops/sec
Right now rate is 20 times bigger.
What I did?
1. Rewrote part of code to make @CompileStatic work
2. disabled indy+invokeDynamic.
So, next time I'll write the whole core in Java/scala. And only few lines
of code will be in groovy. Really, there are 5 lines of code where I really
need distinguishing groovy features. I had to use them with care.
2016-04-17 16:55 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
> On 17.04.2016 14:36, Serega Sheypak wrote:
>
>> ok, I see weird results.
>> I've wrote a test
>> https://gist.github.com/seregasheypak/d59997dd2d2dfbf82e310b16398239ca
>>
>> Here are metrics for method named "play"
>>
>> TriggerPlayer.play-trigger
>>
> [...]
>
>> Here are metrics for method named "executeHere"
>>
> [...]
>
>> "executeHere" faster in 40 times (acrroding to 99th percentile).
>>
>> I just copy-pasted body of method directly to test...
>>
>> What does it mean, why it works faster?
>>
>
> I am not sure I can understand your code, I miss too much context. But if
> copying the body of the methods directly in the test means you get 40 times
> faster, then this is probably due to inlining not working.
>
> bye Jochen
>
Re: Optimizing groovy performace
Posted by Jochen Theodorou <bl...@gmx.org>.
On 17.04.2016 14:36, Serega Sheypak wrote:
> ok, I see weird results.
> I've wrote a test
> https://gist.github.com/seregasheypak/d59997dd2d2dfbf82e310b16398239ca
>
> Here are metrics for method named "play"
>
> TriggerPlayer.play-trigger
[...]
> Here are metrics for method named "executeHere"
[...]
> "executeHere" faster in 40 times (acrroding to 99th percentile).
>
> I just copy-pasted body of method directly to test...
>
> What does it mean, why it works faster?
I am not sure I can understand your code, I miss too much context. But
if copying the body of the methods directly in the test means you get 40
times faster, then this is probably due to inlining not working.
bye Jochen
Re: Optimizing groovy performace
Posted by Jochen Theodorou <bl...@gmx.org>.
On 17.04.2016 14:41, Serega Sheypak wrote:
> Ok
> try{
> mySuperHeavyMethod()
> }catch(){}
>
> runs 10 times slower that without try/catch...
> why :)
Well... ever tried something like this?
boolean booleanThatIsNeverTrue = false
public void aMethod() {
if (booleanThatIsNeverTrue) { foo() }
bar()
}
volatile boolean booleanThatIsNeverTrue = false
public void aMethod() {
if (booleanThatIsNeverTrue) { foo() }
bar()
}
They behave very different in that the later version is almost not
optimized by hotpot - while in the first case you get almost zero
overhead for aMethod and get almost only the the readings from bar. the
first version can be inlined, the second one not. I did not investigate
how these things behave with try-catch, but I see it very possible, that
there are similar poblems.
bye Jochen
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
Ok
try{
mySuperHeavyMethod()
}catch(){}
runs 10 times slower that without try/catch...
why :)
2016-04-17 14:36 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> ok, I see weird results.
> I've wrote a test
> https://gist.github.com/seregasheypak/d59997dd2d2dfbf82e310b16398239ca
>
> Here are metrics for method named "play"
>
> TriggerPlayer.play-trigger
>
> count = 83043
>
> mean rate = 395.52 calls/second
>
> 1-minute rate = 555.30 calls/second
>
> 5-minute rate = 217.78 calls/second
>
> 15-minute rate = 82.92 calls/second
>
> min = 0.29 milliseconds
>
> max = 12.98 milliseconds
>
> mean = 1.59 milliseconds
>
> stddev = 1.08 milliseconds
>
> median = 1.39 milliseconds
>
> 75% <= 2.46 milliseconds
>
> 95% <= 3.14 milliseconds
>
> 98% <= 3.44 milliseconds
>
> 99% <= 3.76 milliseconds
>
> 99.9% <= 12.19 milliseconds
>
>
> Here are metrics for method named "executeHere"
>
> count = 81927
>
> mean rate = 503.31 calls/second
>
> 1-minute rate = 587.41 calls/second
>
> 5-minute rate = 221.08 calls/second
>
> 15-minute rate = 83.42 calls/second
>
> min = 0.06 milliseconds
>
> max = 0.47 milliseconds
>
> mean = 0.08 milliseconds
>
> stddev = 0.03 milliseconds
>
> median = 0.08 milliseconds
>
> 75% <= 0.09 milliseconds
>
> 95% <= 0.13 milliseconds
>
> 98% <= 0.16 milliseconds
>
> 99% <= 0.17 milliseconds
>
> 99.9% <= 0.44 milliseconds
>
>
> "executeHere" faster in 40 times (acrroding to 99th percentile).
>
> I just copy-pasted body of method directly to test...
>
> What does it mean, why it works faster?
>
> 2016-04-16 21:52 GMT+02:00 Serega Sheypak <se...@gmail.com>:
>
>> > Firstly sorry for not answering so fast. I have not so much spare time
>> these days :( And your question requires some thought
>> No problem, I really happy to get any input from community members side.
>> if I now assume this will start 1000 threads, each with its own cache,
>> then you get 20k compilations,
>> It goes a bit different way. I'll explain how does my installation works.
>> Storm starts 4 JVM per node. Each JVM has 5 threads to run my code.
>> Each thread has it's own script factory.
>> Thread that runs code lives forever (until you kill it or apocalypses
>> happens. Storm can restart thread in case of failure)
>>
>> > Doing all that unreflection in the indy part and creating those
>> MethodHandles always takes a little time.
>> It happens only right, correct?
>> I don't have thousands of threads. There are dozens of them.
>>
>> >Is it somehow possible to get a small running program showing the
>> problem?
>> Hm... I'll try to extract it.
>>
>> 2016-04-16 13:27 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>>
>>> Firstly sorry for not answering so fast. I have not so much spare time
>>> these days :( And your question requires some thought
>>>
>>> On 11.04.2016 12:58, Serega Sheypak wrote:
>>> [...]
>>>
>>>> *>do you reuse the scripts, or do you make a new script each time?*
>>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>>
>>>> *Hi, here is naive caching factory:*
>>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>>
>>>> *>To give an advice I have to understand your setup better first*
>>>> I run http://storm.apache.org/ topology.
>>>> If you are not familiar with it, assume I have a running thread-safe
>>>> java application. Java application keeps 1000 User objects and 20
>>>> GroovyScripts in memory.
>>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>>> executions each second.
>>>> If script for User object returns true, I do something special.
>>>>
>>>> I try to cache script instances (see my gist).
>>>> User object is just a bean with some logic incapsulated.
>>>>
>>>
>>> if I now assume this will start 1000 threads, each with its own cache,
>>> then you get 20k compilations, as well as 20k first time executions. Doing
>>> all that unreflection in the indy part and creating those MethodHandles
>>> always takes a little time. So it might very well be, that this is the
>>> problem, but not sure what we could do then.
>>>
>>> Is it somehow possible to get a small running program showing the
>>> problem? Then I could check myseld what really happens. JVisualVM helps a
>>> bit, but is imho no proper replacement for a real profiler.
>>>
>>> bye Jochen
>>>
>>>
>>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
ok, I see weird results.
I've wrote a test
https://gist.github.com/seregasheypak/d59997dd2d2dfbf82e310b16398239ca
Here are metrics for method named "play"
TriggerPlayer.play-trigger
count = 83043
mean rate = 395.52 calls/second
1-minute rate = 555.30 calls/second
5-minute rate = 217.78 calls/second
15-minute rate = 82.92 calls/second
min = 0.29 milliseconds
max = 12.98 milliseconds
mean = 1.59 milliseconds
stddev = 1.08 milliseconds
median = 1.39 milliseconds
75% <= 2.46 milliseconds
95% <= 3.14 milliseconds
98% <= 3.44 milliseconds
99% <= 3.76 milliseconds
99.9% <= 12.19 milliseconds
Here are metrics for method named "executeHere"
count = 81927
mean rate = 503.31 calls/second
1-minute rate = 587.41 calls/second
5-minute rate = 221.08 calls/second
15-minute rate = 83.42 calls/second
min = 0.06 milliseconds
max = 0.47 milliseconds
mean = 0.08 milliseconds
stddev = 0.03 milliseconds
median = 0.08 milliseconds
75% <= 0.09 milliseconds
95% <= 0.13 milliseconds
98% <= 0.16 milliseconds
99% <= 0.17 milliseconds
99.9% <= 0.44 milliseconds
"executeHere" faster in 40 times (acrroding to 99th percentile).
I just copy-pasted body of method directly to test...
What does it mean, why it works faster?
2016-04-16 21:52 GMT+02:00 Serega Sheypak <se...@gmail.com>:
> > Firstly sorry for not answering so fast. I have not so much spare time
> these days :( And your question requires some thought
> No problem, I really happy to get any input from community members side.
> if I now assume this will start 1000 threads, each with its own cache,
> then you get 20k compilations,
> It goes a bit different way. I'll explain how does my installation works.
> Storm starts 4 JVM per node. Each JVM has 5 threads to run my code.
> Each thread has it's own script factory.
> Thread that runs code lives forever (until you kill it or apocalypses
> happens. Storm can restart thread in case of failure)
>
> > Doing all that unreflection in the indy part and creating those
> MethodHandles always takes a little time.
> It happens only right, correct?
> I don't have thousands of threads. There are dozens of them.
>
> >Is it somehow possible to get a small running program showing the
> problem?
> Hm... I'll try to extract it.
>
> 2016-04-16 13:27 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
>
>> Firstly sorry for not answering so fast. I have not so much spare time
>> these days :( And your question requires some thought
>>
>> On 11.04.2016 12:58, Serega Sheypak wrote:
>> [...]
>>
>>> *>do you reuse the scripts, or do you make a new script each time?*
>>> I suppose I do reuse, please have a look at "ScriptFactory"
>>>
>>> *Hi, here is naive caching factory:*
>>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>>
>>> *>To give an advice I have to understand your setup better first*
>>> I run http://storm.apache.org/ topology.
>>> If you are not familiar with it, assume I have a running thread-safe
>>> java application. Java application keeps 1000 User objects and 20
>>> GroovyScripts in memory.
>>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>>> executions each second.
>>> If script for User object returns true, I do something special.
>>>
>>> I try to cache script instances (see my gist).
>>> User object is just a bean with some logic incapsulated.
>>>
>>
>> if I now assume this will start 1000 threads, each with its own cache,
>> then you get 20k compilations, as well as 20k first time executions. Doing
>> all that unreflection in the indy part and creating those MethodHandles
>> always takes a little time. So it might very well be, that this is the
>> problem, but not sure what we could do then.
>>
>> Is it somehow possible to get a small running program showing the
>> problem? Then I could check myseld what really happens. JVisualVM helps a
>> bit, but is imho no proper replacement for a real profiler.
>>
>> bye Jochen
>>
>>
>
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
> Firstly sorry for not answering so fast. I have not so much spare time
these days :( And your question requires some thought
No problem, I really happy to get any input from community members side.
if I now assume this will start 1000 threads, each with its own cache, then
you get 20k compilations,
It goes a bit different way. I'll explain how does my installation works.
Storm starts 4 JVM per node. Each JVM has 5 threads to run my code.
Each thread has it's own script factory.
Thread that runs code lives forever (until you kill it or apocalypses
happens. Storm can restart thread in case of failure)
> Doing all that unreflection in the indy part and creating those
MethodHandles always takes a little time.
It happens only right, correct?
I don't have thousands of threads. There are dozens of them.
>Is it somehow possible to get a small running program showing the problem?
Hm... I'll try to extract it.
2016-04-16 13:27 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
> Firstly sorry for not answering so fast. I have not so much spare time
> these days :( And your question requires some thought
>
> On 11.04.2016 12:58, Serega Sheypak wrote:
> [...]
>
>> *>do you reuse the scripts, or do you make a new script each time?*
>> I suppose I do reuse, please have a look at "ScriptFactory"
>>
>> *Hi, here is naive caching factory:*
>> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>>
>> *>To give an advice I have to understand your setup better first*
>> I run http://storm.apache.org/ topology.
>> If you are not familiar with it, assume I have a running thread-safe
>> java application. Java application keeps 1000 User objects and 20
>> GroovyScripts in memory.
>> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
>> executions each second.
>> If script for User object returns true, I do something special.
>>
>> I try to cache script instances (see my gist).
>> User object is just a bean with some logic incapsulated.
>>
>
> if I now assume this will start 1000 threads, each with its own cache,
> then you get 20k compilations, as well as 20k first time executions. Doing
> all that unreflection in the indy part and creating those MethodHandles
> always takes a little time. So it might very well be, that this is the
> problem, but not sure what we could do then.
>
> Is it somehow possible to get a small running program showing the problem?
> Then I could check myseld what really happens. JVisualVM helps a bit, but
> is imho no proper replacement for a real profiler.
>
> bye Jochen
>
>
Re: Optimizing groovy performace
Posted by Jochen Theodorou <bl...@gmx.org>.
Firstly sorry for not answering so fast. I have not so much spare time
these days :( And your question requires some thought
On 11.04.2016 12:58, Serega Sheypak wrote:
[...]
> *>do you reuse the scripts, or do you make a new script each time?*
> I suppose I do reuse, please have a look at "ScriptFactory"
>
> *Hi, here is naive caching factory:*
> https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
>
> *>To give an advice I have to understand your setup better first*
> I run http://storm.apache.org/ topology.
> If you are not familiar with it, assume I have a running thread-safe
> java application. Java application keeps 1000 User objects and 20
> GroovyScripts in memory.
> I run 1000 (users) *20 (script: def evaluateExpression(Map context))
> executions each second.
> If script for User object returns true, I do something special.
>
> I try to cache script instances (see my gist).
> User object is just a bean with some logic incapsulated.
if I now assume this will start 1000 threads, each with its own cache,
then you get 20k compilations, as well as 20k first time executions.
Doing all that unreflection in the indy part and creating those
MethodHandles always takes a little time. So it might very well be, that
this is the problem, but not sure what we could do then.
Is it somehow possible to get a small running program showing the
problem? Then I could check myseld what really happens. JVisualVM helps
a bit, but is imho no proper replacement for a real profiler.
bye Jochen
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
*>you wrote there are 20 such replacement cases... executing 20 scripts is
hardly a problem, so it must happen in a loop...*
Sorry, didn't understand, what do you mean by loop?
*>do you reuse the scripts, or do you make a new script each time? *
I suppose I do reuse, please have a look at "ScriptFactory"
*Hi, here is naive caching factory:*
https://gist.github.com/seregasheypak/2d7f5a16fa31018c8212754c8e94f9a4
*>To give an advice I have to understand your setup better first*
I run http://storm.apache.org/ topology.
If you are not familiar with it, assume I have a running thread-safe java
application. Java application keeps 1000 User objects and 20 GroovyScripts in
memory.
I run 1000 (users) *20 (script: def evaluateExpression(Map context))
executions each second.
If script for User object returns true, I do something special.
I try to cache script instances (see my gist).
User object is just a bean with some logic incapsulated.
2016-04-11 11:30 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
> On 10.04.2016 20:52, Serega Sheypak wrote:
>
>> >
>> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
>> I see this method after switching to indy version with invkoedynamic
>>
>> That is a hint, that you have a lot of invalid callsites going on
>>>
>> What does it mean? What can I do in order fix it?
>>
>> Those are all strings?
>>>
>> Script template is:
>>
>> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>>
>> dynamically substituted expression could contain anything
>>
>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>
>> user object holds info about a user doing something online. attr1,
>> attr2, attr3 could returns various types. Sometimes they return object
>> and I override returned object equals method.
>>
>
> you wrote there are 20 such replacement cases... executing 20 scripts is
> hardly a problem, so it must happen in a loop... do you reuse the scripts,
> or do you make a new script each time? Because if you do them new each
> time, you have of course never ever an caching going on... but then the
> compiler should appear in your measurement as bad factor as well actually.
>
> To give an advice I have to understand your setup better first
>
>
> For example, 'user' object has method:
>> LastPage getLastPage(){
>> def attr = getLatestAttribute(PAGE)
>> if(attr){
>> lastPage = attr.value
>> }
>> new LastPage(lastPage: lastPage)
>> }
>>
>> then you can write expression:
>> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
>> last page mathes provided regex.
>>
>> Here is LastPage impl:
>> class LastPage {
>> String lastPage
>>
>> @Override
>> String toString(){
>> lastPage
>> }
>>
>> @Override
>> int hashCode(){
>> 1
>> }
>>
>> @Override
>> boolean equals(Object object){
>> if(object == null || !(object instanceof String || object
>> instanceof LastPage)){
>> return false
>> }
>> if(object instanceof LastPage){
>> lastPage.equals(object.lastPage)
>> }
>> else{
>> Pattern pattern = Pattern.compile((String)object)
>> lastPage ==~ pattern
>> }
>> }
>> }
>>
>
> and pattern compilation here does not have a negative factor in your
> measurement?
>
> bye Jochen
>
Re: Optimizing groovy performace
Posted by Jochen Theodorou <bl...@gmx.org>.
On 10.04.2016 20:52, Serega Sheypak wrote:
> > org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet
> I see this method after switching to indy version with invkoedynamic
>
>> That is a hint, that you have a lot of invalid callsites going on
> What does it mean? What can I do in order fix it?
>
>>Those are all strings?
> Script template is:
>
> |defevaluateExpression(Mapcontext){defuser =context.user %s }|
>
> dynamically substituted expression could contain anything
>
> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>
> user object holds info about a user doing something online. attr1,
> attr2, attr3 could returns various types. Sometimes they return object
> and I override returned object equals method.
you wrote there are 20 such replacement cases... executing 20 scripts is
hardly a problem, so it must happen in a loop... do you reuse the
scripts, or do you make a new script each time? Because if you do them
new each time, you have of course never ever an caching going on... but
then the compiler should appear in your measurement as bad factor as
well actually.
To give an advice I have to understand your setup better first
> For example, 'user' object has method:
> LastPage getLastPage(){
> def attr = getLatestAttribute(PAGE)
> if(attr){
> lastPage = attr.value
> }
> new LastPage(lastPage: lastPage)
> }
>
> then you can write expression:
> user.lastPage == 'somePageRegEx/.*' when you want to get true if user
> last page mathes provided regex.
>
> Here is LastPage impl:
> class LastPage {
> String lastPage
>
> @Override
> String toString(){
> lastPage
> }
>
> @Override
> int hashCode(){
> 1
> }
>
> @Override
> boolean equals(Object object){
> if(object == null || !(object instanceof String || object
> instanceof LastPage)){
> return false
> }
> if(object instanceof LastPage){
> lastPage.equals(object.lastPage)
> }
> else{
> Pattern pattern = Pattern.compile((String)object)
> lastPage ==~ pattern
> }
> }
> }
and pattern compilation here does not have a negative factor in your
measurement?
bye Jochen
Re: Optimizing groovy performace
Posted by Serega Sheypak <se...@gmail.com>.
> org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doC
allSiteTargetSet
I see this method after switching to indy version with invkoedynamic
> That is a hint, that you have a lot of invalid callsites going on
What does it mean? What can I do in order fix it?
>Those are all strings?
Script template is:
def evaluateExpression(Map context){
def user = context.user
%s}
dynamically substituted expression could contain anything
user.attr1 == '1' || user.attr2 == '2' || user.attr3 = '3'
user object holds info about a user doing something online. attr1, attr2,
attr3 could returns various types. Sometimes they return object and I
override returned object equals method.
For example, 'user' object has method:
LastPage getLastPage(){
def attr = getLatestAttribute(PAGE)
if(attr){
lastPage = attr.value
}
new LastPage(lastPage: lastPage)
}
then you can write expression:
user.lastPage == 'somePageRegEx/.*' when you want to get true if user last
page mathes provided regex.
Here is LastPage impl:
class LastPage {
String lastPage
@Override
String toString(){
lastPage
}
@Override
int hashCode(){
1
}
@Override
boolean equals(Object object){
if(object == null || !(object instanceof String || object
instanceof LastPage)){
return false
}
if(object instanceof LastPage){
lastPage.equals(object.lastPage)
}
else{
Pattern pattern = Pattern.compile((String)object)
lastPage ==~ pattern
}
}
}
2016-04-10 20:37 GMT+02:00 Jochen Theodorou <bl...@gmx.org>:
> On 10.04.2016 19:38, Serega Sheypak wrote:
>
>> I'm working on groovy code performance optimization. I've used jvisualvm
>> to connect to running application and gather CPU samples. Samples say
>> that/org.codehaus.groovy.reflection.CachedMethod.inkove/ takes the most
>> CPU time. I don't see any other application methods in samples.
>>
>> What is the right way to dig into /CachedMethod.invoke/ and understand
>> what code lines really give performance penalties?
>>
>> Then I switched to groovy-indy version, now I see different results in
>> jviusalvm.
>>
>
> I don't think your analysis hits the spot. If you have
>
> def foo(){ bar() }
>
> then there is a time spent in foo itself and the total time for foo, which
> is the time spent in foo itself and the time spent in bar.
> imagine CachedMethod.invoke to be like that. Are we then really talking
> about time spent in invoke itself or does it include the time added through
> the method called from there?
>
> [...]
>
>> Most of CPU time spent in
>>
>> *org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet*
>>
>
> That is a hint, that you have a lot of invalid callsites going on
>
> I have a code that dynamically composes groovy script. Script template
>> looks this way:
>>
>> |def evaluateExpression(Map context){def user = context.user %s }|
>>
>> where *%s* replaced with
>>
>> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>>
>> There is a set (20 in total) of replacements have taken from Databases.
>> The code gets replacements from DB, creates GroovyScript and evaluates
>> it. I suppose the bottleneck is in the script execution. What is the
>> right way to fix it?
>>
>
> Those are all strings?
>
> bye Jochen
>
>
>
Re: Optimizing groovy performace
Posted by Jochen Theodorou <bl...@gmx.org>.
On 10.04.2016 19:38, Serega Sheypak wrote:
> I'm working on groovy code performance optimization. I've used jvisualvm
> to connect to running application and gather CPU samples. Samples say
> that/org.codehaus.groovy.reflection.CachedMethod.inkove/ takes the most
> CPU time. I don't see any other application methods in samples.
>
> What is the right way to dig into /CachedMethod.invoke/ and understand
> what code lines really give performance penalties?
>
> Then I switched to groovy-indy version, now I see different results in
> jviusalvm.
I don't think your analysis hits the spot. If you have
def foo(){ bar() }
then there is a time spent in foo itself and the total time for foo,
which is the time spent in foo itself and the time spent in bar.
imagine CachedMethod.invoke to be like that. Are we then really talking
about time spent in invoke itself or does it include the time added
through the method called from there?
[...]
> Most of CPU time spent in
> *org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet*
That is a hint, that you have a lot of invalid callsites going on
> I have a code that dynamically composes groovy script. Script template
> looks this way:
>
> |def evaluateExpression(Map context){def user = context.user %s }|
>
> where *%s* replaced with
>
> |user.attr1 =='1'||user.attr2 =='2'||user.attr3 ='3'|
>
> There is a set (20 in total) of replacements have taken from Databases.
> The code gets replacements from DB, creates GroovyScript and evaluates
> it. I suppose the bottleneck is in the script execution. What is the
> right way to fix it?
Those are all strings?
bye Jochen
Re: Optimizing groovy performace
Posted by emmanuel r <go...@gmail.com>.
Take a look at BTrace. You can create a script to get method execution
times and run it against your app. Especially if you tell it to log all
method calls; keeping in mind the performance penalty, of course. I wrote
my script to write out in csv format, and then I opened the log in Libre
Office for analysis.
On Apr 10, 2016 1:39 PM, "Serega Sheypak" <se...@gmail.com> wrote:
I'm working on groovy code performance optimization. I've used jvisualvm to
connect to running application and gather CPU samples. Samples say that
*org.codehaus.groovy.reflection.CachedMethod.inkove* takes the most CPU
time. I don't see any other application methods in samples.
What is the right way to dig into *CachedMethod.invoke* and understand what
code lines really give performance penalties?
Then I switched to groovy-indy version, now I see different results in
jviusalvm.
Probably, my problem a bit similar to this thread: Call site caching faster
than invokedynamic?
<http://groovy.329449.n5.nabble.com/Call-site-caching-faster-than-invokedynamic-tp5721659p5721674.html>
Most of CPU time spent in
*org.codehaus.groovy.vmplugin.v7.Selector$MethodSelector.doCallSiteTargetSet*
I have a code that dynamically composes groovy script. Script template
looks this way:
def evaluateExpression(Map context){
def user = context.user
%s}
where *%s* replaced with
user.attr1 == '1' || user.attr2 == '2' || user.attr3 = '3'
There is a set (20 in total) of replacements have taken from Databases. The
code gets replacements from DB, creates GroovyScript and evaluates it. I
suppose the bottleneck is in the script execution. What is the right way to
fix it?