You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Philippe Mouawad <ph...@gmail.com> on 2016/01/21 15:37:24 UTC
JEXL3 Performance degradation compared to 2.1.1
Hello,
I integrated today jexl3 to JMeter trunk:
-
https://github.com/apache/jmeter/commit/e1bfb7bb01169278d57f4fea7bf0cfaeba44ac50
I made some rapid benchmark using JMeter.
My Test plan contains only 1 thread Group with a Debug Sampler as a child:
Debug Sampler has Name field set to: DS-${__jexl3(3+2,RESULT)}
Using Java 8u45 on a Mac OSX 10.9.5 / 2.7 GHz Intel Core i7 / 16 Gb 1600
MHz DD3
1 Thread:
JEXL2 : Generate Summary Results = 20000 in 00:00:00 = 75471.7/s Avg:
0 Min: 0 Max: 1 Err: 0 (0.00%)
JEXL3 : Generate Summary Results = 20000 in 00:00:01 = 32102.7/s Avg:
0 Min: 0 Max: 1 Err: 0 (0.00%)
4 Threads:
JEXL2 : Generate Summary Results = 80000 in 00:00:00 = 285714.3/s Avg:
0 Min: 0 Max: 1 Err: 0 (0.00%)
JEXL3 : Generate Summary Results = 80000 in 00:00:01 = 103092.8/s Avg:
0 Min: 0 Max: 1 Err: 0 (0.00%)
As you can see the throughput decreased:
1 Thread : -57%
4 Threads : -68%
My benchmark is not as clean as a Java Micro Benchmark , but I lack of time
:-)
I thought you might be interested.
Regards
Philippe M.
@philmdot
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by henrib <he...@apache.org>.
Hello Philippe;Interesting results indeed.I've been trying to interpret /
replicate them but JMeter is a bit too complex for me to understand what the
benchmark is achieving and what is measured.Is it 20000 loops of an
expression that is '3 + 2' ? And the same is performed with 4 threads in
parallel using the same engine ?Any (easy) way to extract just the evaluated
expressions / scripts with minimal dependencies (unit, jexl2/3)?Thanks for
the info,CheersHenrib
--
View this message in context: http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682109.html
Sent from the Commons - User mailing list archive at Nabble.com.
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by henrib <he...@apache.org>.
Hi Philippe;
I think it's ok to leave the default as debug(true) since it is easy (and
often necessary) to create a 'custom' JEXL instance be it for contexts,
namespaces and/or arithmetic. It is convenient to provide JexlInfo instances
when creating scripts to track potential errors; in that case, there is no
(little) cost to debug(true).
In the general case, overloading the JexlArithmetic (ala JMeterArithmetic)
is a good and efficient workaround to JEXL-186; you might even want to
customize it further (overloaded versions of add(...) for instance).
Re-publishing a version/patch for that one bug fix seems overkill imho,
especially with a decent/easy workaround. But if you are willing to go
through the release hoops, please do. :-)
Cheers,
Henrib
--
View this message in context: http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682135.html
Sent from the Commons - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi Henri,
I just answered in JIRA :-)
It's now ok with JMeterArithmetic and debug(false)
Bug shouldn't debug be false by default ?
Second question, what is the impact of keeping JMeterArithmetic ? Or do you
plan a new release of commons-jexl3 ?
Anyway, thanks for your very fast support ! Great !
Regards
On Fri, Jan 22, 2016 at 10:14 PM, henrib <he...@apache.org> wrote:
> Hi Philippe;
> Default debug mode in JEXL3 seems like the other big slowdown cause.
> Check JEXL-186 for an updated engine startup sequence :-).
> Cheers,
> Henrib
>
>
>
> --
> View this message in context:
> http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682123.html
> Sent from the Commons - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
--
Cordialement.
Philippe Mouawad.
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by henrib <he...@apache.org>.
Hi Philippe;
Default debug mode in JEXL3 seems like the other big slowdown cause.
Check JEXL-186 for an updated engine startup sequence :-).
Cheers,
Henrib
--
View this message in context: http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682123.html
Sent from the Commons - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi Henri,
I attached a simple Main class to :
https://issues.apache.org/jira/browse/JEXL-186
and the results.
Regards
Philippe
On Fri, Jan 22, 2016 at 8:19 PM, henrib <he...@apache.org> wrote:
> Hi;
> Since performance is the same with the workaround, we've got
> another/different performance issue. Is there any easy way to replicate
> your
> test - or at least its essence - ? What does your benchmark do ? What are
> the evaluated expressions / scripts ?
>
> Back too your note: the 'size' method is discovered through introspection
> as
> an overload of the 'size' operator; this creates an entry in a cache for
> that arithmetic class avoiding to re-discover the operator overloads
> (Uberspect) on each execution. So, no, this method is not called; it is
> just
> meant as a workaround of the bug.
>
> Regards
>
>
>
> --
> View this message in context:
> http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682115.html
> Sent from the Commons - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
--
Cordialement.
Philippe Mouawad.
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by henrib <he...@apache.org>.
Hi;
Since performance is the same with the workaround, we've got
another/different performance issue. Is there any easy way to replicate your
test - or at least its essence - ? What does your benchmark do ? What are
the evaluated expressions / scripts ?
Back too your note: the 'size' method is discovered through introspection as
an overload of the 'size' operator; this creates an entry in a cache for
that arithmetic class avoiding to re-discover the operator overloads
(Uberspect) on each execution. So, no, this method is not called; it is just
meant as a workaround of the bug.
Regards
--
View this message in context: http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682115.html
Sent from the Commons - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi,
Thanks for the suggestion.
I tested it but performance results are the same.
Note I don't understand the fix as size method doesn't have this signature
in ancestor and I don't understand neither why it would be called in this
context, even after changing the signature to match the parent, Debugging
it, it is not called.
Regards
On Friday, January 22, 2016, henrib <he...@apache.org> wrote:
> Hi again;
> It seems you've found a real performance regression.
> I've been playing some more and it seems the logic around the operator
> overloading resolution/caching is flawed in the simplest case, i.e. when
> there is no overload.
> As a workaround, you may want to try using an overloading JexlArithmetic as
> is:
> {code}
>
> public static class JMeterArithmetic extends JexlArithmetic {
> public JMeterArithmetic(boolean astrict) {
> super(astrict);
> }
>
> /**
> * A workaround to create an operator overload
> * @param jma an improbable parameter class
> * @return 1
> */
> public int size(JMeterArithmetic jma) {
> return 1;
> }
> }
> {code}
>
> And use it at engine creation time:
> {code}
>
> JexlEngine jexl = new JexlBuilder()
> .cache(512)
> .silent(true)
> .strict(true)
> .arithmetic(new JMeterArithmetic(true))
> .create();
> {code}
>
> Let us know how it goes (if/when you find the time).
>
>
>
>
>
> --
> View this message in context:
> http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682110.html
> Sent from the Commons - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org <javascript:;>
> For additional commands, e-mail: user-help@commons.apache.org
> <javascript:;>
>
>
--
Cordialement.
Philippe Mouawad.
Re: JEXL3 Performance degradation compared to 2.1.1
Posted by henrib <he...@apache.org>.
Hi again;
It seems you've found a real performance regression.
I've been playing some more and it seems the logic around the operator
overloading resolution/caching is flawed in the simplest case, i.e. when
there is no overload.
As a workaround, you may want to try using an overloading JexlArithmetic as
is:
{code}
public static class JMeterArithmetic extends JexlArithmetic {
public JMeterArithmetic(boolean astrict) {
super(astrict);
}
/**
* A workaround to create an operator overload
* @param jma an improbable parameter class
* @return 1
*/
public int size(JMeterArithmetic jma) {
return 1;
}
}
{code}
And use it at engine creation time:
{code}
JexlEngine jexl = new JexlBuilder()
.cache(512)
.silent(true)
.strict(true)
.arithmetic(new JMeterArithmetic(true))
.create();
{code}
Let us know how it goes (if/when you find the time).
--
View this message in context: http://apache-commons.680414.n4.nabble.com/JEXL3-Performance-degradation-compared-to-2-1-1-tp4682086p4682110.html
Sent from the Commons - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org