You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jmeter.apache.org by bu...@apache.org on 2019/05/15 15:56:41 UTC
[Bug 63442] New: Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Bug ID: 63442
Summary: Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionPrope
rty' doesn
Product: JMeter
Version: 5.1.1
Hardware: PC
Status: NEW
Severity: normal
Priority: P2
Component: HTTP
Assignee: issues@jmeter.apache.org
Reporter: eric.wolfe@csgi.com
Target Milestone: JMETER_5.1.1
We need to simulate automated third-party systems hitting our public APIs. We
don't want wait times between the the calls, because these are not humans that
need to think but other computers which don't wait very long between actions. I
noticed that the script is taking 3x longer than I would expect from the API
call time totals. After studying the debug logging I noticed that 'merging in
class ...CollectionProperty' is taking a long time to complete, usually 7
seconds but can be over 10 seconds using -n (CLI). I tried running in GUI mode.
With a visualizer attached this operation usually took a few milliseconds (100
or less). Max is about 3 seconds. Without the visualizer in GUI mode 7 seconds
seems to be the norm.
Note this behavior is also observed on a Windows Server 2016 build server.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #6 from Eric Wolfe <er...@csgi.com> ---
I was cloning the repo when I got called for a family problem. I'm not able to
try the patch today. I'm not a java guy, but I will try to figure out how
patch and build tomorrow. Thank you very much for taking a look so quickly!
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Eric Wolfe <er...@csgi.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
Status|NEEDINFO |RESOLVED
--- Comment #16 from Eric Wolfe <er...@csgi.com> ---
You nailed it. Sorry for the rookie mistake. It would have been helpful if the
debug logs would have included '200 millisecond wait starting' messages.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale.
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Eric Wolfe <er...@csgi.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |eric.wolfe@csgi.com
OS| |All
Summary|Header Manager 'merging in |Header Manager 'merging in
|class |class
|org.apache.jmeter.testeleme |org.apache.jmeter.testeleme
|nt.property.CollectionPrope |nt.property.CollectionPrope
|rty' doesn |rty' doesn't scale.
--- Comment #1 from Eric Wolfe <er...@csgi.com> ---
working on attaching files...
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Eric Wolfe <er...@csgi.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #7 from Felix Schumacher <fe...@internetallee.de> ---
@Eric did you find time to check?
Author: fschumacher
Date: Fri May 17 17:00:32 2019
New Revision: 1859442
URL: http://svn.apache.org/viewvc?rev=1859442&view=rev
Log:
Reduce scanning for LogParser implementations in AccessLogSamplerBeanInfo.
I am not sure, that it fixes 63442, but it seems nonsense to re-scan all
classes
over and over again. The classpath doesn't change that often.
Bugzilla 63442
Modified:
jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/AccessLogSamplerBeanInfo.java
jmeter/trunk/xdocs/changes.xml
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Hello,
Thanks for report.
I am not sure to understand your report.
1/ Are you saying response times are better in GUI mode vs NON GUI mode ?
2/ How are you measuring those response times for this merge operation ?
3/ Can you give more details on your testing approach ?
Thank you
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale in
GUI mode
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Header Manager 'merging in |Header Manager 'merging in
|class |class
|org.apache.jmeter.testeleme |org.apache.jmeter.testeleme
|nt.property.CollectionPrope |nt.property.CollectionPrope
|rty' doesn't scale. |rty' doesn't scale in GUI
| |mode
CC| |p.mouawad@ubik-ingenierie.c
| |om
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Eric Wolfe <er...@csgi.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #14 from Eric Wolfe <er...@csgi.com> ---
I have added a sample.zip to the shared box account. It has a sample jmx which
uses an include controller who only job is to ping Google. I was able to
reproduce the issue using this simple setup. There is some noise in the log
from groovy complaining about something which I couldn't figure out, but the
results of the merging in the class were the same.
I tried some thread dumps following the instructions found here:
https://stackoverflow.com/questions/3843461/jmeter-load-test-thread-dump
No luck on the thread dumps. The only data was 'errorlevel=1'
I was using java 8. I switched to 12 jdk and reconfirmed the issue. Then
turned on the flight recorder for 60 seconds. The results are uploaded as
sample.jfr.
Also ran with the line set
VERBOSE_GC=-Xlog:gc*,gc+age=trace,gc+heap=debug:file=gc_jmeter_%%p.log in the
jmeter.bat uncommented in order to turn on GC logging. Results are uploaded as
gc_jmeter_<XXX>.log files.
As to your question: "Can you tell us more about the settings you used? Like
the size of JVM heap, the number of threads you used. Which visualizer did you
activate/add to make it run faster?"
I have no idea what the JVM heap is set to - default. One thread is running
for the group. When the 'View Result Tree' listener was attached it seemed to
run faster. Not sure that result is reproducible.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #13 from Felix Schumacher <fe...@internetallee.de> ---
Can you tell us more about the settings you used? Like the size of JVM heap,
the number of threads you used. Which visualizer did you activate/add to make
it run faster?
I always look for GC problems when I have unexplainable pauses. Try to enable
GC logging and look for full GCs.
If you use Java 11 you can enable the JVM flightrecorder, which might provide
useful insight to where the time is spent.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #5 from Felix Schumacher <fe...@internetallee.de> ---
Created attachment 36588
--> https://bz.apache.org/bugzilla/attachment.cgi?id=36588&action=edit
Only search once for classes in AccessLogSamperBeanInfo
I think this comes from the AccessLogSamplerBeanInfo, that is refetching the
list of classes for different implementations again and again.
Can you try the attached patch, if it helps your use case? Note I haven't tried
it, I will go to sleep now :)
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #9 from Eric Wolfe <er...@csgi.com> ---
The patch did not help:
2019-05-17 14:01:57,158 DEBUG o.a.j.t.p.AbstractProperty: merging in class
org.apache.jmeter.testelement.property.CollectionProperty
2019-05-17 14:02:03,468 DEBUG o.a.j.g.GuiPackage: Updating current node View
Results Tree
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #15 from Felix Schumacher <fe...@internetallee.de> ---
Could it be, that you misunderstood the scope of the timers in JMeter?
Every timer on the same level as the samplers will be added together. That
probably explains, why you observe long sleep intervals.
If you want to add a special timer to one of your samplers, add it as a child
to that sampler, or use a Flow Control Action.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #12 from Eric Wolfe <er...@csgi.com> ---
working on a sample - won't have it ready until next week.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #4 from Eric Wolfe <er...@csgi.com> ---
files were too large to attach.
Logs, JTLs and schematic views are located here.
https://app.box.com/s/2k1o958g70bxox7cave8i3bm6288ipk5
I don't really care about GUI mode; the concern is that command line run takes
too much time. I mentioned GUI mode trying to provide context on what I tried
before opening this ticket up.
While creating a 'small sample' of the code for this incident I noticed that
the class merge took just 1 millisecond in the log file.
So, for small JMX runs there is no problem.
However for larger JMX with several components you can see in the log file that
the class merge takes 7-8 seconds before every single API call.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #11 from Felix Schumacher <fe...@internetallee.de> ---
And could you take a few thread dumps when the test is between those two log
lines?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #10 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
hello,
Would you be able to provide a simple test plan reproducing isseu ?
Thanks
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #8 from Eric Wolfe <er...@csgi.com> ---
In between planning meetings yesterday and today I got the latest JDK and ANT,
Got eclipse run and built Jmeter locally.
When I run the tests I get an error complaining about many groovy scripts:
2019-05-17 12:50:23,723 ERROR o.a.j.p.j.s.JSR223Sampler: Problem in JSR223
script [Script] - Generate SubscriberFirstName SubscriberLastName
SubscriberLogin, message: javax.script.ScriptException:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 1: unable to resolve class
org.apache.commons.lang.RandomStringUtils
@ line 1, column 1.
import org.apache.commons.lang.RandomStringUtils;
I believe this is due to a namespace mismatch as I see everything in the jmeter
code is using org.apache.commons.lang3.StringUtils but that is just a guess at
this time.
Changing the scripts to use lang3 caused a different error.
So I'm thinking of either just replacing the many scripts with hardcodes right
now just to test the results as I don't know what the right way to solve this
problem is being new to the Java world.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
--- Comment #3 from Eric Wolfe <er...@csgi.com> ---
files were too large to attach.
Logs, JTLs and schematic views are located here.
https://app.box.com/s/2k1o958g70bxox7cave8i3bm6288ipk5
I don't really care about GUI mode; the concern is that command line run takes
too much time. I mentioned GUI mode trying to provide context on what I tried
before opening this ticket up.
While creating a 'small sample' of the code for this incident I noticed that
the class merge took just 1 millisecond in the log file.
So, for small JMX runs there is no problem.
However for larger JMX with several components you can see in the log file that
the class merge takes 7-8 seconds before every single API call.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 63442] Header Manager 'merging in class
org.apache.jmeter.testelement.property.CollectionProperty' doesn't scale
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63442
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Header Manager 'merging in |Header Manager 'merging in
|class |class
|org.apache.jmeter.testeleme |org.apache.jmeter.testeleme
|nt.property.CollectionPrope |nt.property.CollectionPrope
|rty' doesn't scale in GUI |rty' doesn't scale
|mode |
--
You are receiving this mail because:
You are the assignee for the bug.