You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jmeter-dev@jakarta.apache.org by Jordi Salvat i Alabart <js...@atg.com> on 2003/11/26 19:03:33 UTC
Parser performance tests, some conclusions, some questions
Hi.
I've been running some tests. Test procedure:
1/ Unpack the attached .tgz in your apache root (apache and JMeter in
the same machine).
2/ Edit bin/jmeter to contain:
««
#! /bin/sh
ARGS="-Xms256m -Xmx256m -XX:NewSize=128m -XX:MaxNewSize=128m
-XX:SurvivorRatio=8 -XX:PermSize=64m -XX:MaxPermSize=64m
-XX:MaxLiveObjectEvacuationRatio=10% -XX:TargetSurvivorRatio=80%
-XX:MaxTenuringThreshold=3"
ARGS="$ARGS -verbose:gc -XX:+PrintTenuringDistribution"
time java $ARGS -jar `dirname $0`/ApacheJMeter.jar "$@"
»»
These parameters are probably suboptimal for regular usage, but were
optimal for this particular test (total GC times were negligible in all
tests).
3/ Edit jmeter.properties to choose a parser
4/ Run the attached script with:
bin/jmeter -n -t TestHttpSamplerFullPerformance.jmx
And here's the results:
HtmlParser:
60 scavenges
real 2m24.800s
user 1m58.440s
sys 0m1.780s
Regexp:
14 scavenges
real 2m14.892s
user 1m52.820s
sys 0m2.020s
JTidy:
34 scavenges
real 3m41.518s
user 3m8.690s
sys 0m7.900s
Regexp updated (factor out the '<'):
14 scavenges
real 1m4.732s
user 0m42.380s
sys 0m2.950s
JTidy updated (one single DOM scan):
34 scavenges
real 3m33.466s
user 3m3.810s
sys 0m7.140s
Notes:
- I'm only showing one result, but I ran several to check they were
reasonably repeatable.
- Each scavenge represents arouns 102MB of garbage (the size of the
Eden) cleaned away.
- None of the tests ever hit a Full GC -- and I don't think they will in
human-reasonable times.
- The test does 2000 samples, each sample representing 15 HTTP requests
(14 for HtmlParserHTMLParser).
Conclusions:
- The new RegexpHTMLParser makes above 30 samples, or 450 requests per
second. That's about what I got on simple samplers (without parsing), so
I'd say it's nonsense to spend more time optimizing RegexpHTMLParser
(although there's opportunities, such as using Awk instead of Perl regexps).
- HtmlParserHTMLParser could get better in memory usage (and probably
gain some speed in the process: creating objects also costs CPU cycles),
but I don't think it can catch up with the RegexpHTMLParser.
- JTidyHTMLParser falls well behind in speed, although it's better in
garbage generation.
Questions:
- ORO PerlMatcher Regexps seems to be tremendously sensible in terms of
performance. That's scary. We should find a way to integrate a minimal
performance test in the unit tests, but: how to factor out the CPU
speed? A BogoMeter?
- Is it worth keeping JTidyHTMLParser around? The only reason I see is
that HtmlParser seems still too buggy to rely on it...
--
Salut,
Jordi.
Re: Parser performance tests, some conclusions, some questions
Posted by peter lin <jm...@yahoo.com>.
thanks for running those tests. interesting results and good food for thought. unless we are in a hurry to release the next version I am planning on debugging htmlparser and figuring where the bug is.
the code for the images should be isolated in 2 classes, so tracking down the bug should be straight forward. unfortunately I won't have time until friday/saturday at the earliest. We can continue to use JTidy as the default until I fix Htmlparser.
does that sound reasonable?
peter
Jordi Salvat i Alabart <js...@atg.com> wrote:
Hi.
I've been running some tests. Test procedure:
1/ Unpack the attached .tgz in your apache root (apache and JMeter in
the same machine).
2/ Edit bin/jmeter to contain:
��
#! /bin/sh
ARGS="-Xms256m -Xmx256m -XX:NewSize=128m -XX:MaxNewSize=128m
-XX:SurvivorRatio=8 -XX:PermSize=64m -XX:MaxPermSize=64m
-XX:MaxLiveObjectEvacuationRatio=10% -XX:TargetSurvivorRatio=80%
-XX:MaxTenuringThreshold=3"
ARGS="$ARGS -verbose:gc -XX:+PrintTenuringDistribution"
time java $ARGS -jar `dirname $0`/ApacheJMeter.jar "$@"
��
These parameters are probably suboptimal for regular usage, but were
optimal for this particular test (total GC times were negligible in all
tests).
3/ Edit jmeter.properties to choose a parser
4/ Run the attached script with:
bin/jmeter -n -t TestHttpSamplerFullPerformance.jmx
And here's the results:
HtmlParser:
60 scavenges
real 2m24.800s
user 1m58.440s
sys 0m1.780s
Regexp:
14 scavenges
real 2m14.892s
user 1m52.820s
sys 0m2.020s
JTidy:
34 scavenges
real 3m41.518s
user 3m8.690s
sys 0m7.900s
Regexp updated (factor out the '<'):
14 scavenges
real 1m4.732s
user 0m42.380s
sys 0m2.950s
JTidy updated (one single DOM scan):
34 scavenges
real 3m33.466s
user 3m3.810s
sys 0m7.140s
Notes:
- I'm only showing one result, but I ran several to check they were
reasonably repeatable.
- Each scavenge represents arouns 102MB of garbage (the size of the
Eden) cleaned away.
- None of the tests ever hit a Full GC -- and I don't think they will in
human-reasonable times.
- The test does 2000 samples, each sample representing 15 HTTP requests
(14 for HtmlParserHTMLParser).
Conclusions:
- The new RegexpHTMLParser makes above 30 samples, or 450 requests per
second. That's about what I got on simple samplers (without parsing), so
I'd say it's nonsense to spend more time optimizing RegexpHTMLParser
(although there's opportunities, such as using Awk instead of Perl regexps).
- HtmlParserHTMLParser could get better in memory usage (and probably
gain some speed in the process: creating objects also costs CPU cycles),
but I don't think it can catch up with the RegexpHTMLParser.
- JTidyHTMLParser falls well behind in speed, although it's better in
garbage generation.
Questions:
- ORO PerlMatcher Regexps seems to be tremendously sensible in terms of
performance. That's scary. We should find a way to integrate a minimal
performance test in the unit tests, but: how to factor out the CPU
speed? A BogoMeter?
- Is it worth keeping JTidyHTMLParser around? The only reason I see is
that HtmlParser seems still too buggy to rely on it...
--
Salut,
Jordi.
org.apache.jmeter.config.gui.ArgumentsPanel
org.apache.jmeter.config.Arguments
Argument List
true
org.apache.jmeter.control.gui.TestPlanGui
org.apache.jmeter.testelement.TestPlan
false
Test Plan
true
false
org.apache.jmeter.threads.gui.ThreadGroupGui
0
org.apache.jmeter.threads.ThreadGroup
org.apache.jmeter.control.gui.LoopControlPanel
200
org.apache.jmeter.control.LoopController
Loop Controller
true
false
Thread Group
0
continue
true
10
false
1
/test-httpsamplerfull/Yahoo!.htm
org.apache.jmeter.protocol.http.sampler.HTTPSampler
/test-httpsamplerfull/Yahoo!.htm
GET
true
http
true
true
true
org.apache.jmeter.protocol.http.gui.HTTPArgumentsPanel
org.apache.jmeter.config.Arguments
Argument List
true
org.apache.jmeter.protocol.http.control.gui.HttpTestSampleGui
HTTP Request
localhost
org.apache.jmeter.visualizers.StatVisualizer
org.apache.jmeter.reporters.ResultCollector
Aggregate Report
false
false
org.apache.jmeter.visualizers.ViewResultsFullVisualizer
org.apache.jmeter.reporters.ResultCollector
View Results Tree
false
false
> ATTACHMENT part 3 application/x-gtar name=test-httpsamplerfull.tgz
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-dev-help@jakarta.apache.org
---------------------------------
Do you Yahoo!?
Free Pop-Up Blocker - Get it now