You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@velocity.apache.org by Bojan Smojver <bo...@binarix.com> on 2001/10/10 01:30:13 UTC
Benchmark results
Hi there Velocity people! More good news...
Just did some benchmarks on my 1 GHz Athlon system, Velocity versus JSP.
The environment is Apache 1.3.20 + mod_jk 1.2.0 + Tomcat 3.3 from CVS.
The commands used were:
--------------------------------------------
ab -c 20 -t 20 http://somedomain/somepath/velocitypage.vm
--------------------------------------------
and
--------------------------------------------
ab -c 20 -t 20 http://somedomain/somepath/javaserverpage.jsp
--------------------------------------------
Both pages do pretty much the same thing - they are simple inquiry
forms. Both tests were run 3 times on a new instance of TC 3.3 and best
results for each page type were taken.
Velocity:
--------------------------------------------
Document Path: /contact/inquiry.vm
Document Length: 10302 bytes
Concurrency Level: 20
Time taken for tests: 20.001 seconds
Complete requests: 3224
Failed requests: 2326
(Connect: 0, Length: 2326, Exceptions: 0)
Total transferred: 34130847 bytes
HTML transferred: 33215616 bytes
Requests per second: 161.19
Transfer rate: 1706.46 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 16 108
Processing: 5 106 761
Total: 5 122 869
--------------------------------------------
JSP:
--------------------------------------------
Document Path: /home/inquiry.jsp
Document Length: 10165 bytes
Concurrency Level: 20
Time taken for tests: 20.009 seconds
Complete requests: 3172
Failed requests: 2787
(Connect: 0, Length: 2787, Exceptions: 0)
Total transferred: 33081630 bytes
HTML transferred: 32337661 bytes
Requests per second: 158.53
Transfer rate: 1653.34 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 6 46
Processing: 4 118 1118
Total: 4 124 1164
--------------------------------------------
Velocity wins by a narrow margin, but one would expect that 'generated
code of JSP' would have to be much faster then 'interpreted code of VM'.
Reality is a bit different though ;-)
Velocity pages were delivered by my PumpServlet, which isn't all that
smart. It creates new VelocityContext and new ByteArrayOutputStream
buffer at every doPost() execution, something JSP doesn't have to worry
about at all.
To paraphrase Jon, YMTD...
Bojan
PS. The connection 'failures' have to do with the document length change
due to jsessionid being encoded into them, which then varies the page in
length. It is a bug in ab. All connections actually go through OK.
Re: slightly off-topic: Re: Benchmark results
Posted by Bojan Smojver <bo...@binarix.com>.
Joerg Erdmenger wrote:
>
> Hi,
>
> > --------------------------------------------
> > ab -c 20 -t 20 http://somedomain/somepath/velocitypage.vm
> > --------------------------------------------
>
> as a Java/velocity newbie (albeit having followed that list for a couple of
> month now): What is that ab command/program you use to gather benchmark
> results?
Comes with Apache... Apache HTTP server benchmarking tool.
Bojan
slightly off-topic: Re: Benchmark results
Posted by Joerg Erdmenger <jo...@woerd.com>.
Hi,
> --------------------------------------------
> ab -c 20 -t 20 http://somedomain/somepath/velocitypage.vm
> --------------------------------------------
as a Java/velocity newbie (albeit having followed that list for a couple of
month now): What is that ab command/program you use to gather benchmark
results?
Joerg
Re: Benchmark results
Posted by Nick Bauman <ni...@cortexity.com>.
This is exactly consistent with my own benchmarks using ab as well. The two
technologies are about equally as fast.
> Hi there Velocity people! More good news...
>
> Just did some benchmarks on my 1 GHz Athlon system, Velocity versus
> JSP. The environment is Apache 1.3.20 + mod_jk 1.2.0 + Tomcat 3.3 from
> CVS. The commands used were:
>
> --------------------------------------------
> ab -c 20 -t 20 http://somedomain/somepath/velocitypage.vm
> --------------------------------------------
>
> and
>
> --------------------------------------------
> ab -c 20 -t 20 http://somedomain/somepath/javaserverpage.jsp
> --------------------------------------------
>
> Both pages do pretty much the same thing - they are simple inquiry
> forms. Both tests were run 3 times on a new instance of TC 3.3 and best
> results for each page type were taken.
>
> Velocity:
>
> --------------------------------------------
> Document Path: /contact/inquiry.vm
> Document Length: 10302 bytes
>
> Concurrency Level: 20
> Time taken for tests: 20.001 seconds
> Complete requests: 3224
> Failed requests: 2326
> (Connect: 0, Length: 2326, Exceptions: 0)
> Total transferred: 34130847 bytes
> HTML transferred: 33215616 bytes
> Requests per second: 161.19
> Transfer rate: 1706.46 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 16 108
> Processing: 5 106 761
> Total: 5 122 869
> --------------------------------------------
>
> JSP:
>
> --------------------------------------------
> Document Path: /home/inquiry.jsp
> Document Length: 10165 bytes
>
> Concurrency Level: 20
> Time taken for tests: 20.009 seconds
> Complete requests: 3172
> Failed requests: 2787
> (Connect: 0, Length: 2787, Exceptions: 0)
> Total transferred: 33081630 bytes
> HTML transferred: 32337661 bytes
> Requests per second: 158.53
> Transfer rate: 1653.34 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 6 46
> Processing: 4 118 1118
> Total: 4 124 1164
> --------------------------------------------
>
> Velocity wins by a narrow margin, but one would expect that 'generated
> code of JSP' would have to be much faster then 'interpreted code of
> VM'. Reality is a bit different though ;-)
>
> Velocity pages were delivered by my PumpServlet, which isn't all that
> smart. It creates new VelocityContext and new ByteArrayOutputStream
> buffer at every doPost() execution, something JSP doesn't have to worry
> about at all.
>
> To paraphrase Jon, YMTD...
>
> Bojan
>
> PS. The connection 'failures' have to do with the document length
> change due to jsessionid being encoded into them, which then varies the
> page in length. It is a bug in ab. All connections actually go through
> OK.
--
Will America never be the same after September 11? I would phrase the
question differently. Will America again be so captured by fear as to cast a
net of suspicion over growing numbers of its own citizens?
Re: Benchmark results
Posted by Bojan Smojver <bo...@binarix.com>.
"Geir Magnusson Jr." wrote:
>
> Excellent. I want to post this to help debunk 'the myth'. Maybe there is a
> little more we can do to round this out. In my own tests, I found the gap
> bigger when you started doing interesting things like accessing data through
> tags, looping, etc, so I am interested to see what your page looks like.
>
> Can you
>
> 1) Send the jsp and vm?
Sure. I'll make two pages that are almost a mirror copy of one another:
JSP for VM. Then the comparison will be even closer and JSP people won't
be able to scream foul. Once they're ready, I'll post them to the list.
The pages I used have similar functionality, but are far from being a
mirror copy of one another.
Actually, the JSP page I initially used was smaller then the VM page and
the difference in kb/s was much bigger in favour of VM's.
> 2) Send all of the results, not just the best?
Yes. Actually I should run just one test but for a longer period of
time. Maybe 5 minutes or something.
> Are you interested in
>
> 1) Repeating with TC3.2.x and TC4 ? (for grins)
Once I send the pages, then anyone can verify for themselves.
> 2) Repeating if we modify the pages?
As above.
> I am assuming in each case you hit the page once to let JSP compile the page
> and let velocity parse and cache.
Yep.
> I assume caching was on in Velocity.
Yep.
> > Velocity wins by a narrow margin, but one would expect that 'generated
> > code of JSP' would have to be much faster then 'interpreted code of VM'.
> > Reality is a bit different though ;-)
>
> Well, it's "interpreted" once, and then each rendering is executing compiled
> code, right? There are a few method calls jumping around the syntax tree,
> but I suspect the generated JSP might do similar things. (I have looked at
> the jasper code a little, but not deeply...)
A JSP page is a servlet. It loads as a class and executes numerous
println's (amongst other things). Velocity is still 'interpreted'
although 'prepared' in cache because it's has to go through the tree -
there are no direct println calls that just blast characters out like in
JSP's - that's what I meant.
And, in my servlet there is an extra step where I first put everything
into ByteArrayOutputStream and then later writeTo the servlet output
stream.
> > Velocity pages were delivered by my PumpServlet, which isn't all that
> > smart. It creates new VelocityContext and new ByteArrayOutputStream
> > buffer at every doPost() execution, something JSP doesn't have to worry
> > about at all.
>
> Right - you can always pool the BAOS's like we do in Velocity servlet if you
> really care. I don't think the new VelocityContext is all that expensive
> either.
Did some benchmarks and the difference was not that big, so I opted for
simplicity.
This benchmark was just to indicate to myself that I'm not doing
something incredibly stupid in my servlet. It was far from scientific or
anything like that. I was actually expecting JSP to win hands down...
Bojan
Re: Benchmark results
Posted by "Geir Magnusson Jr." <ge...@optonline.net>.
Excellent. I want to post this to help debunk 'the myth'. Maybe there is a
little more we can do to round this out. In my own tests, I found the gap
bigger when you started doing interesting things like accessing data through
tags, looping, etc, so I am interested to see what your page looks like.
Can you
1) Send the jsp and vm?
2) Send all of the results, not just the best?
Are you interested in
1) Repeating with TC3.2.x and TC4 ? (for grins)
2) Repeating if we modify the pages?
I am assuming in each case you hit the page once to let JSP compile the page
and let velocity parse and cache.
I assume caching was on in Velocity.
Inline...
On 10/9/01 7:30 PM, "Bojan Smojver" <bo...@binarix.com> wrote:
> Hi there Velocity people! More good news...
>
> Just did some benchmarks on my 1 GHz Athlon system, Velocity versus JSP.
> The environment is Apache 1.3.20 + mod_jk 1.2.0 + Tomcat 3.3 from CVS.
> The commands used were:
>
> --------------------------------------------
> ab -c 20 -t 20 http://somedomain/somepath/velocitypage.vm
> --------------------------------------------
>
> and
>
> --------------------------------------------
> ab -c 20 -t 20 http://somedomain/somepath/javaserverpage.jsp
> --------------------------------------------
>
> Both pages do pretty much the same thing - they are simple inquiry
> forms. Both tests were run 3 times on a new instance of TC 3.3 and best
> results for each page type were taken.
>
> Velocity:
>
> --------------------------------------------
> Document Path: /contact/inquiry.vm
> Document Length: 10302 bytes
>
> Concurrency Level: 20
> Time taken for tests: 20.001 seconds
> Complete requests: 3224
> Failed requests: 2326
> (Connect: 0, Length: 2326, Exceptions: 0)
> Total transferred: 34130847 bytes
> HTML transferred: 33215616 bytes
> Requests per second: 161.19
> Transfer rate: 1706.46 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 16 108
> Processing: 5 106 761
> Total: 5 122 869
> --------------------------------------------
>
> JSP:
>
> --------------------------------------------
> Document Path: /home/inquiry.jsp
> Document Length: 10165 bytes
>
> Concurrency Level: 20
> Time taken for tests: 20.009 seconds
> Complete requests: 3172
> Failed requests: 2787
> (Connect: 0, Length: 2787, Exceptions: 0)
> Total transferred: 33081630 bytes
> HTML transferred: 32337661 bytes
> Requests per second: 158.53
> Transfer rate: 1653.34 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 6 46
> Processing: 4 118 1118
> Total: 4 124 1164
> --------------------------------------------
>
> Velocity wins by a narrow margin, but one would expect that 'generated
> code of JSP' would have to be much faster then 'interpreted code of VM'.
> Reality is a bit different though ;-)
Well, it's "interpreted" once, and then each rendering is executing compiled
code, right? There are a few method calls jumping around the syntax tree,
but I suspect the generated JSP might do similar things. (I have looked at
the jasper code a little, but not deeply...)
>
> Velocity pages were delivered by my PumpServlet, which isn't all that
> smart. It creates new VelocityContext and new ByteArrayOutputStream
> buffer at every doPost() execution, something JSP doesn't have to worry
> about at all.
Right - you can always pool the BAOS's like we do in Velocity servlet if you
really care. I don't think the new VelocityContext is all that expensive
either.
geir
> To paraphrase Jon, YMTD...
>
> Bojan
>
> PS. The connection 'failures' have to do with the document length change
> due to jsessionid being encoded into them, which then varies the page in
> length. It is a bug in ab. All connections actually go through OK.
--
Geir Magnusson Jr. geirm@optonline.net
System and Software Consulting
"Whoever would overthrow the liberty of a nation must begin by subduing the
freeness of speech." - Benjamin Franklin