tomcat vs orion vs weblogic testing results (Load Runner)

[Manne Anliot <Ma...@framfab.se>]

Hi all..

I'm doing work load testing on a webapplication built on javascript, .JSP:s
and EJB:s. We've load tested on many different setups and are getting some
interesting results:

Simulated number of users: 25
Computer setup: NT4 sp6, 1 GB RAM, Pentium III ~600 MHz
Tomcat setup: 1 ajp12-worker.. (We need out-of-process workers for
scalability)

<snip>
setup1: webserver - apache 1.3.x, jsp - tomcat 3.2b6 (mod_jk.dll)
setup2: webserver - weblogic, jsp - weblogic (latest versions as of this
date)
setup3: webserver - orion, jsp - orion (latest versions as of this date)
<snip>

setup2 and setup3 are 8 (eight) times faster in our tests (Load Runner).
Even if we suspected tomcat to be a bit slower, this is truly remarkable..
We've tried optimizing Tomcat by:
1. using ajp13 protocol - failed, we're just getting exceptions from tomcat
2. reloadable=false - of course
3. fiddling with JVM memory settings - minor improvements

Any ideas on what's wrong here? This must be a configuration problem. Have
anyone else found the same results?

Regards,
Manne Anliot.

Re: tomcat vs orion vs weblogic testing results (Load Runner)

[Miles Daffin <mi...@cwcom.net>]

> Hi all..
>
> I'm doing work load testing on a webapplication built on javascript,
.JSP:s
> and EJB:s. We've load tested on many different setups and are getting some
> interesting results:
>
> Simulated number of users: 25
> Computer setup: NT4 sp6, 1 GB RAM, Pentium III ~600 MHz
> Tomcat setup: 1 ajp12-worker.. (We need out-of-process workers for
> scalability)
>
> <snip>
> setup1: webserver - apache 1.3.x, jsp - tomcat 3.2b6 (mod_jk.dll)
> setup2: webserver - weblogic, jsp - weblogic (latest versions as of this
> date)
> setup3: webserver - orion, jsp - orion (latest versions as of this date)
> <snip>
>
> setup2 and setup3 are 8 (eight) times faster in our tests (Load Runner).
> Even if we suspected tomcat to be a bit slower, this is truly remarkable..
> We've tried optimizing Tomcat by:
> 1. using ajp13 protocol - failed, we're just getting exceptions from
tomcat
> 2. reloadable=false - of course
> 3. fiddling with JVM memory settings - minor improvements
>
> Any ideas on what's wrong here? This must be a configuration problem. Have
> anyone else found the same results?

We had similar results here.

There is nothing wrong - Orion, I am reasonably sure, is simply a better
product.
The drawback is that it aint free. But it's cheaper than WebSphere, for
example.
Tomcat is the cheapest, and when you pay peanuts....

Miles

RE: tomcat vs orion vs weblogic testing results (Load Runner)

[John Bateman <jo...@cyberworldgroup.com>]

Hi

Would you be willling to post the results of your tests here so we can see
what, and where, the bottlenecks are?

Thanks


> -----Original Message-----
> From: Manne Anliot [mailto:Manne.Anliot@framfab.se]
> Sent: Monday, November 20, 2000 5:15 AM
> To: 'tomcat-user@jakarta.apache.org'
> Subject: tomcat vs orion vs weblogic testing results (Load Runner)
>
>
> Hi all..
>
> I'm doing work load testing on a webapplication built on
> javascript, .JSP:s
> and EJB:s. We've load tested on many different setups and are
> getting some
> interesting results:
>
> Simulated number of users: 25
> Computer setup: NT4 sp6, 1 GB RAM, Pentium III ~600 MHz
> Tomcat setup: 1 ajp12-worker.. (We need out-of-process workers for
> scalability)
>
> <snip>
> setup1: webserver - apache 1.3.x, jsp - tomcat 3.2b6 (mod_jk.dll)
> setup2: webserver - weblogic, jsp - weblogic (latest versions
> as of this
> date)
> setup3: webserver - orion, jsp - orion (latest versions as of
> this date)
> <snip>
>
> setup2 and setup3 are 8 (eight) times faster in our tests
> (Load Runner).
> Even if we suspected tomcat to be a bit slower, this is truly
> remarkable..
> We've tried optimizing Tomcat by:
> 1. using ajp13 protocol - failed, we're just getting
> exceptions from tomcat
> 2. reloadable=false - of course
> 3. fiddling with JVM memory settings - minor improvements
>
> Any ideas on what's wrong here? This must be a configuration
> problem. Have
> anyone else found the same results?
>
> Regards,
> Manne Anliot.

RE: tomcat vs orion vs weblogic testing results (Load Runner)

[Jarek Skreta <Ja...@nesscomp.com>]

Hi there,

What you've got may be consistent with some other benchmarks. Here is the
link to the Orion web site where they publish some of their results.

http://www.orionserver.com/benchmarks/benchmark.html

They put lot of caveats on their test but the trend is similar to yours.

Regds,

Jarek Skreta


> -----Original Message-----
> From: Manne Anliot [mailto:Manne.Anliot@framfab.se]
> Sent: 20 November 2000 10:15
> To: 'tomcat-user@jakarta.apache.org'
> Subject: tomcat vs orion vs weblogic testing results (Load Runner)
>
>
> Hi all..
>
> I'm doing work load testing on a webapplication built on
> javascript, .JSP:s
> and EJB:s. We've load tested on many different setups and are getting some
> interesting results:
>
> Simulated number of users: 25
> Computer setup: NT4 sp6, 1 GB RAM, Pentium III ~600 MHz
> Tomcat setup: 1 ajp12-worker.. (We need out-of-process workers for
> scalability)
>
> <snip>
> setup1: webserver - apache 1.3.x, jsp - tomcat 3.2b6 (mod_jk.dll)
> setup2: webserver - weblogic, jsp - weblogic (latest versions as of this
> date)
> setup3: webserver - orion, jsp - orion (latest versions as of this date)
> <snip>
>
> setup2 and setup3 are 8 (eight) times faster in our tests (Load Runner).
> Even if we suspected tomcat to be a bit slower, this is truly remarkable..
> We've tried optimizing Tomcat by:
> 1. using ajp13 protocol - failed, we're just getting exceptions
> from tomcat
> 2. reloadable=false - of course
> 3. fiddling with JVM memory settings - minor improvements
>
> Any ideas on what's wrong here? This must be a configuration problem. Have
> anyone else found the same results?
>
> Regards,
> Manne Anliot.