daytrader benchmark results

[Maxim Berkultsev <ma...@gmail.com>]

Hello all,

I've looked through Daytrader workload results and analysis for Geronimo
published at

http://www-1.ibm.com/support/docview.wss?uid=swg27006724

For the bar diagrams, which show Geronimo and 'target' server indicators,
immediate questions are:

- what is the measure unit for values in Y-axis?
- how are the values for 'target' performance received?

Also, could someone clarify if the performance data was collected on a
client or a server side?

Thank you in advance for the answer.

 --
Best regards,
Maxim Berkultsev, Intel Middleware Products Division

Re: daytrader benchmark results

[Matt Hogstrom <ma...@hogstrom.org>]

Hi Maxim,

Comments inline...

Maxim Berkultsev wrote:
> Hello all,
> 
> I've looked through Daytrader workload results and analysis for Geronimo
> published at
> 
> http://www-1.ibm.com/support/docview.wss?uid=swg27006724
> 
> For the bar diagrams, which show Geronimo and 'target' server indicators,
> immediate questions are:
> 
> - what is the measure unit for values in Y-axis?

The Y-Axis represents transactions per second.

> - how are the values for 'target' performance received?

One problem with performance measurments is that they typically incite many 
people to claim victory or cry foul depending on the results.  This is also 
complicated by the fact that many commercial products have a no benchmark 
publish clause in their licenses.  In order to have something to compare 
Geronimo to as well as avoid the inevitable fallout of naming a comparison point 
I did a series of measurments on other Open Source and commercial AppServer 
products.  I thook these results and created the competitive target metric.

The competitive target metric is simply my estimation of a respectable 
performance measurement.  Meaning, as we achieve that metric one should be happy 
with the results.

Note that these numbers are pretty old.  I have a new set of data that I've been 
saying will be out for a while.  I guess its time to actually make that happen. 
  So, for purposes of consumption I would wait a few days for the new report to 
surface.  We'll post it on our site.


> 
> Also, could someone clarify if the performance data was collected on a
> client or a server side?
>

All the TPS metrics were gathered from the client.  The goal was to get the 
server to 100% CPU utilization.

Also, if you guys are willing we could use some of the newer Potomac 3.6Ghz chips :)

> Thank you in advance for the answer.
> 
>  --
> Best regards,
> Maxim Berkultsev, Intel Middleware Products Division
>