Load driving scripts for Daytrader

[Slava McDougald <sl...@gmail.com>]

Hi all,

It is great to have the Daytrader sample in the Geronimo projects.
Daytrader is an end-to-end application that allows a full functional
testing of Geronimo as well as measuring the performance of the
server.

However, in order to do a performance test, a users need to drive the
application with some kind of a load driving software.  And for that
they need to spend time and effort to develop scripts for driving the
sample.  Very often, they do not have the time or the desire to invest
in learning all the specifics about a new load driving tool and/or the
application just to be able to perform a test.  But if they are
provided will the scripts they will gladly use them.

I think that it would be very useful for both developers and users of
Daytrader to package the benchmark with scripts for load driving it.

There are a couple of load driving software project in the open source
arena that we can use for that purpose.  Recently, I looked into a
couple of them (for my own testing purposes).  I think that the Apache
Jackarta JMeter (http://jakarta.apache.org/jmeter/) would be suitable
choice for driving Daytrader.


Thoughts?  Suggestions?

Slava

Re: Load driving scripts for Daytrader

[Slava McDougald <sl...@gmail.com>]

Hi Gianny,

I did look at the Grinder 3 as well.  It is  less CPU intensive than
JMeter but not significantly (at least for the testing I did).

However, in my opinion JMeter is more mature product.  There are a lot
of features that a still "under development" in Grinder 3. It seemed
to me that not many people are working on developing those.

Plus, to take full advantage of the Grinder 3 capabilities a user
needs to install and run jython and very likely deal with versioning
issues...

At the present time, JMeter has larger and more active development
community and therefore better support.

But creating  scripts for Grinder3 is pretty easy and straight forward.

So, each tools has its pros and cons.  It is a matter on deciding on
one of them and go with it.

Or create two sets of script...one for Grinder and one for JMeter...

Thoughts?

Slava
On 10/24/06, Gianny Damour <gi...@optusnet.com.au> wrote:
> Hello,
>
> I think that the Grinder definitively is worth to consider. I have
> used it a couple of times and I think that it is "better" than JMeter.
>
> Thanks,
> Gianny
>
>
> On 25/10/2006, at 3:44 AM, Slava McDougald wrote:
>
> > Hi Matt,
> >
> >
> > Well, my short evaluation of JMeter confirmed your observation: JMeter
> > is definately not a great performing load driving tool.
> >
> > However, I tried/read about a few other open source load drivers and
> > unfortunately I didn't find anything that performs really well.
> >
> > Since JMeter has the largest and  the most active development
> > community that provides a good user's support, I think that we can
> > start using it to drive Daytrader (at least until we find something
> > better).
> >
> > Slava
> >
> >
> > On 10/24/06, Slava McDougald <sl...@gmail.com> wrote:
> >> Hi Piuysh,
> >>
> >> Writing your own driver is certainly an option but it wouldn't be my
> >> preferred option.  You are right; it takes a lot of time, efforts,
> >> good design and implementation to build a good load driving tool.
> >>
> >> In my opinion, it is better to just use whatever it is available out
> >> in the open source space.  Something like JMeter for example has a
> >> large community of developers working on improving and supporting it
> >> as w ell as a lot of users that can provide hits/feedback on its use.
> >> In addition, the source code is available to download and can change
> >> according to your needs.  That is why I proposed to use the existing
> >> load driver instead of developing one from scratch.
> >>
> >> For my testing, I used just a simple JMeter scripts, mostly for my
> >> own
> >> educational purposes.  However, I am working on creating a more
> >> complex JMeter workload script for Daytrader.
> >>
> >> Once completed, I will share it with you (the Geronimo development
> >> community).
> >>
> >> Then we can work together on changing/improving it and maybe package
> >> it with Daytrader sample.
> >>
> >> Sounds good?
> >>
> >> Slava
> >>
> >>
> >> On 10/24/06, Matt Hogstrom <ma...@hogstrom.org> wrote:
> >> > If you can get JMeter going that would be great.  I did try it
> >> and it used
> >> > an excessive amount of CPU but I may not have had it configured
> >> well.
> >> >
> >> >
> >> > On Oct 24, 2006, at 11:22 AM, Piyush Agarwal wrote:
> >> >
> >> >
> >> > Hi all,
> >> >
> >> >
> >> >
> >> >
> >> > It is great to have the Daytrader sample in the Geronimo projects.
> >> >
> >> > Daytrader is an end-to-end application that allows a full
> >> functional
> >> >
> >> > testing of Geronimo as well as measuring the performance of the
> >> >
> >> > server.
> >> >
> >> >
> >> >
> >> >
> >> > However, in order to do a performance test, a users need to
> >> drive the
> >> >
> >> > application with some kind of a load driving software.  And for
> >> that
> >> >
> >> > they need to spend time and effort to develop scripts for
> >> driving the
> >> >
> >> > sample.  Very often, they do not have the time or the desire to
> >> invest
> >> >
> >> > in learning all the specifics about a new load driving tool and/
> >> or the
> >> >
> >> > application just to be able to perform a test.  But if they are
> >> >
> >> > provided will the scripts they will gladly use them.
> >> >
> >> > Matt Hogstrom
> >> > matt@hogstrom.org
> >> >
> >> >
> >> >
> >> >
> >>
>
>

Re: Load driving scripts for Daytrader

[Gianny Damour <gi...@optusnet.com.au>]

Hello,

I think that the Grinder definitively is worth to consider. I have  
used it a couple of times and I think that it is "better" than JMeter.

Thanks,
Gianny


On 25/10/2006, at 3:44 AM, Slava McDougald wrote:

> Hi Matt,
>
>
> Well, my short evaluation of JMeter confirmed your observation: JMeter
> is definately not a great performing load driving tool.
>
> However, I tried/read about a few other open source load drivers and
> unfortunately I didn't find anything that performs really well.
>
> Since JMeter has the largest and  the most active development
> community that provides a good user's support, I think that we can
> start using it to drive Daytrader (at least until we find something
> better).
>
> Slava
>
>
> On 10/24/06, Slava McDougald <sl...@gmail.com> wrote:
>> Hi Piuysh,
>>
>> Writing your own driver is certainly an option but it wouldn't be my
>> preferred option.  You are right; it takes a lot of time, efforts,
>> good design and implementation to build a good load driving tool.
>>
>> In my opinion, it is better to just use whatever it is available out
>> in the open source space.  Something like JMeter for example has a
>> large community of developers working on improving and supporting it
>> as w ell as a lot of users that can provide hits/feedback on its use.
>> In addition, the source code is available to download and can change
>> according to your needs.  That is why I proposed to use the existing
>> load driver instead of developing one from scratch.
>>
>> For my testing, I used just a simple JMeter scripts, mostly for my  
>> own
>> educational purposes.  However, I am working on creating a more
>> complex JMeter workload script for Daytrader.
>>
>> Once completed, I will share it with you (the Geronimo development  
>> community).
>>
>> Then we can work together on changing/improving it and maybe package
>> it with Daytrader sample.
>>
>> Sounds good?
>>
>> Slava
>>
>>
>> On 10/24/06, Matt Hogstrom <ma...@hogstrom.org> wrote:
>> > If you can get JMeter going that would be great.  I did try it  
>> and it used
>> > an excessive amount of CPU but I may not have had it configured  
>> well.
>> >
>> >
>> > On Oct 24, 2006, at 11:22 AM, Piyush Agarwal wrote:
>> >
>> >
>> > Hi all,
>> >
>> >
>> >
>> >
>> > It is great to have the Daytrader sample in the Geronimo projects.
>> >
>> > Daytrader is an end-to-end application that allows a full  
>> functional
>> >
>> > testing of Geronimo as well as measuring the performance of the
>> >
>> > server.
>> >
>> >
>> >
>> >
>> > However, in order to do a performance test, a users need to  
>> drive the
>> >
>> > application with some kind of a load driving software.  And for  
>> that
>> >
>> > they need to spend time and effort to develop scripts for  
>> driving the
>> >
>> > sample.  Very often, they do not have the time or the desire to  
>> invest
>> >
>> > in learning all the specifics about a new load driving tool and/ 
>> or the
>> >
>> > application just to be able to perform a test.  But if they are
>> >
>> > provided will the scripts they will gladly use them.
>> >
>> > Matt Hogstrom
>> > matt@hogstrom.org
>> >
>> >
>> >
>> >
>>

Re: Load driving scripts for Daytrader

[Slava McDougald <sl...@gmail.com>]

Hi Matt,


Well, my short evaluation of JMeter confirmed your observation: JMeter
is definately not a great performing load driving tool.

However, I tried/read about a few other open source load drivers and
unfortunately I didn't find anything that performs really well.

Since JMeter has the largest and  the most active development
community that provides a good user's support, I think that we can
start using it to drive Daytrader (at least until we find something
better).

Slava


On 10/24/06, Slava McDougald <sl...@gmail.com> wrote:
> Hi Piuysh,
>
> Writing your own driver is certainly an option but it wouldn't be my
> preferred option.  You are right; it takes a lot of time, efforts,
> good design and implementation to build a good load driving tool.
>
> In my opinion, it is better to just use whatever it is available out
> in the open source space.  Something like JMeter for example has a
> large community of developers working on improving and supporting it
> as w ell as a lot of users that can provide hits/feedback on its use.
> In addition, the source code is available to download and can change
> according to your needs.  That is why I proposed to use the existing
> load driver instead of developing one from scratch.
>
> For my testing, I used just a simple JMeter scripts, mostly for my own
> educational purposes.  However, I am working on creating a more
> complex JMeter workload script for Daytrader.
>
> Once completed, I will share it with you (the Geronimo development community).
>
> Then we can work together on changing/improving it and maybe package
> it with Daytrader sample.
>
> Sounds good?
>
> Slava
>
>
> On 10/24/06, Matt Hogstrom <ma...@hogstrom.org> wrote:
> > If you can get JMeter going that would be great.  I did try it and it used
> > an excessive amount of CPU but I may not have had it configured well.
> >
> >
> > On Oct 24, 2006, at 11:22 AM, Piyush Agarwal wrote:
> >
> >
> > Hi all,
> >
> >
> >
> >
> > It is great to have the Daytrader sample in the Geronimo projects.
> >
> > Daytrader is an end-to-end application that allows a full functional
> >
> > testing of Geronimo as well as measuring the performance of the
> >
> > server.
> >
> >
> >
> >
> > However, in order to do a performance test, a users need to drive the
> >
> > application with some kind of a load driving software.  And for that
> >
> > they need to spend time and effort to develop scripts for driving the
> >
> > sample.  Very often, they do not have the time or the desire to invest
> >
> > in learning all the specifics about a new load driving tool and/or the
> >
> > application just to be able to perform a test.  But if they are
> >
> > provided will the scripts they will gladly use them.
> >
> > Matt Hogstrom
> > matt@hogstrom.org
> >
> >
> >
> >
>

Re: Load driving scripts for Daytrader

[Slava McDougald <sl...@gmail.com>]

Hi Piuysh,

Writing your own driver is certainly an option but it wouldn't be my
preferred option.  You are right; it takes a lot of time, efforts,
good design and implementation to build a good load driving tool.

In my opinion, it is better to just use whatever it is available out
in the open source space.  Something like JMeter for example has a
large community of developers working on improving and supporting it
as w ell as a lot of users that can provide hits/feedback on its use.
In addition, the source code is available to download and can change
according to your needs.  That is why I proposed to use the existing
load driver instead of developing one from scratch.

For my testing, I used just a simple JMeter scripts, mostly for my own
educational purposes.  However, I am working on creating a more
complex JMeter workload script for Daytrader.

Once completed, I will share it with you (the Geronimo development community).

Then we can work together on changing/improving it and maybe package
it with Daytrader sample.

Sounds good?

Slava


On 10/24/06, Matt Hogstrom <ma...@hogstrom.org> wrote:
> If you can get JMeter going that would be great.  I did try it and it used
> an excessive amount of CPU but I may not have had it configured well.
>
>
> On Oct 24, 2006, at 11:22 AM, Piyush Agarwal wrote:
>
>
> Hi all,
>
>
>
>
> It is great to have the Daytrader sample in the Geronimo projects.
>
> Daytrader is an end-to-end application that allows a full functional
>
> testing of Geronimo as well as measuring the performance of the
>
> server.
>
>
>
>
> However, in order to do a performance test, a users need to drive the
>
> application with some kind of a load driving software.  And for that
>
> they need to spend time and effort to develop scripts for driving the
>
> sample.  Very often, they do not have the time or the desire to invest
>
> in learning all the specifics about a new load driving tool and/or the
>
> application just to be able to perform a test.  But if they are
>
> provided will the scripts they will gladly use them.
>
> Matt Hogstrom
> matt@hogstrom.org
>
>
>
>

Re: Load driving scripts for Daytrader

[Matt Hogstrom <ma...@hogstrom.org>]

If you can get JMeter going that would be great.  I did try it and it  
used an excessive amount of CPU but I may not have had it configured  
well.

On Oct 24, 2006, at 11:22 AM, Piyush Agarwal wrote:

> Hi all,
>
> It is great to have the Daytrader sample in the Geronimo projects.
> Daytrader is an end-to-end application that allows a full functional
> testing of Geronimo as well as measuring the performance of the
> server.
>
> However, in order to do a performance test, a users need to drive the
> application with some kind of a load driving software.  And for that
> they need to spend time and effort to develop scripts for driving the
> sample.  Very often, they do not have the time or the desire to invest
> in learning all the specifics about a new load driving tool and/or the
> application just to be able to perform a test.  But if they are
> provided will the scripts they will gladly use them.

Matt Hogstrom
matt@hogstrom.org

Re: Load driving scripts for Daytrader

[Piyush Agarwal <pv...@gmail.com>]

Hi Slava,

I agree with your thoughts. Daytrader as a appserver performance test suite
is incomplete without having a way to drive load to it. I was contemplating
the thought of writing a Java based standalone client to drive it, but
haven't done that because of couple of afterthoughts. First was the time
required to do that, second writing it in such a way that it remains
configurable and extendable to suit different test scenarios.

The option of using an opensource load driver is definitely more appealing.
Its the learning curve reqd to use them and then coming up with the
scripts, but once someone does it for you and gives you a template its easy
to take off with it. I am sure we can use the scripts as-is and if need be
modify/augment it to suit individual testing needs. What kind of scenarios
have you tested with JMeter?

Regards,
Piyush Agarwal


On 10/24/06, Slava McDougald <sl...@gmail.com> wrote:
>
> Hi all,
>
> It is great to have the Daytrader sample in the Geronimo projects.
> Daytrader is an end-to-end application that allows a full functional
> testing of Geronimo as well as measuring the performance of the
> server.
>
> However, in order to do a performance test, a users need to drive the
> application with some kind of a load driving software.  And for that
> they need to spend time and effort to develop scripts for driving the
> sample.  Very often, they do not have the time or the desire to invest
> in learning all the specifics about a new load driving tool and/or the
> application just to be able to perform a test.  But if they are
> provided will the scripts they will gladly use them.
>
> I think that it would be very useful for both developers and users of
> Daytrader to package the benchmark with scripts for load driving it.
>
> There are a couple of load driving software project in the open source
> arena that we can use for that purpose.  Recently, I looked into a
> couple of them (for my own testing purposes).  I think that the Apache
> Jackarta JMeter (http://jakarta.apache.org/jmeter/) would be suitable
> choice for driving Daytrader.
>
>
> Thoughts?  Suggestions?
>
> Slava
>