You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Bryan Henry <br...@mail.utexas.edu> on 2002/03/12 22:45:04 UTC
performance testing - emulating real world use
Anyone know of good guides or general info on
performance testing and emulating real use of
an application.
I would like to understand how to identify
potential bottlenecks before I deploy web apps.
thank you,
~ b r y a n
Re: performance testing - emulating real world use
Posted by Paul Lindner <li...@inuus.com>.
On Tue, Mar 12, 2002 at 01:52:36PM -0800, clayton cottingham wrote:
> Bryan Henry wrote:
> >
> > Anyone know of good guides or general info on
> > performance testing and emulating real use of
> > an application.
> >
> > I would like to understand how to identify
> > potential bottlenecks before I deploy web apps.
> >
> > thank you,
> > ~ b r y a n
I've used HTTPD::Bench::ApacheBench (available on CPAN) to do load
testing. It seems to do a good job. The hardest part is writing the
testing script (especially for form transactions..).
However, if you can do your requests with LWP it's fairly
straightforward to convert over to the ApacheBench data structures.
I'm considering writing a little mod_perl proxy server that records
the different transactions. Then I could just munge the separate
Authorization: headers to do some serious load testing...
>
> try httpd.apache.org/test/
>
> and perl framework
> there in
>
> as well look on freshmeat for
>
> siege
> it does testing too
--
Paul Lindner lindner@inuus.com ||||| | | | | | | | | |
mod_perl Developer's Cookbook http://www.modperlcookbook.org/
Human Rights Declaration http://www.unhchr.ch/udhr/
Re: performance testing - emulating real world use
Posted by clayton cottingham <dr...@telus.net>.
Bryan Henry wrote:
>
> Anyone know of good guides or general info on
> performance testing and emulating real use of
> an application.
>
> I would like to understand how to identify
> potential bottlenecks before I deploy web apps.
>
> thank you,
> ~ b r y a n
try httpd.apache.org/test/
and perl framework
there in
as well look on freshmeat for
siege
it does testing too
Re: performance testing - emulating real world use
Posted by Andrew Ho <an...@tellme.com>.
Hello,
AH>So you're correct. My point though is not so much that the load profile of
AH>what pages get loaded in what order, and what data calls and dynamic
AH>scripts are run in what order are genuine. If you simulate the timing
AH>between requests, you'll even get spikes that are similar to the real
AH>thing. It's definitely not reality! You also miss anomalies like users
AH>closing browsers and (unless you capture full headers) which clients
AH>support keep-alives for example. But, it's closer to reality than most
AH>scripts that are invented (especially be developers ;)).
Man, I can't type worth anything today. The gist of what I meant to type
was this: the exact load of the production server will not be replicated
in your simulation; but the load from data calls and dynamically generated
content will be similar in nature, patterned after how your server is hit
in real life. This will likely be a better exercise of your server than a
developer-invented test script.
If you have a production environment (or proxy) set up that can capture
real user requests, this is also far less work for creating a convincing
simulation load than having to sit down and write a new script every time
your application changes.
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------
Re: performance testing - emulating real world use
Posted by Andrew Ho <an...@tellme.com>.
Hello,
ABH>Not really; you also have to emulate the connection speeds of the
ABH>users. Or does the tools you mentioned do that?
Both of the commercially produced tools I mentioned (SilkPerformer and the
free Microsoft Web Stress program) can throttle bandwidth. Rolling your
own is a bunch harder.
So you're correct. My point though is not so much that the load profile of
what pages get loaded in what order, and what data calls and dynamic
scripts are run in what order are genuine. If you simulate the timing
between requests, you'll even get spikes that are similar to the real
thing. It's definitely not reality! You also miss anomalies like users
closing browsers and (unless you capture full headers) which clients
support keep-alives for example. But, it's closer to reality than most
scripts that are invented (especially be developers ;)).
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------
Re: performance testing - emulating real world use
Posted by Ask Bjoern Hansen <as...@valueclick.com>.
On Tue, 12 Mar 2002, Andrew Ho wrote:
[...]
> This is extremely effective if you have enough real user data because
> you're not inventing user load. You're using real user load.
Not really; you also have to emulate the connection speeds of the
users. Or does the tools you mentioned do that?
- ask
--
ask bjoern hansen, http://ask.netcetera.dk/ !try; do();
more than a billion impressions per week, http://valueclick.com
Re: performance testing - emulating real world use
Posted by Perrin Harkins <pe...@elem.com>.
Jauder Ho wrote:
> Another application (commercial) is Mercury Interactive's LoadRunner.
My experience with commercial load-testing apps is that they are
outrageously expensive, a pain to program, don't really scale all that
well, and mostly have to run on Windows with someone sitting at the
mouse. There are some that work better than others, but the free stuff
in this areas is quite good.
I recommend httperf and http_load for banging on lists of URLs really
hard. At eToys, one of our developers rigged up some shell scripts that
would play back log files through httperf and that worked pretty well.
If you want to record browser sessions for testing specific paths
through the site, look at http://sourceforge.net/projects/http-recorder/
or http://sourceforge.net/projects/roboweb/. There's also webchatpp,
HTTP::WebTest, and HTTP::MonkeyWrench on CPAN. All of these have been
discussed on this list before.
- Perrin
Re: performance testing - emulating real world use
Posted by Jauder Ho <ja...@carumba.com>.
Heh. Forgot to state that it does cost an arm and a leg but it's one of
the few software packages that is worth considering paying money for IMO.
However, with the economy being the way it is, it is possible to "rent"
the software for a period of time but this is done by special arrangement
on a case by case basis.
If you ask, they may be willing to give you a copy of the software. The
out-the-box install allows you to record and playback as well as load up
to 10 (iirc) users which nicely lets you test out the functionality. I
think I have a copy of it somewhere around here. If there is interest, I
can get you in touch with the right people.
--Jauder
On Wed, 13 Mar 2002, Matt Sergeant wrote:
> On Tue, 12 Mar 2002, Jauder Ho wrote:
>
> >
> > Another application (commercial) is Mercury Interactive's LoadRunner. It
> > actually records events and plays it back on "load generator" machines.
> > It's fairly complex, has LOTs of knobs to turn and can load test quite a
> > bit more than just web apps, I use it to load test/benchmark Oracle 11i
> > for instance. The software is not cheap but definitely worth looking into
> > if you are serious about testing. (www.merc-int.com)
> >
> > They also sell something called ActiveTest which may be more suited to a
> > web applications. In this case, they will test your site for you using
> > their hardware at a colo site.
>
> Before anyone even looks into this, be warned they quoted me �50,000 once
> for LoadRunner. Needless to say I was flabbergasted (though their software
> did look kinda cool).
>
> --
> <!-- Matt -->
> <:->Get a smart net</:->
>
>
Re: performance testing - emulating real world use
Posted by Matt Sergeant <ma...@sergeant.org>.
On Tue, 12 Mar 2002, Jauder Ho wrote:
>
> Another application (commercial) is Mercury Interactive's LoadRunner. It
> actually records events and plays it back on "load generator" machines.
> It's fairly complex, has LOTs of knobs to turn and can load test quite a
> bit more than just web apps, I use it to load test/benchmark Oracle 11i
> for instance. The software is not cheap but definitely worth looking into
> if you are serious about testing. (www.merc-int.com)
>
> They also sell something called ActiveTest which may be more suited to a
> web applications. In this case, they will test your site for you using
> their hardware at a colo site.
Before anyone even looks into this, be warned they quoted me £50,000 once
for LoadRunner. Needless to say I was flabbergasted (though their software
did look kinda cool).
--
<!-- Matt -->
<:->Get a smart net</:->
Re: performance testing - emulating real world use
Posted by Jauder Ho <ja...@carumba.com>.
Another application (commercial) is Mercury Interactive's LoadRunner. It
actually records events and plays it back on "load generator" machines.
It's fairly complex, has LOTs of knobs to turn and can load test quite a
bit more than just web apps, I use it to load test/benchmark Oracle 11i
for instance. The software is not cheap but definitely worth looking into
if you are serious about testing. (www.merc-int.com)
They also sell something called ActiveTest which may be more suited to a
web applications. In this case, they will test your site for you using
their hardware at a colo site.
--Jauder
On Tue, 12 Mar 2002, Andrew Ho wrote:
> Heyas,
>
> BH>Anyone know of good guides or general info on
> BH>performance testing and emulating real use of
> BH>an application.
>
> As a general rule, it's easiest if you have a production system already
> running. Record all information that you need to reproduce the requests
> (typically, HTTP request headers and POST data if applicable), from a
> production server and you can replay any amount of data on a sandboxed QA
> environment. You can either eliminate or proportionally shorten the time
> period between requests to space out load arbitrarily.
>
> This is extremely effective if you have enough real user data because
> you're not inventing user load. You're using real user load.
>
> I don't know of any product that does this all at once, but it's not hard
> to hack together. If your site is entirely GET based, you can probably
> just make do with parsing access logs and turning those into requests. I
> believe Apache::DumpHeaders might get you most of the way on the capturing
> side if you need special headers, cookies, or POST information.
>
> Feeding scripts into commercial products like SilkPerformer will give you
> the best client side testing and reports. However, a homebrew Perl user
> agent will do okay, too. Unfortunately, ab doesn't support taking in a
> custom URL workload.
>
> For a simple record/replay load test tool that works remarkably well,
> check out the resource CD that ships with Windows 2000 and you will find
> the Microsoft Web Stress Tester. It's free and GUI based and can record IE
> sessions and replay them with an arbitrary number of threads. It uses
> Access databases to hold the tests and results so you can probably use
> Perl on Windows to populate it with your custom tests.
>
> Humbly,
>
> Andrew
>
> ----------------------------------------------------------------------
> Andrew Ho http://www.tellme.com/ andrew@tellme.com
> Engineer info@tellme.com Voice 650-930-9062
> Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
> ----------------------------------------------------------------------
>
>
Re: performance testing - emulating real world use
Posted by Andrew Ho <an...@tellme.com>.
Heyas,
BH>Anyone know of good guides or general info on
BH>performance testing and emulating real use of
BH>an application.
As a general rule, it's easiest if you have a production system already
running. Record all information that you need to reproduce the requests
(typically, HTTP request headers and POST data if applicable), from a
production server and you can replay any amount of data on a sandboxed QA
environment. You can either eliminate or proportionally shorten the time
period between requests to space out load arbitrarily.
This is extremely effective if you have enough real user data because
you're not inventing user load. You're using real user load.
I don't know of any product that does this all at once, but it's not hard
to hack together. If your site is entirely GET based, you can probably
just make do with parsing access logs and turning those into requests. I
believe Apache::DumpHeaders might get you most of the way on the capturing
side if you need special headers, cookies, or POST information.
Feeding scripts into commercial products like SilkPerformer will give you
the best client side testing and reports. However, a homebrew Perl user
agent will do okay, too. Unfortunately, ab doesn't support taking in a
custom URL workload.
For a simple record/replay load test tool that works remarkably well,
check out the resource CD that ships with Windows 2000 and you will find
the Microsoft Web Stress Tester. It's free and GUI based and can record IE
sessions and replay them with an arbitrary number of threads. It uses
Access databases to hold the tests and results so you can probably use
Perl on Windows to populate it with your custom tests.
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------