You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by "G.W. Haywood" <ge...@www.jubileegroup.co.uk> on 2000/10/27 23:02:43 UTC
Re: how to really bang on a script?
Hi there,
On Fri, 27 Oct 2000, Christopher L. Everett wrote:
> I've written some mod_perl scripts that need testing over a million
> hits or so before I deploy it.
ab (distributed with Apache, 'man ab' for help) can give you a million
hits with one command.
I don't know if you're going to get a real-life test other than by
going live. Scripts tend to test what you think will go wrong.
Invariably something else goes wrong in real life. Putting a million
records in a logfile will test your disc capacity and little else.
> I need to prove to myself and my marketing guy that my script has
> certain statistical properties, not the least of which is the
> question of whether my activity logs match what actually happened.
You've been spending too much time with your marketing guy. "Certain
statistical properties" is gobbledygook. What properties? Activity
logs don't match statistically. Either they match or they don't. If
they don't then either the logging is turned off or it isn't working.
> Also, there's concurrency issues to make sure I've got right.
Get yourself a bunch of users. Probably the easiest way to really
bang on your script would be to advertise a new sex site (sorry:).
Anyway I'd really hate you to do all that work only to find that you
hadn't tested the test script well enough...
73,
Ged.
Re: how to really bang on a script?
Posted by "G.W. Haywood" <ge...@www.jubileegroup.co.uk>.
Hi Dave,
On 31 Oct 2000, David Hodgkinson wrote:
> Hey, you leave ValueClick alone! They're the least worst out there!
> (Is that a compliment?)
Ok, truce. There's room in the world for all sorts (even you and me:)
At the moment.
73,
Ged.
Re: how to really bang on a script?
Posted by David Hodgkinson <da...@hodgkinson.org>.
"G.W. Haywood" <ge...@www.jubileegroup.co.uk> writes:
> Hi all,
>
> On Mon, 30 Oct 2000, Ask Bjoern Hansen wrote:
> >
> > On Sat, 28 Oct 2000, Christopher L. Everett wrote:
> >
> > > OK, I confess: I've written (probably yet another) mod_perl banner
> > > exchange. I need to know that when we serve 100K banners to 40K
> >
> > if anyone doubts that perl and mod_perl is a good solution for that,
> > you can tell them that at ValueClick we can serve thousands and
> > thousands of banners per second on our technology that is close to
> > 100% Pure Perl.
>
> And if anyone wants to know how to write an ad-blocking proxy using
> the same technology it's on pages 374-381 of the first edition of the
> Eagle Book, "Writing Apache Modules with Perl and C" ISBN 1-56592-567-X.
Hey, you leave ValueClick alone! They're the least worst out there!
(Is that a compliment?)
--
Dave Hodgkinson, http://www.hodgkinson.org
Editor-in-chief, The Highway Star http://www.deep-purple.com
Apache, mod_perl, MySQL, Sybase hired gun for, well, hire
-----------------------------------------------------------------
Re: how to really bang on a script?
Posted by "G.W. Haywood" <ge...@www.jubileegroup.co.uk>.
Hi all,
On Mon, 30 Oct 2000, Ask Bjoern Hansen wrote:
>
> On Sat, 28 Oct 2000, Christopher L. Everett wrote:
>
> > OK, I confess: I've written (probably yet another) mod_perl banner
> > exchange. I need to know that when we serve 100K banners to 40K
>
> if anyone doubts that perl and mod_perl is a good solution for that,
> you can tell them that at ValueClick we can serve thousands and
> thousands of banners per second on our technology that is close to
> 100% Pure Perl.
And if anyone wants to know how to write an ad-blocking proxy using
the same technology it's on pages 374-381 of the first edition of the
Eagle Book, "Writing Apache Modules with Perl and C" ISBN 1-56592-567-X.
Works a treat.
73,
Ged.
Re: how to really bang on a script?
Posted by Ask Bjoern Hansen <as...@develooper.com>.
On Sat, 28 Oct 2000, Christopher L. Everett wrote:
[...]
> OK, I confess: I've written (probably yet another) mod_perl banner
> exchange. I need to know that when we serve 100K banners to 40K
Hi Christopher,
if anyone doubts that perl and mod_perl is a good solution for that,
you can tell them that at ValueClick we can serve thousands and
thousands of banners per second on our technology that is close to
100% Pure Perl.
:-)
[...]
> place. And again, my questions are: How would I go about proving to
> myself that my script does what I designed it to do? Has anyone else
> dealt with a similar problem, and how did they go about doing it?
Look at your system and then make up a lot of ways you can think it
could go wrong and then write some utilities to automatically do
those things and test in our output / logs / databases if what
happended was what you expected.
> If I solve it for myself, would anyone else find the solution
> useful, and how would I make it more useful to them?
That depends on how you solve it, but chances are pretty good that
you'll solve it very specifically to your problem (which might be a
good thing).
> Usually, I would test by running through the script a few times with
> some variations, but we are so freaked out by our experience with the
> 2 other banner exchange scripts we tried, we find a lot of value in
> being certain.
Sounds scary; I don't want to know more.
:-)
- ask
--
ask bjoern hansen - <http://www.netcetera.dk/~ask/>
more than 70M impressions per day, <http://valueclick.com>
Re: how to really bang on a script?
Posted by "G.W. Haywood" <ge...@www.jubileegroup.co.uk>.
Hi again,
On Sat, 28 Oct 2000, Christopher L. Everett wrote:
> OK, I confess: I've written (probably yet another) mod_perl banner
> exchange.
Argh.
[snipped impressive numbers]
> That's what I meant by "certain statistical properties".
Interesting. Nothing to do with statistics, but interesting.
> So, I apologize for not describing my problem clearly
No need for that.
> I need to know that the 500 sites that hosted our banner ads are
> getting the 50K banners that we promised them
The only way to *know* that is to get information from the sites.
If I wanted to be sure, I'd connect to them and look in their logs.
You probaly already know that people cheat a lot with this kind of
thing. Can't understand why they'd do that.
> and the 30K banners that we sold, we really did serve.
Is there a reason you're only selling - no, don't answer that.
> Also, I want to know that the banners my logs say the script
> sent are really the ones people saw on their browsers.
There you go again. The only way to know is to go and look. But you
might try something along the lines of tcpdump on your firewall box,
so that you could at least remove a whole slew of uncertainties from
the equation. You could pull your magic numbers out from the packets
and see from the packet headers if they were addressed to where they
were suposed to be addressed to. But without going further afield I
think that's about the best you will be able to do.
And don't write off ab - remember, you saw it here first...
73,
Ged.
Re: how to really bang on a script?
Posted by "Christopher L. Everett" <ce...@ceverett.com>.
Perrin Harkins wrote:
>
> On Sat, 28 Oct 2000, Matthew Byng-Maddick wrote:
>
> > On Sat, 28 Oct 2000, Matt Sergeant wrote:
> > > exactly the same thing (changing server logs into a benchmark tool) at
> > > ApacheCon, only I can't for the life of me remember who it was.
> >
> > Theo, during the mod_backhand talk, or at lunch just before, I can't
> > remember.
>
> It was during the talk. The tool is called Daquiri, and he said it was
> available in the mod_backhand CVS tree.
>
> I have also found httperf and http_load pretty useful for this stuff,
> although they don't support logfile playback.
>
Hey, I went by http://www.mod_backhand.org/, and found no mention of
anything called Daquiri, nor any description of how to access the
mod_backhand CVS tree, not even an email address to ask for information.
I downloaded the mod_backhand source, and I found nothing there either.
Does anyone have a clue about where to start getting my hands on this
neat and nifty tool?
--Christopher
Re: how to really bang on a script?
Posted by Matthew Byng-Maddick <mb...@colondot.net>.
On Mon, 30 Oct 2000, Perrin Harkins wrote:
> On Sat, 28 Oct 2000, Matthew Byng-Maddick wrote:
> > Theo, during the mod_backhand talk, or at lunch just before, I can't
> > remember.
> It was during the talk. The tool is called Daquiri, and he said it was
> available in the mod_backhand CVS tree.
Ah yes, I remember someone joking about what flavour it was next to me. :)
MBM
--
Matthew Byng-Maddick Home: <mb...@colondot.net> +44 20 8981 8633 (Home)
http://colondot.net/ Work: <ma...@codix.net> +44 7956 613942 (Mobile)
perl -e '$_="Oyvv bsswjfw Thtm mefmfw2\n";while(m([^\n])){$_=$'"'"';$a=$&;
$a=($a=~m(^\s)?$a:pack "c",unpack("c",$a)-5+($i++%5));print $a}print"\n";'
Re: how to really bang on a script?
Posted by Perrin Harkins <pe...@primenet.com>.
On Sat, 28 Oct 2000, Matthew Byng-Maddick wrote:
> On Sat, 28 Oct 2000, Matt Sergeant wrote:
> > exactly the same thing (changing server logs into a benchmark tool) at
> > ApacheCon, only I can't for the life of me remember who it was.
>
> Theo, during the mod_backhand talk, or at lunch just before, I can't
> remember.
It was during the talk. The tool is called Daquiri, and he said it was
available in the mod_backhand CVS tree.
I have also found httperf and http_load pretty useful for this stuff,
although they don't support logfile playback.
- Perrin
Re: how to really bang on a script?
Posted by Matthew Byng-Maddick <mb...@colondot.net>.
On Sat, 28 Oct 2000, Matt Sergeant wrote:
> exactly the same thing (changing server logs into a benchmark tool) at
> ApacheCon, only I can't for the life of me remember who it was.
Theo, during the mod_backhand talk, or at lunch just before, I can't
remember.
MBM
--
Matthew Byng-Maddick Home: <mb...@colondot.net> +44 20 8981 8633 (Home)
http://colondot.net/ Work: <ma...@codix.net> +44 7956 613942 (Mobile)
perl -e '$_="Oyvv bsswjfw Thtm mefmfw2\n";while(m([^\n])){$_=$'"'"';$a=$&;
$a=($a=~m(^\s)?$a:pack "c",unpack("c",$a)-5+($i++%5));print $a}print"\n";'
Re: how to really bang on a script?
Posted by Matt Sergeant <ma...@sergeant.org>.
On Sat, 28 Oct 2000, Christopher L. Everett wrote:
> So, I apologize for not describing my problem clearly in the first
> place. And again, my questions are: How would I go about proving to
> myself that my script does what I designed it to do? Has anyone else
> dealt with a similar problem, and how did they go about doing it? If
> I solve it for myself, would anyone else find the solution useful, and
> how would I make it more useful to them?
Basically you've got exactly the right idea, and it *has* been done before
- I recall vaguely in the back of my head someone mentioning doing almost
exactly the same thing (changing server logs into a benchmark tool) at
ApacheCon, only I can't for the life of me remember who it was.
On the plus side though, provided you've got no POST parameters, its a
pretty trivial script. The hard part is getting a scalable engine to
execute all those requests. Really you have to do that in C I think - and
you should probably just look towards the source of ab.
FWIW, I think it was Theo Schlossnagel (mod_backhand guy) talking about
the utility, in fact I'm almost certain. Why don't you drop him a line.
--
<Matt/>
/|| ** Director and CTO **
//|| ** AxKit.com Ltd ** ** XML Application Serving **
// || ** http://axkit.org ** ** XSLT, XPathScript, XSP **
// \\| // ** Personal Web Site: http://sergeant.org/ **
\\//
//\\
// \\
Re: how to really bang on a script?
Posted by "Christopher L. Everett" <ce...@ceverett.com>.
"G.W. Haywood" wrote:
>
> Hi there,
>
> On Fri, 27 Oct 2000, Christopher L. Everett wrote:
>
> <snipped helpful advice>
>
> > I need to prove to myself and my marketing guy that my script has
> > certain statistical properties, not the least of which is the
> > question of whether my activity logs match what actually happened.
>
> You've been spending too much time with your marketing guy. "Certain
> statistical properties" is gobbledygook. What properties? Activity
> logs don't match statistically. Either they match or they don't. If
> they don't then either the logging is turned off or it isn't working.
>
OK, I confess: I've written (probably yet another) mod_perl banner
exchange. I need to know that when we serve 100K banners to 40K
different IP addresses a day, and we are selling 30K banners/day,
the 500 sites that hosted our banner ads are getting the 50K banners
that we promised them, and the 30K banners that we sold, we really did
serve. Also, I want to know that the banners my logs say the script
sent are really the ones people saw on their browsers. That's what I
meant by "certain statistical properties".
So, I apologize for not describing my problem clearly in the first
place. And again, my questions are: How would I go about proving to
myself that my script does what I designed it to do? Has anyone else
dealt with a similar problem, and how did they go about doing it? If
I solve it for myself, would anyone else find the solution useful, and
how would I make it more useful to them?
Usually, I would test by running through the script a few times with
some variations, but we are so freaked out by our experience with the
2 other banner exchange scripts we tried, we find a lot of value in
being certain.
Thanks again for your kind help.
--Christopher Everett
Re: how to really bang on a script?
Posted by Ask Bjoern Hansen <as...@apache.org>.
On Fri, 27 Oct 2000, G.W. Haywood wrote:
> > I need to prove to myself and my marketing guy that my script has
> > certain statistical properties, not the least of which is the
> > question of whether my activity logs match what actually happened.
>
> You've been spending too much time with your marketing guy. "Certain
> statistical properties" is gobbledygook. What properties?
able to handle N hits in M seconds with a maximum of K concurrent
sessions in any given second, L percent of the hits on these files,
J percent of the hits on this handler, U percent of these with these
parameters and W percent with those.
You add more to the list.
It's a perfectly sensibly thing to do, also from a technical
standpoint.
Maybe you've spend too little time on larger sites where those
things matters. ;-)
> Activity logs don't match statistically. Either they match or
> they don't. If they don't then either the logging is turned off
> or it isn't working.
Ha. Ask [ insert name of large internet advertising company ] about
that. Actually, just ask any given internet advertising company
about that. (in that industry the logs are real money and not just
input data to get pretty colored graphs).
More to the point: What do you know about how Christopher is
logging? Maybe he is using mod_log_spread or something fancy like
that and is unsure if it's working correctly.
> > Also, there's concurrency issues to make sure I've got right.
>
> Get yourself a bunch of users.
It's a real hassle to be dependent on thousands of users just to
exercise your test environment.
Christopher, depending on what you need to test and what kind of
site you have I'd look at (some combination of)
http_load - http://www.acme.com/software/http_load/ - to test how
your site can handle large amounts of hits to random combinations of
a predefined list of urls.
ab - comes with apache, see src/support/ab(\.[ch])? - good for
testing performance parameters of a certain URL.
LWP - Roll your own tests with the LWP modules to have an automated
test of functionality of the site.
- ask
--
ask bjoern hansen - <http://www.netcetera.dk/~ask/>
more than 70M impressions per day, <http://valueclick.com>