You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Milo Hyson <mi...@cyberlifelabs.com> on 2002/01/24 07:37:02 UTC
When to cache
I'm interested to know what the opinions are of those on this list with
regards to caching objects during database write operations. I've encountered
different views and I'm not really sure what the best approach is.
Take a typical caching scenario: Data/objects are locally stored upon loading
from a database to improve performance for subsequent requests. But when
those objects change, what's the best method for refreshing the cache? There
are two possible approaches (maybe more?):
1) The old cache entry is overwritten with the new.
2) The old cache entry is expired, thus forcing a database hit (and
subsequent cache load) on the next request.
The first approach would tend to yield better performance. However there's no
guarantee the data will ever be read. The cache could end up with a large
amount of data that's never referenced. The second approach would probably
allow for a smaller cache by ensuring that data is only cached on reads.
In the end, this probably boils down to application requirements. RAM and
disk storage is so cheap these days that the first method is probably fine
for most purposes. However I'm sure there are situations where resources are
limited and the second is more effective. What does everyone think?
--
Milo Hyson
CyberLife Labs, LLC
Re: When to cache
Posted by Rob Nagler <na...@bivio.biz>.
> When you dig into it, most sites have a lot of data that can be out of sync
> for some period.
Agreed. We run an accounting application which just happens to be
delivered via the web. This definitely colors (distorts?) my view.
> heavy SQL. Some people would say to denormalize the database at that point,
> but that's really just another form of caching.
Absolutely. Denormalization is the root of all evil. ;-)
> No need to do that yourself. Just use DBIx::Profile to find the hairy
> queries.
History. Also, another good trick is to make sure your select
statements are as similar as possible. It is often better to bundle a
couple of similar queries into a single one. The query compiler
caches queries.
> Ironically, I am quoted in Philip Greenspun's book on web publishing saying
> just what you are saying: that databases should be fast enough without
> middle-tier caching. Sadly, sometimes they just aren't.
Every system design decision often has an equally valid converse.
The art is knowing when to buy and when to sell. And Greenspun's book
is a great resource btw.
Rob
Re: When to cache
Posted by Perrin Harkins <pe...@elem.com>.
> Perrin Harkins writes:
> > To fix this, we moved to not generating anything until it was requested.
We
> > would fetch the data the first time it was asked for, and then cache it
for
> > future requests. (I think this corresponds to your option 2.) Of
course
> > then you have to decide on a cache consistency approach for keeping that
> > data fresh. We used a simple TTL approach because it was fast and easy
to
> > implement ("good enough").
>
> I'd be curious to know the cache hit stats.
In this case, there was a high locality of access, so we got about a 99% hit
rate. Obviously not every cache will be this successful.
> BTW, this case seems to
> be an example of immutable data, which is definitely worth caching if
> performance dictates.
It wasn't immutable, but it was data that we could allow to be out of sync
for a certain amount of time that was dictated by the business requirements.
When you dig into it, most sites have a lot of data that can be out of sync
for some period.
> I agree with latter clause, but take issue with the former. Typical
> sites get a few hits a second at peak times. If a site isn't
> returning "typical" pages in under a second using mod_perl, it
> probably has some type of basic problem imo.
Some sites have complex requirements. eToys may have been an anomaly
because of the amount of traffic, but the thing that forced us to cache was
database performance. Tuning the perl stuff was not very hard, and it was
all pretty fast to begin with. Tuning the database access hit a wall when
our DBAs had gone over the queries, indexes had been adjusted, and some
things were still slow. The nature of the site design (lots of related data
on a single page) required many database calls and some of them were fairly
heavy SQL. Some people would say to denormalize the database at that point,
but that's really just another form of caching.
> Use a profiler on the actual code.
Agreed.
> Add
> performance stats in your code. For example, we encapsulate all DBI
> accesses and accumulate the time spent in DBI on any request.
No need to do that yourself. Just use DBIx::Profile to find the hairy
queries.
> Adding a cache is piling more code onto a solution. It sometimes is
> like adding lots of salt to bad cooking. You do it when you have to,
> but you end up paying for it later.
It may seem like the wrong direction to add code in order to make things go
faster, but you have to consider the relative speeds: Perl code is really
fast, databases are often slower than we want them to be.
Ironically, I am quoted in Philip Greenspun's book on web publishing saying
just what you are saying: that databases should be fast enough without
middle-tier caching. Sadly, sometimes they just aren't.
- Perrin
Re: Apache::args vs Apache::Request speed
Posted by Stas Bekman <st...@stason.org>.
mark warren bracher wrote:
> I didn't ever actually see a post with newer numbers, so here goes......
>
> I tested the same 50 clients/5000 requests as stas' test in the guide.
> one pass with 2 uri params; another with 26. naturally I ran it all on
> a server big (and quiescent) enough to handle the 50 concurrent
> requests. I left out CGI since we already know it is slow.
>
> /test/params and /test/args are mod_perl handlers (I don't use
> Apache::Registry for anything) ParamsTest and ArgsTest respectively. the
> code for both handlers and the relevant pieces of ab output are pasted
> in below.
>
> -------------------------------------------------------------
> name query_length | avtime completed failed rps
> -------------------------------------------------------------
> apache_args 25 | 33.77 5000 0 1481
> apache_request 25 | 33.17 5000 0 1507
> apache_args 337 | 43.51 5000 0 1141
> apache_request 337 | 45.31 5000 0 1103
> --------------------------------------------------------------
> Non-varying sub-test parameters:
> --------------------------------------------------------------
> concurrency : 50
> connections : 5000
>
> so $apr->param is marginally faster than $r->args for the shorter query,
> marginally slower for the longer one. I think this may be because we
> can return the full hash $r->args whereas we need to map over
> $apr->param to get a hash (so param gets called three times for the
> short query, 27 times for the larger one). still, much closer numbers
> than the former test...
Thanks Mark for pushing me to rerun the benchmark :)
Actually from the tests that I just run Apache::Request::param is
actually kicks $r->args on long inputs, and a bit faster on the short
query strings. Even though as you have noticed we call $q->param() 2 x
keys times more for each request.
Here are the results:
concurrency connections name query_length | avtime
completed failed rps
--------------------------------------------------------------------------------------------
50 5000 apache_request_param 25 | 53
5000 0 900
50 2000 apache_request_param 25 | 54
2000 0 884
50 5000 r_args 25 | 55
5000 0 879
50 2000 apache_request_param 105 | 54
2000 0 879
10 5000 apache_request_param 25 | 10
5000 0 878
50 5000 r_args 105 | 55
5000 0 876
10 2000 r_args 105 | 10
2000 0 869
50 5000 apache_request_param 105 | 56
5000 0 865
10 5000 apache_request_param 105 | 10
5000 0 855
10 5000 r_args 25 | 11
5000 0 850
10 2000 apache_request_param 105 | 11
2000 0 836
10 2000 r_args 25 | 11
2000 0 835
10 2000 apache_request_param 25 | 11
2000 0 832
50 2000 r_args 25 | 58
2000 0 827
10 5000 r_args 105 | 11
5000 0 810
50 5000 apache_request_param 207 | 64
5000 0 754
50 2000 apache_request_param 337 | 64
2000 0 750
10 2000 apache_request_param 207 | 12
2000 0 749
10 2000 apache_request_param 337 | 12
2000 0 749
50 2000 apache_request_param 207 | 64
2000 0 749
10 5000 apache_request_param 207 | 12
5000 0 746
50 2000 r_args 105 | 64
2000 0 744
10 5000 apache_request_param 337 | 12
5000 0 732
50 5000 r_args 207 | 72
5000 0 671
10 2000 r_args 337 | 14
2000 0 665
10 5000 r_args 207 | 14
5000 0 661
50 2000 r_args 337 | 73
2000 0 660
10 2000 r_args 207 | 14
2000 0 657
50 5000 apache_request_param 337 | 74
5000 0 647
50 2000 r_args 207 | 75
2000 0 645
10 5000 r_args 337 | 15
5000 0 627
50 5000 r_args 337 | 81
5000 0 601
I'll update this and other benchmarks. Something that should be done
once in a while since the software's performance tend to change over the
time :)
I'll probably run many more tests and build graphs, so it'll be much
easier to see what's happening, than from looking at many numbers.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Apache::args vs Apache::Request speed
Posted by mark warren bracher <br...@citysearch.com>.
I didn't ever actually see a post with newer numbers, so here goes......
I tested the same 50 clients/5000 requests as stas' test in the guide.
one pass with 2 uri params; another with 26. naturally I ran it all on
a server big (and quiescent) enough to handle the 50 concurrent
requests. I left out CGI since we already know it is slow.
/test/params and /test/args are mod_perl handlers (I don't use
Apache::Registry for anything) ParamsTest and ArgsTest respectively.
the code for both handlers and the relevant pieces of ab output are
pasted in below.
-------------------------------------------------------------
name query_length | avtime completed failed rps
-------------------------------------------------------------
apache_args 25 | 33.77 5000 0 1481
apache_request 25 | 33.17 5000 0 1507
apache_args 337 | 43.51 5000 0 1141
apache_request 337 | 45.31 5000 0 1103
--------------------------------------------------------------
Non-varying sub-test parameters:
--------------------------------------------------------------
concurrency : 50
connections : 5000
so $apr->param is marginally faster than $r->args for the shorter query,
marginally slower for the longer one. I think this may be because we
can return the full hash $r->args whereas we need to map over
$apr->param to get a hash (so param gets called three times for the
short query, 27 times for the larger one). still, much closer numbers
than the former test...
- mark
package ParamsTest;
use strict;
use Apache;
use Apache::Constants qw( OK );
use Apache::Request;
sub handler {
my $r = Apache::Request->new( shift );
$r->send_http_header( 'text/plain' );
my %args = map { $_ => $r->param( $_ ) } $r->param();
$r->print( join( "\n",
map { join( '',
$_ , ' => ' , $args{$_}
) }
keys %args
)
);
return OK;
}
1;
package ArgsTest;
use strict;
use Apache;
use Apache::Constants qw( OK );
sub handler {
my $r = shift;
$r->send_http_header( 'text/plain' );
my %args = $r->args();
$r->print( join( "\n",
map { join( '',
$_ , ' => ' , $args{$_}
) }
keys %args
)
);
return OK;
}
1;
Document Path: /test/params?a=eeeeeeeeee&b=eeeeeeeeee
Document Length: 31 bytes
Concurrency Level: 50
Time taken for tests: 3.317 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 883520 bytes
HTML transferred: 155620 bytes
Requests per second: 1507.39 [#/sec] (mean)
Time per request: 33.17 [ms] (mean)
Time per request: 0.66 [ms] (mean, across all concurrent requests)
Transfer rate: 266.36 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 5 1.9 5 16
Processing: 14 27 4.6 27 50
Waiting: 9 27 4.7 27 49
Total: 14 32 3.7 32 54
Document Path: /test/args?a=eeeeeeeeee&b=eeeeeeeeee
Document Length: 31 bytes
Concurrency Level: 50
Time taken for tests: 3.377 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 883168 bytes
HTML transferred: 155558 bytes
Requests per second: 1480.60 [#/sec] (mean)
Time per request: 33.77 [ms] (mean)
Time per request: 0.68 [ms] (mean, across all concurrent requests)
Transfer rate: 261.52 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 5 1.8 5 18
Processing: 12 27 4.5 28 63
Waiting: 8 27 4.5 27 63
Total: 12 32 3.7 32 65
Document Path:
/test/params?a=eeeeeeeeee&b=eeeeeeeeee&c=eeeeeeeeee&d=eeeeeeeeee&e=eeeeeeeeee&f=eeeeeeeeee&g=eeeeeeeeee&h=eeeeeeeeee&i=eeeeeeeeee&j=eeeeeeeeee&k=eeeeeeeeee&l=eeeeeeeeee&m=eeeeeeeeee&n=eeeeeeeeee&o=eeeeeeeeee&p=eeeeeeeeee&q=eeeeeeeeee&r=eeeeeeeeee&s=eeeeeeeeee&t=eeeeeeeeee&u=eeeeeeeeee&v=eeeeeeeeee&w=eeeeeeeeee&x=eeeeeeeeee&y=eeeeeeeeee&z=eeeeeeeeee
Document Length: 415 bytes
Concurrency Level: 50
Time taken for tests: 4.531 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 2810640 bytes
HTML transferred: 2082885 bytes
Requests per second: 1103.51 [#/sec] (mean)
Time per request: 45.31 [ms] (mean)
Time per request: 0.91 [ms] (mean, across all concurrent requests)
Transfer rate: 620.31 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 7 2.9 6 22
Processing: 18 37 11.2 36 496
Waiting: 12 36 11.1 36 495
Total: 18 44 10.8 43 501
Document Path:
/test/args?a=eeeeeeeeee&b=eeeeeeeeee&c=eeeeeeeeee&d=eeeeeeeeee&e=eeeeeeeeee&f=eeeeeeeeee&g=eeeeeeeeee&h=eeeeeeeeee&i=eeeeeeeeee&j=eeeeeeeeee&k=eeeeeeeeee&l=eeeeeeeeee&m=eeeeeeeeee&n=eeeeeeeeee&o=eeeeeeeeee&p=eeeeeeeeee&q=eeeeeeeeee&r=eeeeeeeeee&s=eeeeeeeeee&t=eeeeeeeeee&u=eeeeeeeeee&v=eeeeeeeeee&w=eeeeeeeeee&x=eeeeeeeeee&y=eeeeeeeeee&z=eeeeeeeeee
Document Length: 415 bytes
Concurrency Level: 50
Time taken for tests: 4.381 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 2808960 bytes
HTML transferred: 2081640 bytes
Requests per second: 1141.29 [#/sec] (mean)
Time per request: 43.81 [ms] (mean)
Time per request: 0.88 [ms] (mean, across all concurrent requests)
Transfer rate: 641.17 [Kbytes/sec] received
Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 7 3.0 7 35
Processing: 13 36 6.5 35 86
Waiting: 8 35 6.5 35 86
Total: 13 42 5.6 42 90
Stas Bekman wrote:
> Joe Schaefer wrote:
>> Right- param() was rewritten as XS about 6-8 months ago; since then
>> I've benchmarked it a few times and found param() to be a bit faster than
>> args(). We'll be releasing a 1.0 version of libapreq as soon as Jim
>> approves of the current CVS version. Here's what I got using it on
>> your benchmark (some differences: the tests were run against localhost
>> running perl 5.00503 + mod_perl 1.26 + apache 1.3.22 and using Perl
>> handlers instead of Apache::RegistryLoader scripts):
>
> Great! Now we have an even broader benchmark. Please tell me when 1.0 is
> released (in case I get carried away with other things and don't notice
> the announce) and I'll make sure to update my benchmarking package,
> re-run the benchmarks and correct the results in the guide.
>
> Thanks Joe!
Re: Apache::args vs Apache::Request speed
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Also I was only a user of Apache::Request, so I know very little about
>>it. It's about a time I should learn its guts. Does this library work
>>for 2.0 as is, or is there a new implementation for 2.0? I see that ToDo
>>says that it should be ported to 2.0.
>>
>
> I've never tried using it with 2.0, but I'd guess that it won't run
> there as-is. But that should be our top priority after we release
> 1.0. It's really important that we get out a release version that
> we promise not to tinker around with API-wise, so people can start
> developing derivative products without too much worry that we'll break
> their code. Lots of people use apreq outside of the Perl community,
> and getting a 1.x release out there for the C API is very important
> to them.
Certainly.
>>So first the C lib should be ported and then the Perl glue should be
>>written, right?
>>
>
> Yes, exactly. Choosing a new C API for apreq that's geared toward
> apache 2.0 AND improves upon some of the deficiencies of the current
> libapreq is an important first step, and IMO needs to be debated and
> created on the apreq-dev list NOW. It would be great if you signed up
> to that list so we can start discussing these issues there. I've cc-d
> this email to the list just in case :)
I'm on the list now. Let's start.
--
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: Apache::args vs Apache::Request speed
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Also I was only a user of Apache::Request, so I know very little about
> it. It's about a time I should learn its guts. Does this library work
> for 2.0 as is, or is there a new implementation for 2.0? I see that ToDo
> says that it should be ported to 2.0.
I've never tried using it with 2.0, but I'd guess that it won't run
there as-is. But that should be our top priority after we release
1.0. It's really important that we get out a release version that
we promise not to tinker around with API-wise, so people can start
developing derivative products without too much worry that we'll break
their code. Lots of people use apreq outside of the Perl community,
and getting a 1.x release out there for the C API is very important
to them.
> So first the C lib should be ported and then the Perl glue should be
> written, right?
Yes, exactly. Choosing a new C API for apreq that's geared toward
apache 2.0 AND improves upon some of the deficiencies of the current
libapreq is an important first step, and IMO needs to be debated and
created on the apreq-dev list NOW. It would be great if you signed up
to that list so we can start discussing these issues there. I've cc-d
this email to the list just in case :)
Best.
--
Joe Schaefer
Re: Apache::args vs Apache::Request speed
Posted by Joe Schaefer <jo...@sunstarsys.com>.
John Siracusa <si...@mindspring.com> writes:
> I'm cc-ing this to the Mac OS X Perl list in the hopes that someone can
> provide a test environment for you. (I would, but my OS X box is behind a
> firewall at work.)
>
> So how about it, macosx@perl.org folks, can any of you help get libapreq up
> and running on OS X an long last? (See message quoted below)
>
Maybe this will help:
http://fink.sourceforge.net/doc/porting/porting.html
The Unix build of libapreq goes in 3 steps:
1) build a static c library: libapreq.a
2) build object files Request.o, Cookie.o
3) link each object file against libapreq.a to
make shared libraries Request.so Cookie.so
So each file gets it's own copy of libapreq, symbols and
all. For ELF-based systems, that's not a problem since
the linking executable knows not only the library's filename,
but also the location of the desired symbol within the file
(actually there's something called a jump table inside the
.so that provides some necessary indirection, but that's a
technicality that's not particularly relevant to the issue.)
AIUI, the problem with OS/X is that their linker doesn't
provide the executable with a similar addressing scheme
for unresolved symbols; it just tells the executable which
libraries are needed to resolve them without also telling
it "where to look" for a given symbol. So because Request.bundle
and Cookie.bundle both (accidentally?) provide resolution for
libapreq's symbols, the executable gets confused and bails out.
If I'm right here, then removing the libapreq symbols from
Cookie.bundle and Request.bundle should do the trick. There
are lots of ways to reorganize things in order to achieve this,
and I'm somewhat surprised that none of the OS/X folks have
made any progress in this regard.
So maybe I'm wrong, but I don't think any of us will learn the
answer until some OS/X person actually _attempts_ to fix it.
--
Joe Schaefer
Re: Apache::args vs Apache::Request speed
Posted by Andrew Ho <an...@tellme.com>.
Heyas,
JS>Would someone PLEASE volunteer to try to compile and test
JS>apache+mod_perl & libapreq on OS/X using the experimental
JS>code I posted there? Even if you can't get it working,
JS>ANY feedback about what happened when you tried would be
JS>VERY helpful.
Slightly off topic; I'd like to help with this but I have this curious
problem. I'm trying to build Perl 5.6.itself 1 on Mac OS X (with the
latest 10.1.2 update freshly installed, using the compiler from the
developer tools CD that comes with OS X when you buy the 10.1 boxed
version) before building Apache/mod_perl.
So I go through the entire Configure sequence, and then no Makefile gets
created (it goes through the entire routine of saying it's generating a
Makefile, but whether I run Makefile.SH or have it done through Configure,
no Makefile actually ever gets created).
Has anybody else seen this really weird behavior trying to build Perl
5.6.1 on Mac OS X? A web search didn't turn up any relevant posts.
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------
Re: Apache::args vs Apache::Request speed
Posted by Matt Sergeant <ma...@sergeant.org>.
On 1 Feb 2002, Joe Schaefer wrote:
> Would someone PLEASE volunteer to try to compile and test
> apache+mod_perl & libapreq on OS/X using the experimental
> code I posted there? Even if you can't get it working,
> ANY feedback about what happened when you tried would be
> VERY helpful.
OK, if someone can communicate with me in private, seriously dumbed down
details, I can try this. I'm a libapreq committer, and have sourceforge
farm access, so I'll do my best there - though last time I tried I
couldn't get onto their OSX box...
--
<!-- Matt -->
<:->Get a smart net</:->
Re: Apache::args vs Apache::Request speed
Posted by John Siracusa <si...@mindspring.com>.
On 2/1/02 3:39 PM, Ian Ragsdale wrote:
> On the other hand, I'd be happy to compile it, but what would I need to do
> to test it?
I'm in the process of trying this too (just building a mod_perl httpd in OS
X is a bit tricky...) To test it, I think all you need to do is put these
two lines in your "startup.pl" file (or whatever):
use Apache::Request;
use Apache::Cookie;
If that works, the next step is to make an actual apache handler that uses
both the modules to actually do something. And if that works, post detailed
instructions (starting with the wget of the source tarballs :)
-John
Re: Apache::args vs Apache::Request speed
Posted by Ian Ragsdale <ia...@SKYLIST.net>.
On 2/1/02 2:21 PM, "Joe Schaefer" <jo...@sunstarsys.com> wrote:
> Ian Ragsdale <ia...@SKYLIST.net> writes:
>
>> How about setting something up on SourceForge? I know they have OS X
>> environments available for compiling and testing.
>
> apreq is an ASF project; IMO what we need now is a hero, not a
> change of venue.
>
I'm not suggesting you switch it to stay at sourceforge, I'm just saying
that they have OS X boxes you can compile & test on. Seems pretty simple to
me. I'd volunteer my own computer, but it's an iBook and is constantly
switching IP addresses due to moving around.
On the other hand, I'd be happy to compile it, but what would I need to do
to test it?
Ian
[OT] Mac OS X compilation woes (Was: Apache::args vs Apache::Request
speed)
Posted by Andrew Ho <an...@tellme.com>.
Hello,
JS>An initial build and install of:
JS> http://www.apache.org/~joes/libapreq-1.0-rc1.tar.gz
JS>
JS>on a previously-working apache 1.3.22 mod_perl 1.26 server on OS X 10.1.2
JS>with this:
JS> use Apache::Request;
JS> use Apache::Cookie;
JS>
JS>In its startup.pl file causes the following:
JS>
JS># bin/httpd -d /usr/local/apache -f conf/httpd.conf
JS>dyld: bin/httpd Undefined symbols:
JS>_ap_day_snames
JS>...
JS>_sv2request_rec
I'm having a similar problem but it's for ANY symbols in a .a that you
compile something with. e.g. say I have a C library and it lives in
/usr/local/lib/libfoo.a (include in /usr/local/include/foo.h) and exports
void foo(). If I have a test C program tester.c:
#include <foo.h>
int main { foo(); return 0 }
And I compile it so:
% cc -o tester -lfoo tester.c
And I run it, I'll get the undefined symbols error that you paste above.
This happens for me with a variety of existing open source libraries that
I've built. I theorize your problem with libapreq may stem from a similar
problem (I'm also running 10.1.2).
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------
Re: Apache::args vs Apache::Request speed
Posted by John Siracusa <si...@mindspring.com>.
On 2/1/02 3:21 PM, Joe Schaefer wrote:
> Would someone PLEASE volunteer to try to compile and test
> apache+mod_perl & libapreq on OS/X using the experimental
> code I posted there? Even if you can't get it working,
> ANY feedback about what happened when you tried would be
> VERY helpful.
(The below may not be very helpful, but I've gotta run right now. I'll try
more this weekend if I can.)
An initial build and install of:
http://www.apache.org/~joes/libapreq-1.0-rc1.tar.gz
on a previously-working apache 1.3.22 mod_perl 1.26 server on OS X 10.1.2
with this:
use Apache::Request;
use Apache::Cookie;
In its startup.pl file causes the following:
# bin/httpd -d /usr/local/apache -f conf/httpd.conf
dyld: bin/httpd Undefined symbols:
_ap_day_snames
_ap_find_path_info
_ap_get_client_block
_ap_getword
_ap_getword_conf
_ap_hard_timeout
_ap_ind
_ap_kill_timeout
_ap_log_rerror
_ap_make_array
_ap_make_dirstr_parent
_ap_make_table
_ap_month_snames
_ap_null_cleanup
_ap_pcalloc
_ap_pfclose
_ap_pfdopen
_ap_popenf
_ap_psprintf
_ap_pstrcat
_ap_pstrdup
_ap_pstrndup
_ap_push_array
_ap_register_cleanup
_ap_setup_client_block
_ap_should_client_block
_ap_table_add
_ap_table_do
_ap_table_get
_ap_table_set
_ap_table_unset
_ap_unescape_url
_hvrv2table
_mod_perl_tie_table
_perl_request_rec
_sv2request_rec
More later, I hope... :)
-John
Re: Apache::args vs Apache::Request speed
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Ian Ragsdale <ia...@SKYLIST.net> writes:
> How about setting something up on SourceForge? I know they have OS X
> environments available for compiling and testing.
apreq is an ASF project; IMO what we need now is a hero, not a
change of venue.
[...]
> > On 1/28/02 2:02 PM, Joe Schaefer <jo...@sunstarsys.com> wrote:
[...]
> >> I hope a new release will be just around the corner, but if you want
> >> to test out some of the latest stuff, have a look at
> >>
> >> http://www.apache.org/~joes/
Would someone PLEASE volunteer to try to compile and test
apache+mod_perl & libapreq on OS/X using the experimental
code I posted there? Even if you can't get it working,
ANY feedback about what happened when you tried would be
VERY helpful.
Thanks alot.
--
Joe Schaefer
Re: Apache::args vs Apache::Request speed
Posted by Ian Ragsdale <ia...@SKYLIST.net>.
How about setting something up on SourceForge? I know they have OS X
environments available for compiling and testing.
Ian
On 1/28/02 2:19 PM, "John Siracusa" <si...@mindspring.com> wrote:
> I'm cc-ing this to the Mac OS X Perl list in the hopes that someone can
> provide a test environment for you. (I would, but my OS X box is behind a
> firewall at work.)
>
> So how about it, macosx@perl.org folks, can any of you help get libapreq up
> and running on OS X an long last? (See message quoted below)
>
> -John
>
> On 1/28/02 2:02 PM, Joe Schaefer <jo...@sunstarsys.com> wrote:
>> Stas Bekman <st...@stason.org> writes:
>>> Great! Now we have an even broader benchmark. Please tell me when 1.0 is
>>> released (in case I get carried away with other things and don't notice
>>> the announce) and I'll make sure to update my benchmarking package,
>>> re-run the benchmarks and correct the results in the guide.
>>
>> Great- there's a typo or two in the handler_do sub, but they should be
>> pretty obvious when you try to run it.
>>
>> I hope a new release will be just around the corner, but if you want
>> to test out some of the latest stuff, have a look at
>>
>> http://www.apache.org/~joes/
>>
>> I don't think we'll have a 1.0 that works on OS/X, but I might be able
>> to include a patch in the distro that will build the C api of libapreq
>> directly into httpd. This might allow OS/X to run Apache::Request and
>> Apache::Cookie at the same time, but that platform is unavailable to me
>> for testing.
>
>
Re: Apache::args vs Apache::Request speed
Posted by John Siracusa <si...@mindspring.com>.
I'm cc-ing this to the Mac OS X Perl list in the hopes that someone can
provide a test environment for you. (I would, but my OS X box is behind a
firewall at work.)
So how about it, macosx@perl.org folks, can any of you help get libapreq up
and running on OS X an long last? (See message quoted below)
-John
On 1/28/02 2:02 PM, Joe Schaefer <jo...@sunstarsys.com> wrote:
> Stas Bekman <st...@stason.org> writes:
>> Great! Now we have an even broader benchmark. Please tell me when 1.0 is
>> released (in case I get carried away with other things and don't notice
>> the announce) and I'll make sure to update my benchmarking package,
>> re-run the benchmarks and correct the results in the guide.
>
> Great- there's a typo or two in the handler_do sub, but they should be
> pretty obvious when you try to run it.
>
> I hope a new release will be just around the corner, but if you want
> to test out some of the latest stuff, have a look at
>
> http://www.apache.org/~joes/
>
> I don't think we'll have a 1.0 that works on OS/X, but I might be able
> to include a patch in the distro that will build the C api of libapreq
> directly into httpd. This might allow OS/X to run Apache::Request and
> Apache::Cookie at the same time, but that platform is unavailable to me
> for testing.
Re: Apache::args vs Apache::Request speed
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Great! Now we have an even broader benchmark. Please tell me when 1.0 is
> released (in case I get carried away with other things and don't notice
> the announce) and I'll make sure to update my benchmarking package,
> re-run the benchmarks and correct the results in the guide.
Great- there's a typo or two in the handler_do sub, but they should be
pretty obvious when you try to run it.
I hope a new release will be just around the corner, but if you want
to test out some of the latest stuff, have a look at
http://www.apache.org/~joes/
I don't think we'll have a 1.0 that works on OS/X, but I might be able
to include a patch in the distro that will build the C api of libapreq
directly into httpd. This might allow OS/X to run Apache::Request and
Apache::Cookie at the same time, but that platform is unavailable to me
for testing.
--
Joe Schaefer
Re: Apache::args vs Apache::Request speed
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Well, I've run the benchmark and it wasn't the case. Did it change
>>recently? Or do you think that the benchmark is not fair?
>>
>>we are talking about this item
>>http://perl.apache.org/guide/performance.html#Apache_args_vs_Apache_Request
>>
>
> Right- param() was rewritten as XS about 6-8 months ago; since then
> I've benchmarked it a few times and found param() to be a bit faster than
> args(). We'll be releasing a 1.0 version of libapreq as soon as Jim
> approves of the current CVS version. Here's what I got using it on
> your benchmark (some differences: the tests were run against localhost
> running perl 5.00503 + mod_perl 1.26 + apache 1.3.22 and using Perl
> handlers instead of Apache::RegistryLoader scripts):
Great! Now we have an even broader benchmark. Please tell me when 1.0 is
released (in case I get carried away with other things and don't notice
the announce) and I'll make sure to update my benchmarking package,
re-run the benchmarks and correct the results in the guide.
Thanks Joe!
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: Apache::args vs Apache::Request speed
Posted by John Siracusa <si...@mindspring.com>.
On 1/27/02 3:34 PM, Joe Schaefer wrote:
> param() was rewritten as XS about 6-8 months ago; since then I've benchmarked
> it a few times and found param() to be a bit faster than args(). We'll be
> releasing a 1.0 version of libapreq as soon as Jim approves of the current CVS
> version.
Did I just read that there's a new version of libapreq coming? If so, will
it:
a) eliminate the current confusion between
J/JI/JIMW/libapreq-0.33.tar.gz
and
D/DO/DOUGM/libapreq-0.31.tar.gz
by creating a unified libapreq distribution with a version number greater
than both of the above, and
b) actually work in Mac OS X (please please please!)
(See: http://archive.develooper.com/macosx@perl.org/msg01124.html)
-John
Re: Apache::args vs Apache::Request speed
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Well, I've run the benchmark and it wasn't the case. Did it change
> recently? Or do you think that the benchmark is not fair?
>
> we are talking about this item
> http://perl.apache.org/guide/performance.html#Apache_args_vs_Apache_Request
Right- param() was rewritten as XS about 6-8 months ago; since then
I've benchmarked it a few times and found param() to be a bit faster than
args(). We'll be releasing a 1.0 version of libapreq as soon as Jim
approves of the current CVS version. Here's what I got using it on
your benchmark (some differences: the tests were run against localhost
running perl 5.00503 + mod_perl 1.26 + apache 1.3.22 and using Perl
handlers instead of Apache::RegistryLoader scripts):
Stas's strings:
my $query = [
join("&", map {"$_=".'e' x 10} ('a'..'b')),
join("&", map {"$_=".'e' x 10} ('a'..'z')),
];
Joe's strings:
%Q = qw/ one alpha two beta three gamma four delta /;
my $query = [
join("&", map "$_=$Q{$_}", keys %Q),
join("&", map "$_=".escape($_), %Q),
];
Stas's Query Joe's Query
short long short long
table 124 91 119 112
args 125 93 116 110
do 124 103 121 118
param 132 106 128 123
noparse 138 136 133 131
REQUESTS PER SECOND
Here I used ab with concurrency = 1 to avoid "complications",
but that shouldn't make a difference if we're talking subroutine
performance. The real disappointment here is handler_table,
which would be the fastest if perl's tied hash implementation
didn't suck so badly. IMO perl's performance for tied-variable
access is shameful, but apparently the problem is unfixable in
perl5.
HANDLERS:
sub handler_args {
my $r = shift;
my %args = $r->args;
$r->send_http_header('text/plain');
print join "\n", %args;
}
sub handler_table {
my $r = Apache::Request->new(shift);
my %args = %{ $r->param };
$r->send_http_header('text/plain');
print join "\n", %$args;
}
sub handler_do {
my $r = Apache::Request->new(shift);
my args; $r->param->do( sub {$args{$_[0]}=$_[1];1} );
$r->send_http_header('text/plain');
print join "\n", %$args;
}
sub handler_param {
my $r = Apache::Request->new(shift);
my %args = map +( $_ => $r->param($_) ), $r->param;
$r->send_http_header('text/plain');
print join "\n", %args;
}
sub handler_noparse {
my $r = shift;
$r->send_http_header('text/plain');
print "OK";
}
--
Joe Schaefer
Apache::args vs Apache::Request speed
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
>>mod_perl specific examples from the guide/book ($r->args vs
>>Apache::Request::param, etc)
>>
>
> Well, I've complained about that one before, and since the
> guide's text hasn't changed yet I'll try saying it again:
>
> Apache::Request::param() is FASTER THAN Apache::args(),
> and unless someone wants to rewrite args() IN C, it is
> likely to remain that way. PERIOD.
>
> Of course, if you are satisfied using Apache::args, than it would
> be silly to change "styles".
Well, I've run the benchmark and it wasn't the case. Did it change
recently? Or do you think that the benchmark is not fair?
we are talking about this item
http://perl.apache.org/guide/performance.html#Apache_args_vs_Apache_Request
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: performance coding project? (was: Re: When to cache)
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> I even have a name for the project: Speedy Code Habits :)
>
> The point is that I want to develop a coding style which tries hard to
> do early premature optimizations.
I disagree with the POV you seem to be taking wrt "write-time"
optimizations. IMO, there are precious few situations where
writing Perl in some prescribed style will lead to the fastest code.
What's best for one code segment is often a mediocre (or even stupid)
choice for another. And there's often no a-priori way to predict this
without being intimate with many dirty aspects of perl's innards.
I'm not at all against divining some abstract _principles_ for
"accelerating" a given solution to a problem, but trying to develop a
"Speedy Style" is IMO folly. My best and most universal advice would
be to learn XS (or better Inline) and use a language that was _designed_
for writing finely-tuned sections of code. But that's in the
post-working-prototype stage, *not* before.
[...]
> mod_perl specific examples from the guide/book ($r->args vs
> Apache::Request::param, etc)
Well, I've complained about that one before, and since the
guide's text hasn't changed yet I'll try saying it again:
Apache::Request::param() is FASTER THAN Apache::args(),
and unless someone wants to rewrite args() IN C, it is
likely to remain that way. PERIOD.
Of course, if you are satisfied using Apache::args, than it would
be silly to change "styles".
YMMV
--
Joe Schaefer
Re: performance coding project? (was: Re: When to cache)
Posted by Rob Nagler <na...@bivio.biz>.
> This project's idea is to give stright numbers for some definitely bad
> coding practices (e.g. map() in the void context), and things which vary
> a lot depending on the context, but are interesting to think about (e.g.
> the last example of caching the result of ref() or a method call)
I think this would be handy. I spend a fair bit of time
wondering/testing myself. Would be nice to have a repository of the
tradeoffs.
OTOH, I spend too much time mulling over unimportant performance
optimizations. The foreach/map comparison is a good example of this.
It only starts to matter (read milliseconds) at the +100KB and up
range, I find. If a site is returning 100KB pages for typical
responses, it has a problem at a completely different level than map
vs foreach.
Rob
"Pre-optimization is the root of all evil" -- C.A.R. Hoare
Re: performance coding project? (was: Re: When to cache)
Posted by Stas Bekman <st...@stason.org>.
Issac Goldstand wrote:
> Ah yes, but don't forget that to get this speed, you are sacrificing
> memory. You now have another locally scoped variable for perl to keep
> track of, which increases memory usage and general overhead (allocation
> and garbage collection). Now, those, too, are insignificant with one
> use, but the significance will probably rise with the speed gain as you
> use these techniques more often...
Yes, I know. But from the benchmark you can probably have an idea
whether the 'caching' is worth the speedup (given that the benchmark is
similar to your case). For example it depends on how many times you need
to use the cache. And how big is the value. e.g. may be caching
$foo->bar doesn't worth it, but what about $foo->bar->baz? or if you
have a deeply nested hash and you need to work with only a part of
subtree, do you grab a reference to this sub-tree node and work it or do
you keep on dereferencing all the way up to the root on every call?
But personally I still didn't decide which one is better and every time
I'm in a similar situation, I'm never sure which way to take, to cache
or not to cache. But that's the cool thing about Perl, it keeps you on
your toes all the time (if you want to :).
BTW, if somebody has interesting reasonings for using one technique
versus the other performance-wise (speed+memory), please share them.
This project's idea is to give stright numbers for some definitely bad
coding practices (e.g. map() in the void context), and things which vary
a lot depending on the context, but are interesting to think about (e.g.
the last example of caching the result of ref() or a method call)
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: performance coding project? (was: Re: When to cache)
Posted by Issac Goldstand <ma...@beamartyr.net>.
Ah yes, but don't forget that to get this speed, you are sacrificing
memory. You now have another locally scoped variable for perl to keep
track of, which increases memory usage and general overhead (allocation
and garbage collection). Now, those, too, are insignificant with one
use, but the significance will probably rise with the speed gain as you
use these techniques more often...
Issac
Stas Bekman wrote:
> Rob Nagler wrote:
>
>> Perrin Harkins writes:
>
>
>> Here's a fun example of a design flaw. It is a performance test sent
>> to another list. The author happened to work for one of our
>> competitors. :-)
>>
>>
>> That may well be the problem. Building giant strings using .= can be
>> incredibly slow; Perl has to reallocate and copy the string for each
>> append operation. Performance would likely improve in most
>> situations if an array were used as a buffer, instead. Push new
>> strings onto the array instead of appending them to a string.
>>
>> #!/usr/bin/perl -w
>> ### Append.bench ###
>>
>> use Benchmark;
>>
>> sub R () { 50 }
>> sub Q () { 100 }
>> @array = (" " x R) x Q;
>>
>> sub Append {
>> my $str = "";
>> map { $str .= $_ } @array;
>> }
>>
>> sub Push {
>> my @temp;
>> map { push @temp, $_ } @array;
>> my $str = join "", @temp;
>> }
>>
>> timethese($ARGV[0],
>> { append => \&Append,
>> push => \&Push });
>> <<
>>
>> Such a simple piece of code, yet the conclusion is incorrect. The
>> problem is in the use of map instead of foreach for the performance
>> test iterations. The result of Append is an array of whose length is
>> Q and whose elements grow from R to R * Q. Change the map to a
>> foreach and you'll see that push/join is much slower than .=.
>>
>> Return a string reference from Append. It saves a copy.
>> If this is "the page", you'll see a significant improvement in
>> performance.
>>
>> Interestingly, this couldn't be "the problem", because the hypothesis
>> is incorrect. The incorrect test just validated something that was
>> faulty to begin with. This brings up "you can't talk about it unless
>> you can measure it". Use a profiler on the actual code. Add
>> performance stats in your code. For example, we encapsulate all DBI
>> accesses and accumulate the time spent in DBI on any request. We also
>> track the time we spend processing the entire request.
>
>
> While we are at this topic, I want to suggest a new project. I was
> planning to start working on it long time ago, but other things always
> took over.
>
> The perl.apache.org/guide/performance.html and a whole bunch of
> performance chaptes in the upcoming modperl book have a lot of
> benchmarks, comparing various coding techniques. Such as the example
> you've provided. The benchmarks are doing both pure Perl and mod_perl
> specific code (which requires running Apache, a perfect job for the
> new Apache::Test framework.)
>
> Now throw in the various techniques from 'Effective Perl' book and
> voila you have a great project to learn from.
>
> Also remember that on varous platforms and various Perl versions the
> benchmark results will differ, sometimes very significantly.
>
> I even have a name for the project: Speedy Code Habits :)
>
> The point is that I want to develop a coding style which tries hard to
> do early premature optimizations. Let me give you an example of what I
> mean. Tell me what's faster:
>
> if (ref $b eq 'ARRAY'){
> $a = 1;
> }
> elsif (ref $b eq 'HASH'){
> $a = 1;
> }
>
> or:
>
> my $ref = ref $b;
> if ($ref eq 'ARRAY'){
> $a = 1;
> }
> elsif ($ref eq 'HASH'){
> $a = 1;
> }
>
> Sure, the win can be very little, but it ads up as your code base's
> size grows.
>
> Give you a similar example:
>
> if ($a->lookup eq 'ARRAY'){
> $a = 1;
> }
> elsif ($a->lookup eq 'HASH'){
> $a = 1;
> }
>
> or
>
> my $lookup = $a->lookup;
> if ($lookup eq 'ARRAY'){
> $a = 1;
> }
> elsif ($lookup eq 'HASH'){
> $a = 1;
> }
>
> now throw in sub attributes and re-run the test again.
>
> add examples of map vs for.
> add examples of method lookup vs. procedures
> add examples of concat vs. list vs. other stuff from the guide.
>
> mod_perl specific examples from the guide/book ($r->args vs
> Apache::Request::param, etc)
>
> If you understand where I try to take you, help me to pull this
> project off and I think in a long run we can benefit a lot.
>
> This goes along with the Apache::Benchmark project I think (which is
> yet another thing I want to start...), probably could have these two
> ideas put together.
>
> _____________________________________________________________________
Re: performance coding project? (was: Re: When to cache)
Posted by raptor <ra...@unacs.bg>.
One memory & speed saving is using global VARS, I know it is not
recomended practice, but if from the begining of the project u set a
convention what are the names of global vars it is ok..F.e. I'm using in
all DB pages at the begining :
our $dbh = dbConnect() or dbiError();
See I know (i'm sure) that when I use DB I will always initialize the
var. One other example is (ASP.pm):
our $userID = $$Session{userID};
my $something = $$Request{Params}{something}
This is not saving me memory, but shorten my typewriting especialy if it
is used frequently or if I need to change FORM-param from "something" to
"anything"..etc..
I think in mod_perl world we are about to search MEMORY optimisation
RATHER speed optimisation... :")
raptor
raptor@unacs.bg
Re: performance coding project? (was: Re: When to cache)
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
> Back to your idea: you're obviously interested in the low-level
> optimization stuff, so of course you should go ahead with it. I don't
> think it needs to be a separate project, but improvements to the
> performance section of the guide are always a good idea.
It has to be a run-able code, so people can verify the facts which may
change with different OS/versions of Perl. e.g. Joe says that $r->args
is slower then Apache::Request->param, I saw the opposite. Having these
as a run-able bits, is much nicer.
> I know that I
> have taken all of the DBI performance tips to heart and found them very
> useful.
:)
That's mostly JWB's work I think.
> I'm more interested in writing about higher level performance issues
> (efficient shared data, config tuning, caching), so I'll continue to
> work on those things. I'm submitting a proposal for a talk on data
> sharing techniques at this year's Perl Conference, so hopefully I can
> contribute that to the guide after I finish it.
Go Perrin!
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: performance coding project? (was: Re: When to cache)
Posted by Ask Bjoern Hansen <as...@valueclick.com>.
On Sat, 26 Jan 2002, Perrin Harkins wrote:
> > It all depends on what kind of application do you have. If you code is
> > CPU-bound these seemingly insignificant optimizations can have a very
> > significant influence on the overall service performance.
>
> Do such beasts really exist? I mean, I guess they must, but I've never
> seen a mod_perl application that was CPU-bound. They always seem to be
> constrained by database speed and memory.
At ValueClick we only have a few BerkeleyDBs that are in the
"request loop" for 99% of the traffic; everything else is in fairly
efficient in-memory data structures.
So there we do of course care about the tiny small optimiziations
because there's a direct correlation between saved CPU cycles and
request capacity.
However, it's only that way because we made a good design for the
application in the first place. :-) (And for all the other code we
rarely care about using a few more CPU cycles if it is
easier/cleaner/more flexible/comes to mind first). Who cares if the
perl code gets ready to wait for the database a few milliseconds
faster? :-)
- ask
--
ask bjoern hansen, http://ask.netcetera.dk/ !try; do();
more than a billion impressions per week, http://valueclick.com
Re: performance coding project? (was: Re: When to cache)
Posted by Ed Grimm <ed...@tgape.org>.
On Sat, 26 Jan 2002, Perrin Harkins wrote:
>> It all depends on what kind of application do you have. If you code
>> is CPU-bound these seemingly insignificant optimizations can have a
>> very significant influence on the overall service performance.
>
> Do such beasts really exist? I mean, I guess they must, but I've
> never seen a mod_perl application that was CPU-bound. They always
> seem to be constrained by database speed and memory.
I've seen one. However, it was much like a normal performance problem -
the issue was with one loop which ran one line which was quite
pathological. Replacing loop with an s///eg construct eliminated the
problem; there was no need for seemlingly insignificant optimizations.
(Actually, the problem was *created* by premature optimization - the
coder had utilized code that was more efficient than s/// in one special
case, to handle a vastly different instance.)
However, there could conceivably be code which was more of a performance
issue, especially when the mod_perl utilizes a very successful cache on
a high traffic site.
>> On the other hand how often do you get a chance to profile your code
>> and see how to improve its speed in the real world. Managers never
>> plan for debugging period, not talking about optimizations periods.
Unless there's already a problem, and you have a good manager. We've
had a couple of instances where we were given time (on the schedule,
before the release) to improve speed after a release. It's quite rare,
though, and I've never seen it for a mod_perl project.
Ed
Re: performance coding project? (was: Re: When to cache)
Posted by Ged Haywood <ge...@www2.jubileegroup.co.uk>.
Hi all,
Stas has a point. Perl makes it very easy to do silly things.
This is what I was doing last week:
if( m/\b$Needle\b/ ) {...}
Eight hours. (Silly:)
if( index($Haystack,$Needle) && m/\b$Needle\b/ ) {...}
Twelve minutes.
73,
Ged.
Re: performance coding project? (was: Re: When to cache)
Posted by Milo Hyson <mi...@cyberlifelabs.com>.
On Saturday 26 January 2002 03:40 pm, Sam Tregar wrote:
> Think search engines. Once you've figured out how to get your search
> database to fit in memory (or devised a cachin strategy to get the
> important parts there) you're essentially looking at a CPU-bound problem.
> These days the best solution is probably some judicious use of Inline::C.
> Back when I last tackled the problem I had to hike up mount XS to find my
> grail...
I agree. There are some situations that are just too complex for a DBMS to
handle directly, at least in any sort of efficient fashion. However,
depending on the load in those cases, Perrin's solution for eToys is probably
a good approach (i.e. custom search software written in C/C++).
--
Milo Hyson
CyberLife Labs, LLC
Re: performance coding project? (was: Re: When to cache)
Posted by Sam Tregar <sa...@tregar.com>.
On Sat, 26 Jan 2002, Perrin Harkins wrote:
> > It all depends on what kind of application do you have. If you code is
> > CPU-bound these seemingly insignificant optimizations can have a very
> > significant influence on the overall service performance.
>
> Do such beasts really exist? I mean, I guess they must, but I've never
> seen a mod_perl application that was CPU-bound. They always seem to be
> constrained by database speed and memory.
Think search engines. Once you've figured out how to get your search
database to fit in memory (or devised a cachin strategy to get the
important parts there) you're essentially looking at a CPU-bound problem.
These days the best solution is probably some judicious use of Inline::C.
Back when I last tackled the problem I had to hike up mount XS to find my
grail...
-sam
Re: performance coding project? (was: Re: When to cache)
Posted by Perrin Harkins <pe...@elem.com>.
> It all depends on what kind of application do you have. If you code is
> CPU-bound these seemingly insignificant optimizations can have a very
> significant influence on the overall service performance.
Do such beasts really exist? I mean, I guess they must, but I've never
seen a mod_perl application that was CPU-bound. They always seem to be
constrained by database speed and memory.
> On the other hand how often do you get a chance to profile your code
and
> see how to improve its speed in the real world. Managers never plan
> for debugging period, not talking about optimizations periods.
If you plan a good architecture that avoids the truly slow stuff
(disk/network access) as much as possible, your application is usually
fast enough without spending much time on optimization (except maybe
some database tuning). At my last couple of jobs we actually did have
load testing and optimization as part of the development plan, but
that's because we knew we'd be getting pretty high levels of traffic.
Most people don't need to tune very much if they have a good
architecture, and it's enough for them to fix problems as they become
visible.
Back to your idea: you're obviously interested in the low-level
optimization stuff, so of course you should go ahead with it. I don't
think it needs to be a separate project, but improvements to the
performance section of the guide are always a good idea. I know that I
have taken all of the DBI performance tips to heart and found them very
useful.
I'm more interested in writing about higher level performance issues
(efficient shared data, config tuning, caching), so I'll continue to
work on those things. I'm submitting a proposal for a talk on data
sharing techniques at this year's Perl Conference, so hopefully I can
contribute that to the guide after I finish it.
- Perrin
Re: performance coding project? (was: Re: When to cache)
Posted by Ask Bjoern Hansen <as...@valueclick.com>.
On Sat, 26 Jan 2002, Stas Bekman wrote:
[...]
> > It's much better to build your system, profile it, and fix the bottlenecks.
> > The most effective changes are almost never simple coding changes like the
> > one you showed, but rather large things like using qmail-inject instead of
> > SMTP, caching a slow database query or method call, or changing your
> > architecture to reduce the number of network accesses or inter-process
> > communications.
>
> It all depends on what kind of application do you have. If you code is
> CPU-bound these seemingly insignificant optimizations can have a very
> significant influence on the overall service performance. Of course if
> you app, is IO-bound or depends with some external component, than your
> argumentation applies.
Eh, any real system will be a combination. Sure; when everything
works then it's worth finding the CPU intensive places and fix them
up, but for the most part the system design is far far more
important than any "code optimiziation" you can ever do.
My usual rhetorics: Your average code optimization will gain you at
most a few percent performance gain. A better design can often make
things 10 times faster and use only a fraction of your memory.
> On the other hand how often do you get a chance to profile your code and
> see how to improve its speed in the real world. Managers never plan
> for debugging period, not talking about optimizations periods. And while
> premature optimizations are usually evil, as they will bait you later,
> knowing the differences between coding styles does help in a long run
> and I don't consider these as premature optimizations.
If you don't waste time profiling every little snippet of code you
might have more time to fix the real bottlenecks in the end. ;-)
[...]
> All I want to say is that there is no one-fits-all solution in Perl,
> because of TIMTOWTDI, so you can learn a lot from running benchmarks and
> picking your favorite coding style and change it as the language
> evolves. But you shouldn't blindly apply the outcomes of the benchmarks
> without running your own benchmarks.
Amen.
(And don't get me wrong; I think a repository of information about
the nitty gritty optimization things would be great - I just find it
to be bad advice to not tell people to do the proper design first).
- ask
--
ask bjoern hansen, http://ask.netcetera.dk/ !try; do();
more than a billion impressions per week, http://valueclick.com
Re: performance coding project? (was: Re: When to cache)
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
>>The point is that I want to develop a coding style which tries hard to
>>do early premature optimizations.
>>
>
> We've talked about this kind of thing before. My opinion is still the same
> as it was: low-level speed optimization before you have a working system is
> a waste of your time.
>
> It's much better to build your system, profile it, and fix the bottlenecks.
> The most effective changes are almost never simple coding changes like the
> one you showed, but rather large things like using qmail-inject instead of
> SMTP, caching a slow database query or method call, or changing your
> architecture to reduce the number of network accesses or inter-process
> communications.
It all depends on what kind of application do you have. If you code is
CPU-bound these seemingly insignificant optimizations can have a very
significant influence on the overall service performance. Of course if
you app, is IO-bound or depends with some external component, than your
argumentation applies.
On the other hand how often do you get a chance to profile your code and
see how to improve its speed in the real world. Managers never plan
for debugging period, not talking about optimizations periods. And while
premature optimizations are usually evil, as they will bait you later,
knowing the differences between coding styles does help in a long run
and I don't consider these as premature optimizations.
Definitely this discussion has no end. Everybody is right in their
particular project, since there are no two projects which are the same.
All I want to say is that there is no one-fits-all solution in Perl,
because of TIMTOWTDI, so you can learn a lot from running benchmarks and
picking your favorite coding style and change it as the language
evolves. But you shouldn't blindly apply the outcomes of the benchmarks
without running your own benchmarks.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: performance coding project? (was: Re: When to cache)
Posted by Tatsuhiko Miyagawa <mi...@edge.co.jp>.
On Fri, 25 Jan 2002 21:15:54 +0000 (GMT)
Matt Sergeant <ma...@sergeant.org> wrote:
>
> With qmail, SMTP generally uses inetd, which is slow, or daemontools,
> which is faster, but still slow, and more importantly, it anyway goes:
>
> perl -> SMTP -> inetd -> qmail-smtpd -> qmail-inject.
>
> So with going direct to qmail-inject, your email skips out a boat load of
> processing and goes direct into the queue.
>
> Of course none of this is relevant if you're not using qmail ;-)
Yet another solution:
use Mail::QmailQueue, directly
http://search.cpan.org/search?dist=Mail-QmailQueue
--
Tatsuhiko Miyagawa <mi...@bulknews.net>
Re: performance coding project? (was: Re: When to cache)
Posted by David Wheeler <da...@wheeler.net>.
On Fri, 2002-01-25 at 13:15, Matt Sergeant wrote:
> With qmail, SMTP generally uses inetd, which is slow, or daemontools,
> which is faster, but still slow, and more importantly, it anyway goes:
>
> perl -> SMTP -> inetd -> qmail-smtpd -> qmail-inject.
>
> So with going direct to qmail-inject, your email skips out a boat load of
> processing and goes direct into the queue.
Okay, that makes sense. In my activitymail CVS script I just used
sendmail.
http://www.cpan.org/authors/id/D/DW/DWHEELER/activitymail-0.987
But it looks like this might be more efficient, if qmail happens to be
installed (not sure on SourceForge's servers).
> Of course none of this is relevant if you're not using qmail ;-)
Yes, and in Bricolage, I used Net::SMTP to keep it as
platform-independent as possible. It should work on Windows, even!
Besides, all mail gets sent during the Apache cleanup phase, so there
should be no noticeable delay for users.
David
--
David Wheeler AIM: dwTheory
david@wheeler.net ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory@jabber.org
Re: performance coding project? (was: Re: When to cache)
Posted by Matt Sergeant <ma...@sergeant.org>.
On 25 Jan 2002, David Wheeler wrote:
> On Fri, 2002-01-25 at 09:08, Perrin Harkins wrote:
>
> <snip />
>
> > It's much better to build your system, profile it, and fix the bottlenecks.
> > The most effective changes are almost never simple coding changes like the
> > one you showed, but rather large things like using qmail-inject instead of
> > SMTP, caching a slow database query or method call, or changing your
> > architecture to reduce the number of network accesses or inter-process
> > communications.
>
> qmail-inject? I've just been using sendmail or, preferentially,
> Net::SMTP. Isn't using a system call more expensive? If not, how does
> qmail-inject work?
With qmail, SMTP generally uses inetd, which is slow, or daemontools,
which is faster, but still slow, and more importantly, it anyway goes:
perl -> SMTP -> inetd -> qmail-smtpd -> qmail-inject.
So with going direct to qmail-inject, your email skips out a boat load of
processing and goes direct into the queue.
Of course none of this is relevant if you're not using qmail ;-)
--
<!-- Matt -->
<:->Get a smart net</:->
Re: performance coding project? (was: Re: When to cache)
Posted by David Wheeler <da...@wheeler.net>.
On Fri, 2002-01-25 at 09:08, Perrin Harkins wrote:
<snip />
> It's much better to build your system, profile it, and fix the bottlenecks.
> The most effective changes are almost never simple coding changes like the
> one you showed, but rather large things like using qmail-inject instead of
> SMTP, caching a slow database query or method call, or changing your
> architecture to reduce the number of network accesses or inter-process
> communications.
qmail-inject? I've just been using sendmail or, preferentially,
Net::SMTP. Isn't using a system call more expensive? If not, how does
qmail-inject work?
Thanks,
David
--
David Wheeler AIM: dwTheory
david@wheeler.net ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory@jabber.org
Re: performance coding project? (was: Re: When to cache)
Posted by Perrin Harkins <pe...@elem.com>.
> The point is that I want to develop a coding style which tries hard to
> do early premature optimizations.
We've talked about this kind of thing before. My opinion is still the same
as it was: low-level speed optimization before you have a working system is
a waste of your time.
It's much better to build your system, profile it, and fix the bottlenecks.
The most effective changes are almost never simple coding changes like the
one you showed, but rather large things like using qmail-inject instead of
SMTP, caching a slow database query or method call, or changing your
architecture to reduce the number of network accesses or inter-process
communications.
The exception to this rule is that I do advocate thinking about memory usage
from the beginning. There are no good tools for profiling memory used by
Perl, so you can't easily find the offenders later on. Being careful about
passing references, slurping files, etc. pays off in better scalability
later.
- Perrin
performance coding project? (was: Re: When to cache)
Posted by Stas Bekman <st...@stason.org>.
Rob Nagler wrote:
> Perrin Harkins writes:
> Here's a fun example of a design flaw. It is a performance test sent
> to another list. The author happened to work for one of our
> competitors. :-)
>
>
> That may well be the problem. Building giant strings using .= can be
> incredibly slow; Perl has to reallocate and copy the string for each
> append operation. Performance would likely improve in most
> situations if an array were used as a buffer, instead. Push new
> strings onto the array instead of appending them to a string.
>
> #!/usr/bin/perl -w
> ### Append.bench ###
>
> use Benchmark;
>
> sub R () { 50 }
> sub Q () { 100 }
> @array = (" " x R) x Q;
>
> sub Append {
> my $str = "";
> map { $str .= $_ } @array;
> }
>
> sub Push {
> my @temp;
> map { push @temp, $_ } @array;
> my $str = join "", @temp;
> }
>
> timethese($ARGV[0],
> { append => \&Append,
> push => \&Push });
> <<
>
> Such a simple piece of code, yet the conclusion is incorrect. The
> problem is in the use of map instead of foreach for the performance
> test iterations. The result of Append is an array of whose length is
> Q and whose elements grow from R to R * Q. Change the map to a
> foreach and you'll see that push/join is much slower than .=.
>
> Return a string reference from Append. It saves a copy.
> If this is "the page", you'll see a significant improvement in
> performance.
>
> Interestingly, this couldn't be "the problem", because the hypothesis
> is incorrect. The incorrect test just validated something that was
> faulty to begin with. This brings up "you can't talk about it unless
> you can measure it". Use a profiler on the actual code. Add
> performance stats in your code. For example, we encapsulate all DBI
> accesses and accumulate the time spent in DBI on any request. We also
> track the time we spend processing the entire request.
While we are at this topic, I want to suggest a new project. I was
planning to start working on it long time ago, but other things always
took over.
The perl.apache.org/guide/performance.html and a whole bunch of
performance chaptes in the upcoming modperl book have a lot of
benchmarks, comparing various coding techniques. Such as the example
you've provided. The benchmarks are doing both pure Perl and mod_perl
specific code (which requires running Apache, a perfect job for the new
Apache::Test framework.)
Now throw in the various techniques from 'Effective Perl' book and voila
you have a great project to learn from.
Also remember that on varous platforms and various Perl versions the
benchmark results will differ, sometimes very significantly.
I even have a name for the project: Speedy Code Habits :)
The point is that I want to develop a coding style which tries hard to
do early premature optimizations. Let me give you an example of what I
mean. Tell me what's faster:
if (ref $b eq 'ARRAY'){
$a = 1;
}
elsif (ref $b eq 'HASH'){
$a = 1;
}
or:
my $ref = ref $b;
if ($ref eq 'ARRAY'){
$a = 1;
}
elsif ($ref eq 'HASH'){
$a = 1;
}
Sure, the win can be very little, but it ads up as your code base's size
grows.
Give you a similar example:
if ($a->lookup eq 'ARRAY'){
$a = 1;
}
elsif ($a->lookup eq 'HASH'){
$a = 1;
}
or
my $lookup = $a->lookup;
if ($lookup eq 'ARRAY'){
$a = 1;
}
elsif ($lookup eq 'HASH'){
$a = 1;
}
now throw in sub attributes and re-run the test again.
add examples of map vs for.
add examples of method lookup vs. procedures
add examples of concat vs. list vs. other stuff from the guide.
mod_perl specific examples from the guide/book ($r->args vs
Apache::Request::param, etc)
If you understand where I try to take you, help me to pull this project
off and I think in a long run we can benefit a lot.
This goes along with the Apache::Benchmark project I think (which is yet
another thing I want to start...), probably could have these two ideas
put together.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: When to cache
Posted by Rob Nagler <na...@bivio.biz>.
Perrin Harkins writes:
> To fix this, we moved to not generating anything until it was requested. We
> would fetch the data the first time it was asked for, and then cache it for
> future requests. (I think this corresponds to your option 2.) Of course
> then you have to decide on a cache consistency approach for keeping that
> data fresh. We used a simple TTL approach because it was fast and easy to
> implement ("good enough").
I'd be curious to know the cache hit stats. BTW, this case seems to
be an example of immutable data, which is definitely worth caching if
performance dictates.
> However, for many of us caching is a necessity for decent
> performance.
I agree with latter clause, but take issue with the former. Typical
sites get a few hits a second at peak times. If a site isn't
returning "typical" pages in under a second using mod_perl, it
probably has some type of basic problem imo.
A common problem is a missing database index. Another is too much
memory allocation, e.g. passing around a large scalar instead of a
reference or overuse of objects (classical Java problem). It isn't
always the case that you can fix the problem, but caching doesn't fix
it either. At least understand the performance problem(s) thoroughly
before adding the cache.
Here's a fun example of a design flaw. It is a performance test sent
to another list. The author happened to work for one of our
competitors. :-)
>>
That may well be the problem. Building giant strings using .= can be
incredibly slow; Perl has to reallocate and copy the string for each
append operation. Performance would likely improve in most
situations if an array were used as a buffer, instead. Push new
strings onto the array instead of appending them to a string.
#!/usr/bin/perl -w
### Append.bench ###
use Benchmark;
sub R () { 50 }
sub Q () { 100 }
@array = (" " x R) x Q;
sub Append {
my $str = "";
map { $str .= $_ } @array;
}
sub Push {
my @temp;
map { push @temp, $_ } @array;
my $str = join "", @temp;
}
timethese($ARGV[0],
{ append => \&Append,
push => \&Push });
<<
Such a simple piece of code, yet the conclusion is incorrect. The
problem is in the use of map instead of foreach for the performance
test iterations. The result of Append is an array of whose length is
Q and whose elements grow from R to R * Q. Change the map to a
foreach and you'll see that push/join is much slower than .=.
Return a string reference from Append. It saves a copy.
If this is "the page", you'll see a significant improvement in
performance.
Interestingly, this couldn't be "the problem", because the hypothesis
is incorrect. The incorrect test just validated something that was
faulty to begin with. This brings up "you can't talk about it unless
you can measure it". Use a profiler on the actual code. Add
performance stats in your code. For example, we encapsulate all DBI
accesses and accumulate the time spent in DBI on any request. We also
track the time we spend processing the entire request.
Adding a cache is piling more code onto a solution. It sometimes is
like adding lots of salt to bad cooking. You do it when you have to,
but you end up paying for it later.
Sorry if my post seems pedantic or obvious. I haven't seen this type
of stuff discussed much in this particular context. Besides I'm a
contrarian. ;-)
Rob
Re: New mod_perl Logo
Posted by Gunther Birznieks <gu...@extropia.com>.
I agree. mod_perl is a technology not a platform. Java is called Java,
Servlets are called Servlets, but products on top of the technology (eg app
servers) are usually have the name "Orwellian Pearlz Factory".
If you wanted to use a marketing name, you should have just told them you
were using one of the frameworks on top of mod_perl like AxKit or Mason.
At 08:59 AM 1/30/2002, Robin Berjon wrote:
>Hi ct !
>
>On Wednesday 30 January 2002 01:29, Chris Thompson wrote:
> > Well, I'd like to just throw one idea into the mix. It's something that's
> > bugged me for a long time, no better time than the present.
> >
> > "mod_perl" is a lousy name.
> >
> > There, I've said it.
>
>This discussion has occured already. The conclusion was simple: the name
>won't change. Doug cut the talk saying that he won't change the name, and I
>must say I agree.
>
>That isn't to say your points are invalid, you're concern is justified. The
>solution here is not to change modperl's name. modperl, in a way, is a low
>level thing. It's "just" Perl + Apache. When in need of a convincing name,
>say "I use Foo" where Foo is the name of something that runs on top of
>modperl. "I use AxKit", "I use FooAppFramework", etc. can sound a lot better.
>
>Of course, on top of that you need a good looking website with marketing
>talk. In any case changing the name alone doesn't help. Website content
>welcome ;-)
>
>--
>_______________________________________________________________________
>Robin Berjon <ro...@knowscape.com> -- CTO
>k n o w s c a p e : // venture knowledge agency www.knowscape.com
>-----------------------------------------------------------------------
>Paranoids are people, too; they have their own problems. It's easy
>to criticize, but if everybody hated you, you'd be paranoid too.
__________________________________________________
Gunther Birznieks (gunther.birznieks@eXtropia.com)
eXtropia - The Open Web Technology Company
http://www.eXtropia.com/
Re: New mod_perl Logo
Posted by Joe Pearson <jo...@webdms.com>.
I really don't think a catchy name would have helped in your case. Your
management, like many others, prefer to "manage by magazine". And we all
know who is in all the magazines.
If your management did not like Perl, why not try java, c or php?
If your management did not like MySQL, why not Postgres, Sybase, Oracle,
Informix?
My point is it would not matter. The issue was not Perl, the real issue was
that it was not a Microsoft product.
----- Original Message -----
From: "Chris Thompson" <ct...@cthompson.com>
To: <mo...@apache.org>
Sent: Tuesday, January 29, 2002 5:29 PM
Subject: Re: New mod_perl Logo
> On Tue, 29 Jan 2002, Jonathan M. Hollin wrote:
> > Ideas for logos, banners, "powered by"-type buttons are all welcome.
>
> Well, I'd like to just throw one idea into the mix. It's something that's
> bugged me for a long time, no better time than the present.
>
> "mod_perl" is a lousy name.
>
> There, I've said it.
>
> For any number of reasons, perl does now and will always face an uphill
> struggle in any "Enterprise" application.
>
> For example, at my place of employment, we just went through a rather
> arduous task that I fought against and lost.
>
> We had a production site that handled a decent amount of traffic. Apache,
> mod_perl, Linux and MySQL. It ran and ran with almost no intervention.
>
> The management team of the company that bought us a year ago had been
trying
> to force a change in the product by throwing up various arguments, which
> were always false. ("MySQL doesnt support Transaction", "Yes it does".
"But
> they arent atomic.", "Yes they are". "Well, you can't roll them back.",
"Yes
> you can.")
>
> In the end, I lost. From October to mid January they set about taking our
> fully functional product and "replatforming" it to Win2k/IIS/ASP+VB/MSSQL.
> The final reason? "Responsible enterprises do not use perl."
>
> mod_perl needs a name. Something marketable, something catchy. The java
> folks learned that a long time ago. Their systems are called
> "Tomcat/Jakarta" and "Cocoon" and "Resin".
>
> THAT, in my opinion, is what should happen for mod_perl 2.0. It should be
> "Adirondack" or "Orwell" or any other generic, innocuous name. Even
> "MonkeyButter v1.0" is probably a better deal than mod_perl.
>
> As for logos, Avoiding Camels or Eagles is a requirement. I don't blame
ORA
> for requiring the trademark notices, the twisted concepts of US trademark
> law REQUIRE them to do that. The first time they didnt, they could lose
> their trademark. But we should have a "mascot" that makes sense and is
OURS.
> Linux has the penguin, OpenBSD has the blowfish, the other BSD's have the
> devil. Those images are clearly associated with those products, and can be
> used WITHOUT corporate approval.
>
> --
> _______________
> Chris Thompson
Re: New mod_perl Logo
Posted by Robin Berjon <ro...@knowscape.com>.
Hi ct !
On Wednesday 30 January 2002 01:29, Chris Thompson wrote:
> Well, I'd like to just throw one idea into the mix. It's something that's
> bugged me for a long time, no better time than the present.
>
> "mod_perl" is a lousy name.
>
> There, I've said it.
This discussion has occured already. The conclusion was simple: the name
won't change. Doug cut the talk saying that he won't change the name, and I
must say I agree.
That isn't to say your points are invalid, you're concern is justified. The
solution here is not to change modperl's name. modperl, in a way, is a low
level thing. It's "just" Perl + Apache. When in need of a convincing name,
say "I use Foo" where Foo is the name of something that runs on top of
modperl. "I use AxKit", "I use FooAppFramework", etc. can sound a lot better.
Of course, on top of that you need a good looking website with marketing
talk. In any case changing the name alone doesn't help. Website content
welcome ;-)
--
_______________________________________________________________________
Robin Berjon <ro...@knowscape.com> -- CTO
k n o w s c a p e : // venture knowledge agency www.knowscape.com
-----------------------------------------------------------------------
Paranoids are people, too; they have their own problems. It's easy
to criticize, but if everybody hated you, you'd be paranoid too.
Re: New mod_perl Logo
Posted by Aaron Johnson <so...@gina.net>.
Thanks for sharing your opinion and it has brought up some of my own
that I had held in reserve for sometime now.
I would 100% agree with a claim that a name is effecting usage if we
were selling a food product or something else to the general public, but
even then far more products have failed because they weren't properly
explained to the public. mod_perl is a beast of tool/product. It
harnesses the full power of Apache web server and Perl to allow you do
dam near anything with the data you need to handle. Apache because of
its large platform support and well designed architecture has proven
that it is enterprise ready. That brings me to the rub of this
discussion however, Perl. I love Perl, I use Perl for almost every
programming task that I need in my work and personal computing use.
That isn't enough though. Perl has a rep, MySQL has a rep too. These
*once* true statements such as "Its just a scripting language for Unix"
or "It doesn't support transactions" etc. are becoming more and more the
cripples of the acceptance not the name. These great and power products
don't have someone dispelling the myths on national television while
none computer managers are watching TV, they only remember what the last
consultant said. I am not knocking managers I am just showing human
nature, that is we don't change our opinions unless someone we trust
(the TV? ) more then the last person explains clearly why we should.
We have had several discussions over the last three years on this list
about advocating mod_perl. I think what it really boils down to is a
polished web presence and a strong statement of the power and efficiency
of the product at hand and in this case a <place your favorite open
source application here> page to refute known myths.
Aaron Johnson
Chris Thompson wrote:
>
> On Tue, 29 Jan 2002, Jonathan M. Hollin wrote:
> > Ideas for logos, banners, "powered by"-type buttons are all welcome.
>
> Well, I'd like to just throw one idea into the mix. It's something that's
> bugged me for a long time, no better time than the present.
>
> "mod_perl" is a lousy name.
>
> There, I've said it.
>
> For any number of reasons, perl does now and will always face an uphill
> struggle in any "Enterprise" application.
>
> For example, at my place of employment, we just went through a rather
> arduous task that I fought against and lost.
>
> We had a production site that handled a decent amount of traffic. Apache,
> mod_perl, Linux and MySQL. It ran and ran with almost no intervention.
>
> The management team of the company that bought us a year ago had been trying
> to force a change in the product by throwing up various arguments, which
> were always false. ("MySQL doesnt support Transaction", "Yes it does". "But
> they arent atomic.", "Yes they are". "Well, you can't roll them back.", "Yes
> you can.")
>
> In the end, I lost. From October to mid January they set about taking our
> fully functional product and "replatforming" it to Win2k/IIS/ASP+VB/MSSQL.
> The final reason? "Responsible enterprises do not use perl."
>
> mod_perl needs a name. Something marketable, something catchy. The java
> folks learned that a long time ago. Their systems are called
> "Tomcat/Jakarta" and "Cocoon" and "Resin".
>
> THAT, in my opinion, is what should happen for mod_perl 2.0. It should be
> "Adirondack" or "Orwell" or any other generic, innocuous name. Even
> "MonkeyButter v1.0" is probably a better deal than mod_perl.
>
> As for logos, Avoiding Camels or Eagles is a requirement. I don't blame ORA
> for requiring the trademark notices, the twisted concepts of US trademark
> law REQUIRE them to do that. The first time they didnt, they could lose
> their trademark. But we should have a "mascot" that makes sense and is OURS.
> Linux has the penguin, OpenBSD has the blowfish, the other BSD's have the
> devil. Those images are clearly associated with those products, and can be
> used WITHOUT corporate approval.
>
> --
> _______________
> Chris Thompson
Re: New mod_perl Logo
Posted by Gunther Birznieks <gu...@extropia.com>.
How could another name look nice on your resume if they don't know what it is ?
Why don't you just "nickname" mod_perl as something else and then put THAT
name on your resume if you are so concerned about naming.
At 09:36 PM 1/30/2002, Paul Cotter wrote:
> > > "mod_perl" is a lousy name.
>
>It is causing me a problem. My potential customers have heard of Perl and
>Apache, MySql and Postgres, but they dot like the idea of perl modifying the
>Apache processing. It strikes them as tinkering round with the internals and
>liable to cause problems 'when we upgrade' or 'move to another platform'. It
>also does not look good on a resume when you are sending it to someone who
>has never heard of it. You are never quite sure whether to Wordcap it or
>not. Give it some other marketting name, even if it keeps its original name
>in places like this. Anything will do, WebBlast, Insiouxiance, Perlandra,
>Exsight, Insite, HowtoSite - I really do not mind.
>
>Regards - Paul Cotter
__________________________________________________
Gunther Birznieks (gunther.birznieks@eXtropia.com)
eXtropia - The Open Web Technology Company
http://www.eXtropia.com/
Re: New mod_perl Logo
Posted by Gunther Birznieks <gu...@extropia.com>.
At 04:50 PM 1/31/2002, brian moseley wrote:
>On Wed, 30 Jan 2002, Paul Cotter wrote:
>
> > Give it some other marketting name, even if it keeps its
> > original name in places like this.
>
>didn't you people read perrin's message?
>
>do you think this is the first time this topic has been
>discussed? do you think it's gonna change doug's mind /this/
>time?
>
>if you can't sell your customers and bosses on mod_perl
>because of its technological appropriateness, the name of
>the software is the least of your problems.
I was thinking if we renamed mod_perl as "George Bush" it would do quite
well in the political polls. Might even be elected for president. And then
we'd rule the country!!
Re: New mod_perl Logo
Posted by Ron Savage <ro...@savage.net.au>.
Paul
How about attaching a copy of _the_ mod_perl book each time you send out your resume. The recipients should be impressed and thus
realize it's a Real Program, and I'm sure the authors would consider a bulk discount :-).
(Having worked only 2 out of the last 8 months I do appreciate your position).
Cheers
Ron Savage
ron@savage.net.au
http://savage.net.au/index.html
----- Original Message -----
From: "Paul Cotter" <pa...@powermagic.net>
To: <mo...@apache.org>
Sent: Thursday, January 31, 2002 12:36 AM
Subject: Re: New mod_perl Logo
>
> > > "mod_perl" is a lousy name.
>
> It is causing me a problem. My potential customers have heard of Perl and
Re: New mod_perl Logo
Posted by brian moseley <bc...@maz.org>.
On Wed, 30 Jan 2002, Paul Cotter wrote:
> Give it some other marketting name, even if it keeps its
> original name in places like this.
didn't you people read perrin's message?
do you think this is the first time this topic has been
discussed? do you think it's gonna change doug's mind /this/
time?
if you can't sell your customers and bosses on mod_perl
because of its technological appropriateness, the name of
the software is the least of your problems.
Re: New mod_perl Logo
Posted by Paul Cotter <pa...@powermagic.net>.
> > "mod_perl" is a lousy name.
It is causing me a problem. My potential customers have heard of Perl and
Apache, MySql and Postgres, but they dot like the idea of perl modifying the
Apache processing. It strikes them as tinkering round with the internals and
liable to cause problems 'when we upgrade' or 'move to another platform'. It
also does not look good on a resume when you are sending it to someone who
has never heard of it. You are never quite sure whether to Wordcap it or
not. Give it some other marketting name, even if it keeps its original name
in places like this. Anything will do, WebBlast, Insiouxiance, Perlandra,
Exsight, Insite, HowtoSite - I really do not mind.
Regards - Paul Cotter
Re: New mod_perl Logo
Posted by Dave Hodgkinson <da...@davehodgkinson.com>.
Ged Haywood <ge...@www2.jubileegroup.co.uk> writes:
> Hi there,
>
> On Tue, 29 Jan 2002, Chris Thompson wrote:
>
> > "mod_perl" is a lousy name.
> [snip]
> > mod_perl needs a name. Something marketable, something catchy.
>
> How about "BigFoot"?
Sasquatch.
--
Dave Hodgkinson, Wizard for Hire http://www.davehodgkinson.com
Editor-in-chief, The Highway Star http://www.deep-purple.com
Interim Technical Director, Web Architecture Consultant for hire
[OT] Re: New mod_perl name was [Re: New mod_perl Logo]
Posted by Joe Schaefer <jo...@sunstarsys.com>.
___cliff rayman___ <cl...@genwax.com> writes:
> how about Everest? Niagara? Multiphase? Slipstream?
> DigiServer? Pointillion? Web Mammoth? SharpWeb?
> Web Enterprise? EnterWeb?
% perl -wlne 'print if "delmopr" eq join "", sort /./g' dictionary
premold
%
Actually I just wish we could standardize on one of either
"mod_perl" or "modperl", so I can stop bouncing emails off
to the wrong list :)
--
Joe Schaefer
New mod_perl name was [Re: New mod_perl Logo]
Posted by ___cliff rayman___ <cl...@genwax.com>.
how about Everest? Niagara? Multiphase? Slipstream?
DigiServer? Pointillion? Web Mammoth? SharpWeb?
Web Enterprise? EnterWeb?
Ged Haywood wrote:
> Hi there,
>
> On Tue, 29 Jan 2002, Chris Thompson wrote:
>
> > "mod_perl" is a lousy name.
> [snip]
> > mod_perl needs a name. Something marketable, something catchy.
>
> How about "BigFoot"?
>
> 73,
> Ged.
--
___cliff rayman___cliff@genwax.com___http://www.genwax.com/
Re: New mod_perl Logo
Posted by Ged Haywood <ge...@www2.jubileegroup.co.uk>.
Hi there,
On Tue, 29 Jan 2002, Chris Thompson wrote:
> "mod_perl" is a lousy name.
[snip]
> mod_perl needs a name. Something marketable, something catchy.
How about "BigFoot"?
73,
Ged.
Re: New mod_perl Logo
Posted by Perrin Harkins <pe...@elem.com>.
I respectfully request that people quit bitching about the name. The name
describes exactly what mod_perl is, and is widely known despite what some
HTML-jockeys at a conference might say. I can't believe it's any worse than
dumb-ass names like "Java Servlets" and "Enterprise Java Beans" (a name so
dumb people always use the acronym), or even PHP ("Professional Home
Pages?").
If you really want to do something to help market mod_perl, writing articles
and speaking at conferences has worked pretty well for me. I got a
discussion of mod_perl for large sites onto the front page of Slashdot by
doing this, and more people read Slashdot than just Perl geeks.
- Perrin
Re: New mod_perl Logo
Posted by Bill Moseley <mo...@hank.org>.
At 07:29 PM 01/29/02 -0500, Chris Thompson wrote:
>Well, I'd like to just throw one idea into the mix. It's something that's
>bugged me for a long time, no better time than the present.
>
>"mod_perl" is a lousy name.
I don't know about "lousy", but I do agree. I brought this up on the
docs-dev list:
http://search.apache.org/archives/docs-dev/0236.html
During the week I posted that I had run into PHP programmers at a computer
show, more PHP programmers at a pub (2 in the afternoon -- more out of work
programmers), and ended up talking with a couple of Java programmers one
day. The amazing thing was they all had a completely weird idea about what
mod_perl is or what it does. And all thought it was slow, old, dead, not
scalable, technology. And that was from programmers, not managers. We
all know there is a lot of misinformation out there.
Marketing is not everything, but it's a lot! What we know of "mod_perl" is
more than just perl+Apache, really. It's a development platform, or
development suite. It can be anything our marketing department says it is. ;)
In these tough economic times, repackaging might be helpful. Who knows?
And for some of us we know that mod_perl is also something that makes up a
chunk of our livelihood. So, the promotion of mod_perl is quite important,
unless we want to start spending more afternoons with those PHP programmers
down at the corner pub.
So how would a group like the mod_perl community promote itself in new
ways? Well, other professionals often have professional organizations or
associations to represent and promote their members. I wonder if there are
there enough mod_perl programmers to support something like that. Even if
there were, what could be done? Run a few print ads in magazines that
system admins read? Hire an ad firm for help in developing our "brand?"
mod_perl coffee mugs? ("Tired of that old cup of Java?") Free mod_perl
clinics? Hard to imagine any of that actually happening, really.
So what's a group of programmers to do?
The new web site should help, to some degree, but I'm not sure it will
change any manager's mind on the technology they pick to run their
applications.
Of course, most people here have access to big pipes. So, there's always
bulk mail ads. I got mail just today saying that it's an effective way to
advertise. In fact I got about ten of those today!
--
Bill Moseley
mailto:moseley@hank.org
Re: New mod_perl Logo
Posted by John Armstrong <si...@siberian.org>.
I could not agree more. I can not compute how much of my time would have
been saved over the last few years had I been able to say something
like 'We are using the Orwellian AppSphere 2002 running on Apache' to
VC's and management rather then 'Mod Perl'.....
Perception counts.... I am affectionate towards the mod_perl name after
so many years but its really not optimal.
John-
On Tuesday, January 29, 2002, at 04:29 PM, Chris Thompson wrote:
> On Tue, 29 Jan 2002, Jonathan M. Hollin wrote:
>> Ideas for logos, banners, "powered by"-type buttons are all welcome.
>
> Well, I'd like to just throw one idea into the mix. It's something
> that's
> bugged me for a long time, no better time than the present.
>
> "mod_perl" is a lousy name.
>
> There, I've said it.
>
> For any number of reasons, perl does now and will always face an uphill
> struggle in any "Enterprise" application.
Re: New mod_perl Logo
Posted by Chris Thompson <ct...@cthompson.com>.
On Tue, 29 Jan 2002, Jonathan M. Hollin wrote:
> Ideas for logos, banners, "powered by"-type buttons are all welcome.
Well, I'd like to just throw one idea into the mix. It's something that's
bugged me for a long time, no better time than the present.
"mod_perl" is a lousy name.
There, I've said it.
For any number of reasons, perl does now and will always face an uphill
struggle in any "Enterprise" application.
For example, at my place of employment, we just went through a rather
arduous task that I fought against and lost.
We had a production site that handled a decent amount of traffic. Apache,
mod_perl, Linux and MySQL. It ran and ran with almost no intervention.
The management team of the company that bought us a year ago had been trying
to force a change in the product by throwing up various arguments, which
were always false. ("MySQL doesnt support Transaction", "Yes it does". "But
they arent atomic.", "Yes they are". "Well, you can't roll them back.", "Yes
you can.")
In the end, I lost. From October to mid January they set about taking our
fully functional product and "replatforming" it to Win2k/IIS/ASP+VB/MSSQL.
The final reason? "Responsible enterprises do not use perl."
mod_perl needs a name. Something marketable, something catchy. The java
folks learned that a long time ago. Their systems are called
"Tomcat/Jakarta" and "Cocoon" and "Resin".
THAT, in my opinion, is what should happen for mod_perl 2.0. It should be
"Adirondack" or "Orwell" or any other generic, innocuous name. Even
"MonkeyButter v1.0" is probably a better deal than mod_perl.
As for logos, Avoiding Camels or Eagles is a requirement. I don't blame ORA
for requiring the trademark notices, the twisted concepts of US trademark
law REQUIRE them to do that. The first time they didnt, they could lose
their trademark. But we should have a "mascot" that makes sense and is OURS.
Linux has the penguin, OpenBSD has the blowfish, the other BSD's have the
devil. Those images are clearly associated with those products, and can be
used WITHOUT corporate approval.
--
_______________
Chris Thompson
New mod_perl Logo
Posted by "Jonathan M. Hollin" <ne...@digital-word.com>.
As many of you are aware, a new mod_perl website is currently being
developed and is near to going live. To accompany the new site, and
with mod_perl 2.0 development well under way, I have initiated a
competition to find a new mod_perl logo.
Therefore, I am asking any budding artists/designers amongst you to
submit your efforts to myself (mod_perl@digital-word.com). I will make
all submissions available for public review on the web at:
http://wypug.digital-word.com/mod_perl/ (where a few designs are even
now awaiting your perusal). Once I have accumulated enough variety, I
will run a voting booth so that the winning design can be democratically
chosen.
The reward for your efforts: Eternal fame and fortune, glamour,
admiration, kudos, etc... Well, at the very least, you will get the
pleasure of seeing your design every time you visit the mod_perl
website. It also possible that you would be credited by name/alias
within the site.
So come on folks, fire up the GIMP/PhotoShop and get to work. There are
no rules, no preconceived ideas and no restrictions. You can use the
exisiting icons of mod_perl (the camel, Apache feather, eagle, etc), or
go for something entirely new - it's up to you.
Ideas for logos, banners, "powered by"-type buttons are all welcome.
Please don't send submissions to the mailing list(s) - send them
directly to me (mod_perl@digital-word.com).
May the best man win. :-)
Jonathan M. Hollin - WYPUG Co-ordinator
West Yorkshire Perl User Group
http://wypug.pm.org/
NOTE: The WYPUG webserver is not very stable at the moment. If the
site is unavailable, then please try again a little later. We're
working on a solution right now...
Re: When to cache
Posted by Perrin Harkins <pe...@elem.com>.
> I'm interested to know what the opinions are of those on this list with
> regards to caching objects during database write operations. I've
encountered
> different views and I'm not really sure what the best approach is.
I described some of my views on this in the article on the eToys design,
which is archived at perl.com.
> Take a typical caching scenario: Data/objects are locally stored upon
loading
> from a database to improve performance for subsequent requests. But when
> those objects change, what's the best method for refreshing the cache?
There
> are two possible approaches (maybe more?):
>
> 1) The old cache entry is overwritten with the new.
> 2) The old cache entry is expired, thus forcing a database hit (and
> subsequent cache load) on the next request.
>
> The first approach would tend to yield better performance. However there's
no
> guarantee the data will ever be read. The cache could end up with a large
> amount of data that's never referenced. The second approach would probably
> allow for a smaller cache by ensuring that data is only cached on reads.
There are actually thousands of variations on caching. In this case you
seem to be asking about one specific aspect: what to cache. Another
important question is how to ensure cache consistency. The approach you
choose depends on frequency of updates, single server vs. cluster, etc.
There's a simple answer for what to cache: as much as you can, until you hit
some kind of limit or performance is good enough. Sooner or later you will
hit the point where the tradeoff in storage or in time spent ensuring cache
consistency will force you to limit your cache.
People usually use something like a dbm or Cache::Cache to implement
mod_perl caches, since then you get to share the cache between processes.
Storing the cache on disk means your storage is nearly unlimited, so we'll
ignore that aspect for now. There's a lot of academic research about
deciding what to cache in web proxy servers based on a limited amount of
space which you can look at if you have space limitations. Lots of stuff on
LRU, LFU, and other popular cache expiration algorithms.
The limit you are more likely to hit is that it will start to take too long
to populate the cache with everything. Here's an example from eToys:
We used to generate most of the site as static files by grinding through all
the products in the database and running the data through a templating
system. This is a form of caching, and it gave great performance. One day
we had to add a large number of products that more than doubled the size of
our database. The time to generate all of them became prohibitive in that
our content editors wanted updates to happen within a certain number of
hours but it was taking longer than that number of hours to generate all the
static files.
To fix this, we moved to not generating anything until it was requested. We
would fetch the data the first time it was asked for, and then cache it for
future requests. (I think this corresponds to your option 2.) Of course
then you have to decide on a cache consistency approach for keeping that
data fresh. We used a simple TTL approach because it was fast and easy to
implement ("good enough").
This is just scratching the surface of caching. If you want to learn more,
I would suggest some introductory reading. You can find lots of general
ideas about caching by searching Google for things like "cache consistency."
There are also a couple of good articles on the subject that I've read
recently. Randal has an article that shows an implementation of what I
usually call "lazy reloading":
http://www.linux-mag.com/2001-01/perl_01.html
There's one about cache consistency on O'Reilly's onjava.com, but all the
examples are in Java:
http://www.onjava.com/pub/a/onjava/2002/01/09/dataexp1.html
Also, in reference to Rob Nagler's post, it's obviously better to be in a
position where you don't need to cache to improve performance. Caching adds
a lot of complexity and causes problems that are hard to explain to
non-technical people. However, for many of us caching is a necessity for
decent performance.
- Perrin
Re: When to cache
Posted by Rob Nagler <na...@bivio.biz>.
> 1) The old cache entry is overwritten with the new.
> 2) The old cache entry is expired, thus forcing a database hit (and
> subsequent cache load) on the next request.
3) Cache only stuff which doesn't expire (except on server restarts).
We don't cache any mutable data, and there are no sessions. We let the
database do the caching. We use Oracle, which has a pretty good
cache. We do cache some stuff that doesn't change, e.g. default
permissions, and we release weekly, which involves a server restart
and a refresh of the cache.
If you hit http://www.bivio.com , you'll get a page back in under
300ms. There are probably 10 database queries involved if you are
logged in. This page is complex, but far from our most complex.
For example, this page
http://www.bivio.com/demo_club/accounting/investments
sums up all the holdings of a portfolio from the individual
transactions (buys, sells, splits, etc.). It also comes back in under
300ms.
Sorry if this wasn't the answer you were looking for. :)
Rob