You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Perrin Harkins <pe...@elem.com> on 2006/09/06 20:02:56 UTC
Re: mod_perl reqs/s @ concurrency-- for actual db based
applications
On Wed, 2006-09-06 at 13:46 -0400, Jonathan Vanasco wrote:
> with 2 children running, I'm handling ~70 r/s @ concurrency 10-1000
> 4-8 children seems to be my point for diminishing marginal utility-
> in that range, I'm handling ~100 r/s @ concurrency 10-1000 ; and the
> numbers don't really change no matter how many servers I toss at it.
That probably means you are limited by the database, like everyone else.
> granted, i'm also benching with ab, which is old and not very
> reliable.
I like httperf for benchmarks.
> i'm fairly confident that a LARGE limiting factor is db interaction
> and (b)locking. And i know that the bulk of the request timing is
> taken up in the DB or the DB/App ineraction layer. I wouldn't be
> surprised if I those numbers double , triple , and more with a nicely
> clustered db structure.
You can usually increase your performance greatly just by tuning your
existing SQL and database. Run Apache::DProf or the DBI profiler, find
out where the time is being spent, and work on it. There are many
resources for database performance tuning. Work on the actual queries
and schema structure, not on the database configuration. You always get
more from the former than the latter.
> i'm just wondering what a target range for DB/Templated apps with
> moderate page logic should be ( "hello world" < Moderate Logic <
> "comprehensive statistical analysis" )
There ain't no such thing. Once you fix the obvious architecture things
(by using mod_perl and running a reverse proxy), the performance of your
application depends entirely on what it's doing. You can almost always
improve it by tweaking the code and database more based on profiling.
- Perrin
Re: mod_perl reqs/s @ concurrency-- for actual db based applications
Posted by Patrick Mulvany <pa...@firedrake.org>.
On Wed, Sep 06, 2006 at 04:33:27PM -0500, Frank Wiles wrote:
> On Wed, 06 Sep 2006 14:02:56 -0400
> Perrin Harkins <pe...@elem.com> wrote:
>
> > You can usually increase your performance greatly just by tuning your
> > existing SQL and database. Run Apache::DProf or the DBI profiler,
> > find out where the time is being spent, and work on it. There are
> > many resources for database performance tuning. Work on the actual
> > queries and schema structure, not on the database configuration. You
> > always get more from the former than the latter.
>
> Agree with everything else you said, but have to disagree with the
> last statement. Specifically with PostgreSQL the default server
> configuration is a really low end config, setup to basically run
> on any old hardware you have around.
>
> Configuring it for your particular hardware ( memory size
> specifically ) can reap HUGE performance gains. In case anyone
> is wondering the two most useful tweaks are shared_buffers
> and effective_cache_size which need to be increased on all but
> the lowliest of systems.
>
> PostgreSQL may be the only one where this is true however...
>
Most databases install with setting to run on the minimum hardware spec
required. A good DBA tunes the database so that it can internally cache
as much as possible while preventing the process from ever having to swap
out to disc. There are basically three areas of memory usage that need to
be balanced.
Sort area - how much memory the DB uses to process sort/joins etcs.
Data cache - how much memory the DB uses to prefetch/cache data
Query/Compile area - how much area the DB uses to compile and execute
queries/stored proceedures.
If the Sort area is too small the DB has to do the sort on the disc (v.slow)
If the Data cache is too small then too much disc IO is required (v.slow)
If the Query/Compile area is too small common queries have to be reanalyised
and compiled each time.
Some DBs dynamically mange these settings them selves other require them to
be tweeked but all require that you specify how much memory the DB is allowed
to use. This depends on the DB, your server and whether it is dedicated or not.
The next useful setting relates to concurrency. A 64 processor machine will do
a lot more in parallel than a single CPU box but most default settings are
based on the assumption you have a single CPU.
Some databases have significant connect setup overheads. Apache::DBI's ability
to have persistant connections save you from these each time.
Hope it helps
Paddy
Re: mod_perl reqs/s @ concurrency-- for actual db based
applications
Posted by Frank Wiles <fr...@wiles.org>.
On Wed, 06 Sep 2006 14:02:56 -0400
Perrin Harkins <pe...@elem.com> wrote:
> You can usually increase your performance greatly just by tuning your
> existing SQL and database. Run Apache::DProf or the DBI profiler,
> find out where the time is being spent, and work on it. There are
> many resources for database performance tuning. Work on the actual
> queries and schema structure, not on the database configuration. You
> always get more from the former than the latter.
Agree with everything else you said, but have to disagree with the
last statement. Specifically with PostgreSQL the default server
configuration is a really low end config, setup to basically run
on any old hardware you have around.
Configuring it for your particular hardware ( memory size
specifically ) can reap HUGE performance gains. In case anyone
is wondering the two most useful tweaks are shared_buffers
and effective_cache_size which need to be increased on all but
the lowliest of systems.
PostgreSQL may be the only one where this is true however...
---------------------------------
Frank Wiles <fr...@wiles.org>
http://www.wiles.org
---------------------------------
Re: mod_perl reqs/s @ concurrency-- for actual db based applications
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
> You can usually increase your performance greatly just by tuning your
> existing SQL and database. Run Apache::DProf or the DBI profiler, find
> out where the time is being spent, and work on it. There are many
> resources for database performance tuning. Work on the actual queries
> and schema structure, not on the database configuration. You always get
> more from the former than the latter.
Particularly, the algorithms in the code, and you can sometimes(lots of times)
CACHE the results of SQL queries overtop of the "query cache/result cache" on the DB itself
aka Serialized to disk, or per process / request.
--
------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F
"In all that I've done wrong I know I must have done something right to
deserve a hug every morning and butterfly kisses at night."
__ ___ ___ ____ __
/ |/ /_ __/ __/ __ \/ /
/ /|_/ / // /\ \/ /_/ / /__
/_/ /_/\_, /___/\___\_\___/
<___/
Re: mod_perl reqs/s @ concurrency-- for actual db based applications
Posted by Jonathan Vanasco <mo...@2xlp.com>.
On Sep 6, 2006, at 2:02 PM, Perrin Harkins wrote:
> That probably means you are limited by the database, like everyone
> else.
right. i'm just wondering what the avg numbers for moderate logic
apps are.
> I like httperf for benchmarks.
its ok. it and ab haven't been updated in a while though.
have you seen tsung? its pretty nice, but more importantly- actively
maintained
http://tsung.erlang-projects.org/
one of my friends has been doing a lot of erlang lately, and set me
up on a dev box to profile once. ( elang is a pain to get running,
its got a steep learning curve still )
if you've got time to try setting it up, i recommend it.
there's also a python app that does benchmarks more as unit tests.
can't remember the name though.
> You can usually increase your performance greatly just by tuning your
> existing SQL and database. Run Apache::DProf or the DBI profiler,
> find
> out where the time is being spent, and work on it. There are many
> resources for database performance tuning. Work on the actual queries
> and schema structure, not on the database configuration. You
> always get
> more from the former than the latter.
already done. i 'explain' all statements as I write them, and tweak
SQL and indexes ( gotta love postgres's detailed explain )
> There ain't no such thing.
damnit.
> Once you fix the obvious architecture things
> (by using mod_perl and running a reverse proxy), the performance of
> your
also done.