Memory Usage

[Georg Grabler <gg...@gmail.com>]

Hello list,

I'm developing a quite large project, using SOAP::Lite and some more
modules using mod_perl.

My main problem is: Memory Usage.

Currently, the apache processes take (each) about 150 MB memory,
what's quite a lot. Now, i've been wondering which module and why it
takes that much memory at all.

Usually, the code should not be leaking. What's done is basically
simple, connecting to Oracle DBs, fetching data and returning it using
SOAP::Lite Data objects.
My understanding (perl memory handling, garbage collection) is, that
objects are destroyed as soon as functions are finished. How about
this in mod_perl? It seems as if a lot of unnecessary things are
cached. I can't just measure how much memory is used and restart the
processes using several modules, since that would definitely result in
a runtime performance problem.

So, i did read over the documents measuring apache memory usage and
similar, but couldn't realy figure out how to measure which functions
keep all this memory. Are the SOAP::Data objects destroyed? I don't
really know, though, there never are 150MB data transferred (max about
10 MB), so i don't really know where all this MBs come from.

Initially, the apache processes take 42 MB each (i think this should
be mostly shared memory and a lot of preloaded projects, as usual on a
developer machine). If i connect using SOAP, the process jumps up to
140-150MB, and i just can't think of why this occurs.

Do you have any advice, how to measure functions and / data which is
kept? Or generally what takes that much memory? I definitely don't
need to keep this data, since it would always change (per-request).

Thank you,
Georg

Re: Memory Usage

[Jay Buffington <ja...@gmail.com>]

> > My main problem is: Memory Usage.

> > Currently, the apache processes take (each) about 150 MB memory,
> > what's quite a lot. Now, i've been wondering which module and why it
> > takes that much memory at all.

Check out this thread:
http://aspn.activestate.com/ASPN/Mail/Message/modperl/3173671

> > My understanding (perl memory handling, garbage collection) is, that
> > objects are destroyed as soon as functions are finished. How about
> > this in mod_perl? It seems as if a lot of unnecessary things are
> > cached. I can't just measure how much memory is used and restart the
> > processes using several modules, since that would definitely result in
> > a runtime performance problem.

perl garbage collects when the variable goes out of scope, but it
doesn't return the memory to the system.  See
http://perldoc.perl.org/perlfaq3.html#How-can-I-free-an-array-or-hash-so-my-program-shrinks%3f


> in simple terms: mod_perl destroys the objects, but it holds onto the
> memory for the lifetime of the child.  if a single var ups your
> child's memory by 5mb, that 5mb will stay reserved for the child
> until it dies.

This is correct.  I've argued that this is a good thing:
http://foobarbazbax.livejournal.com/4953.html

> > Do you have any advice, how to measure functions and / data which is
> > kept? Or generally what takes that much memory? I definitely don't
> > need to keep this data, since it would always change (per-request).

See Apache::LogMemoryUsage, which isn't on CPAN, but I pasted in that
same thread I referenced above:
http://aspn.activestate.com/ASPN/Mail/Message/modperl/3175357

Best of Luck,
Jay

Re: Memory Usage

[Jonathan Vanasco <jv...@2xlp.com>]

On Mar 30, 2007, at 8:00 AM, Georg Grabler wrote:

> Hello list,
>
> I'm developing a quite large project, using SOAP::Lite and some more
> modules using mod_perl.
>
> My main problem is: Memory Usage.

If you search the archives using my name, you'll fine a ton of stuff  
about me complaining and going crazy on this from may->september of  
2006.  might be a good starting point :)


> Currently, the apache processes take (each) about 150 MB memory,
> what's quite a lot. Now, i've been wondering which module and why it
> takes that much memory at all.

a_ use apache status, with all the bells and whistles.  it'll show  
you how much memory each module takes up.
b_ be wary of what you use modules for.  I was using File::Find to  
precache templates and do some cross platform file opening -- it took  
up ~6mb of  ram.  i replaced the functionality i needed with 20lines  
of perl code that took up <10k instead.

> Usually, the code should not be leaking. What's done is basically
> simple, connecting to Oracle DBs, fetching data and returning it using
> SOAP::Lite Data objects.

c_ there's what appears to be a descriptor leak in DBI, but its  
really not.  DBI has internal caches that are set at ~120 SVs for  
connections and queries - it can look like you're leaking some ram on  
each request, then you hit 121 and magically all your ram is back :)

> My understanding (perl memory handling, garbage collection) is, that
> objects are destroyed as soon as functions are finished. How about
> this in mod_perl? It seems as if a lot of unnecessary things are
> cached. I can't just measure how much memory is used and restart the
> processes using several modules, since that would definitely result in
> a runtime performance problem.

d_ my suggestion is this:
	1_ use a prefork mpm to take advantage of copy on write and memory  
sharing
	2_ preload ever module in your app with a startup.pl script

in simple terms: mod_perl destroys the objects, but it holds onto the  
memory for the lifetime of the child.  if a single var ups your  
child's memory by 5mb, that 5mb will stay reserved for the child  
until it dies.  i can't remember if its reserved for that variable in  
that child, or just that child, but thats the general idea.

> So, i did read over the documents measuring apache memory usage and
> similar, but couldn't realy figure out how to measure which functions
> keep all this memory. Are the SOAP::Data objects destroyed? I don't
> really know, though, there never are 150MB data transferred (max about
> 10 MB), so i don't really know where all this MBs come from.

~5MB is the apache  process + mp vars.  everything else is perl  
code.  perl code takes up a lot of ram when you compile -- the petal  
framework ,  for example, is ~400k of text, but takes up about ~6mb  
of ram when compiled.  the rose db framework is similar.

> Initially, the apache processes take 42 MB each (i think this should
> be mostly shared memory and a lot of preloaded projects, as usual on a
> developer machine). If i connect using SOAP, the process jumps up to
> 140-150MB, and i just can't think of why this occurs.
>
> Do you have any advice, how to measure functions and / data which is
> kept? Or generally what takes that much memory? I definitely don't
> need to keep this data, since it would always change (per-request).

there's a VERY GOOD chance that  your platform isn't reporting memory  
right.  I've yet to personally use a system where real / shared  
memory was reported correctly.  I think the only way i was able to  
figure out how the memory was actually working was by figuring out  
what it took for me to get apache to start using swap space.   i just  
tried using SizeLimit to figure it out, and couldn't quite understand  
the results.

so i'd look into why the processes take 42mb each.  it could very  
well be a memory reporting issue, and they're not really 42mb

Re: Memory Usage

[gautam chekuri <ga...@gmail.com>]

While I dont know if this would help, but faced with a similar
situation, I might try something on the following lines:
1. write a small script that uses SOAP:Lite (may be copy hibye.cgi
from http://guide.soaplite.com/)
2. setup htttpd.conf to use Apache::Status
3. Run apache in single-process mode (httpd -X)
4. run the test script i wrote
5. see what apache status has to say.

if memory usage looked ok we can add some more code and retry, and we
might be able to figure out where the problem is.

http://perl.apache.org/docs/1.0/guide/performance.html#Measuring_the_Memory_Usage_of_Subroutines


On 3/30/07, Georg Grabler <gg...@gmail.com> wrote:
> Hello list,
>
> I'm developing a quite large project, using SOAP::Lite and some more
> modules using mod_perl.
>
> My main problem is: Memory Usage.
>
> Currently, the apache processes take (each) about 150 MB memory,
> what's quite a lot. Now, i've been wondering which module and why it
> takes that much memory at all.
>
> Usually, the code should not be leaking. What's done is basically
> simple, connecting to Oracle DBs, fetching data and returning it using
> SOAP::Lite Data objects.
> My understanding (perl memory handling, garbage collection) is, that
> objects are destroyed as soon as functions are finished. How about
> this in mod_perl? It seems as if a lot of unnecessary things are
> cached. I can't just measure how much memory is used and restart the
> processes using several modules, since that would definitely result in
> a runtime performance problem.
>
> So, i did read over the documents measuring apache memory usage and
> similar, but couldn't realy figure out how to measure which functions
> keep all this memory. Are the SOAP::Data objects destroyed? I don't
> really know, though, there never are 150MB data transferred (max about
> 10 MB), so i don't really know where all this MBs come from.
>
> Initially, the apache processes take 42 MB each (i think this should
> be mostly shared memory and a lot of preloaded projects, as usual on a
> developer machine). If i connect using SOAP, the process jumps up to
> 140-150MB, and i just can't think of why this occurs.
>
> Do you have any advice, how to measure functions and / data which is
> kept? Or generally what takes that much memory? I definitely don't
> need to keep this data, since it would always change (per-request).
>
> Thank you,
> Georg
>

Re: Memory Usage

[Evaldas Imbrasas <ev...@imbrasas.com>]

> Currently, the apache processes take (each) about 150 MB memory,
> what's quite a lot. Now, i've been wondering which module and why it
> takes that much memory at all.

Georg,

How do you determine how much memory each Apache process is taking? If
you're on Linux, it's tricky to correctly determine the shared memory
size.

> Initially, the apache processes take 42 MB each (i think this should
> be mostly shared memory and a lot of preloaded projects, as usual on a
> developer machine). If i connect using SOAP, the process jumps up to
> 140-150MB, and i just can't think of why this occurs.

That sounds to me like you're using Apache::DBI, which connects to the
database on the first request. It loads the database driver (i.e.,
DBD::Oracle) into the shared memory at that time. Here's how I check
how much memory is actually being used by Apache (assuming you're
using 1.3.x or 2.x prefork):

*) Stop Apache
*) Change the value of StartServers parameter in http.conf to be equal
to MaxClients (say it's 'N')
*) Do `free -m` and note how much free memory you have (minus buffers/cache)
*) Start Apache, but make no requests yet
*) Do `free -m` again, and look how the free memory usage has changed.
This should decrease the free memory by the amount used by Apache
parent process (shared memory) and N Apache children
*) Use your favorite load testing software to send at least N
concurrent requests to your server, and monitor how your free memory
is decreasing. If you see that after the first request the amount of
free memory is decreased by N * 100MB, then you have a problem.
Otherwise, those additional 100MB are shared and you have nothing to
worry about.

-- 
-----------------------------------------------------
Evaldas Imbrasas
http://www.imbrasas.com

Re: Memory Usage

[Kjetil Kjernsmo <kj...@opera.com>]

On Friday 30 March 2007 14:00, Georg Grabler wrote:
> Initially, the apache processes take 42 MB each (i think this should
> be mostly shared memory and a lot of preloaded projects, as usual on
> a developer machine). If i connect using SOAP, the process jumps up
> to 140-150MB, and i just can't think of why this occurs.

I can't help too much with the general problem, but how much is shared 
after time goes by? If a lot is shared, I wouldn't worry too much, but 
if there is little shared, it would become expensive to scale. 

Kjetil
-- 
Kjetil Kjernsmo
Information Systems Developer
Opera Software ASA