You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Jonathan Vanasco <mo...@2xlp.com> on 2006/08/11 23:42:56 UTC
Memory Usage Reporting?
I'm trying to push a project to production, and am trying to optimize
the server.
I'm using Apache2::Status. So far, it does what it claims.
But I somehow have 50mb apache processes, and am trying to track down
the bloat. I expected to have a process 1/10 that size. The code
base for my own libraries is about 900k- that includes all my page
generation stuff and Petal templates which I precompile.
I'm wondering a few things:
1) has anyone written something that recursively goes through all the
modules and aggregates the memory sizes for output?
I'm hoping for something that can give me a view of all the memory
used and where, without having to click myself
2) I see process grow without stopping, even with reload. i assume
this is from the caveat in "PerlSetVar StatusTerseSizeMainSummary On" ?
3) memory usage on the following 'pages' doesn't work:
/ - perl-status/?status_memory_usage
/main (symdump)- perl-status/main?noh_b_package_size
i'm sure that some of this is from unnecessary includes.
i did some tests using prefork with 1 child max, so i could see the
code caching optimization
w/nothing loaded
$ ps aux | grep httpd
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME
COMMAND
root 10386 0.0 0.4 51648 5788 ?? Ss 4:38PM
0:00.14 /usr/local/apache2/bin/httpd -k start
nobody 10387 0.0 0.0 50700 720 ?? S 4:38PM
0:00.00 /usr/local/apache2/bin/httpd -k start
after the first request (just of the perl-status page) 720 ram jumps
to 2616
then i load my libraries, dependencies
$ ps aux | grep httpd
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME
COMMAND
root 10347 0.0 2.0 85040 31660 ?? Ss 4:36PM
0:02.40 /usr/local/apache2/bin/httpd -k start
nobody 10350 0.0 0.2 78384 2716 ?? S 4:36PM
0:00.02 /usr/local/apache2/bin/httpd -k start
after the first request (just of the perl-status page) 2716mb ram
jumps to 10820 , making a request to any of my pages, the children
jump to RSS 17000
my dev box is osx- and I don't know if I'll be able to get gTop
installed to work with modperl. (i installed it with darwin ports ,
which installed a ton of crap. still trying to figure it out )
also i noticed that the parent memory, as time progresses , does this
VSZ RSS
85040 31520
85040 28132
shoudn't the RSS stay the same?
based on what i've written so far, can anyone give me some pointers?
something just doesn't seem right.
i'll grudgingly accept 30mb for a parent process -- but i'm making
heavy use of precompiled code. or at least think i am-- things seem
to have been loaded via startup.pl.
on that note though- does anyone have a recipe for traversing a given
directory and including all .pm files within it? i could alter my
TAL caching script to do it, but that would likely save me 10 minutes
on an already frustrating day.
thanks again in advance
// Jonathan Vanasco
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - -
| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - -
| RoadSound.com - Tools For Bands, Stuff For Fans
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - -
Re: Memory Usage Reporting?
Posted by Jonathan Vanasco <mo...@2xlp.com>.
I did some tests, some interesting results
my processes no longer seem to be 300mb. which is good. however,
when that happened, i was seeing a 300mb process and then 6mb in a
child. now i'm back to seemingly unshared memory.
based on my tests below, i *think* that it may have been because i
was doing apachectl restarts, and not clean stop/starts
i've noticed that my parent and works take on an additional 30mb of
memory every time an apachectl restart is issued.
i thought to myself, ugh - "thats what you get for not using a graceful"
well , graceful does the same thing.
i'm more lost now than i was yesterday
---------------
StartServers 1
MinSpareServers 1
MaxSpareServers 1
MaxClients 2
MaxRequestsPerChild 10000
===========
Trial 1
start
@end of startup.pl
GTop (2207) 23457792
ps aux
root 2210 0.0 5.5 59488 55284
www 2214 0.0 5.5 59548 55340
Note: ps-aux should be larger.
a) in almost every other trial it was showing something closer to
Gtop
b) it is post-init
@four requests
GTop
preview_check Size Before: (2241) 48320512
preview_check Size After: (2241) 48398336
handler Size Pre- Page: (2241) 48410624
handler Size Post- Page: (2241) 48484352
preview_check Size Before: (2214) 48582656
preview_check Size After: (2214) 48582656
handler Size Pre- Page: (2214) 48582656
handler Size Post- Page: (2214) 48582656
Note:
preview_check is run on the init, checks cookies and sets up the
ctx object/page user.
redirects to login screen if request invalid, redirects to
preview site if not in alpha test
handler is the content generation phase
on both the first value is from first line of the handler, the
second value is the last line
ps aux
root 2210 0.0 5.5 59488 55284 ?? Ss 3:21PM 0:02.53 /
usr/local/sbin/httpd
www 2214 0.0 5.5 59900 55608 ?? S 3:21PM 0:00.10 /
usr/local/sbin/httpd
www 2241 0.0 5.5 59860 55584 ?? S 3:26PM 0:00.08 /
usr/local/sbin/httpd
Trial 2
apachectl restart
@end of startup.pl - console
GTOP (2256) 23457792
@end of startup.pl - logs
GTOP (2210) 70836224
ps aux
root 2210 0.0 7.7 81776 77484
www 2261 0.0 7.7 81832 77556
@ four requests
GTOP
preview_check Size Before: (2261) 72237056
preview_check Size After: (2261) 72237056
handler Size Pre- Page: (2261) 72237056
handler Size Post- Page: (2261) 72237056
ps aux
root 2210 0.0 7.7 81776 77484
www 2261 0.0 7.8 82956 78640
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
StartServers 2
MinSpareServers 5
MaxSpareServers 10
MaxClients 10
MaxRequestsPerChild 10000
===========
start
@end of startup.pl
GTop (2307) 23457792
ps aux
root 2310 2.4 5.5 59488 55284
www 2314 0.0 5.5 59548 55340
www 2315 0.0 5.5 59548 55340
www 2320 0.0 5.5 59548 55340
www 2323 0.0 5.5 59548 55340
www 2324 0.0 5.5 59548 55340
@ four requests
preview_check Size Before: (2320) 48734208
preview_check Size After: (2320) 48734208
handler Size Pre- Page: (2320) 48734208
handler Size Post- Page: (2320) 48734208
restart
@ startup.pl -
console
GTop
(2343) 23457792
log
GTop
(2310) 70836224
ps aux
root 2310 0.0 7.7 81776 77484
www 2348 0.0 7.7 82184 77872
www 2349 0.0 7.7 82184 77872
www 2354 0.0 7.7 82184 77872
www 2357 0.0 7.7 82184 77872
www 2358 0.0 7.7 82184 77872
@ four requests
preview_check Size Before: (2349) 71454720
preview_check Size After: (2349) 71454720
handler Size Pre- Page: (2349) 71454720
handler Size Post- Page: (2349) 71454720
root 2310 0.0 7.7 81776 77484
www 2348 0.0 7.7 82184 77872
www 2349 0.0 7.7 82184 77872
www 2354 0.0 7.7 82184 77872
www 2357 0.0 7.7 82184 77872
www 2358 0.0 7.7 82184 77872