You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mod_python-dev@quetz.apache.org by dharana <dh...@dharana.net> on 2005/07/13 01:54:45 UTC
Re: [mod_python] Help with memory problems
Uh oh, no it doesn't goes away. Check this (custom handler):
------ controller.py ---------------
import os
from time import time
from mod_python import apache
from mod_python import Session
def handler(req):
start_time = time()
req.sess = Session.FileSession(req)
req.content_type = 'text/html'
end_time = time()
totaltime = (end_time - start_time)*1000
req.write('%.0f ms' % ((totaltime)))
return apache.OK
------------------------------------
This takes 75ms on a 1Ghz with 1Gb RAM server. After some page refreshes the
memory load is over 70Mb. Is this right? (I'm asking for both the load time and
memory usage)
dharana wrote:
> Please, ignore my previous email. The massive leak was being caused by
> FileSession. I was using an earlier snapshot and after upgrading to:
>
> http://svn.apache.org/snapshots/mod_python/mod_python_20050712224304.tar.gz
>
> Everything went back normal. After checking everything works go upgrade
> production. Load problem solved. I feel much better now, python handles
> the heavy complexity I imposed to him with no problems.
>
>
> How much is left for 3.2? I can't use an stable mp release right now
> because I need several things included in 3.2 (segfaults for psp,
> filesession, etc) and using svn HEAD in a prod server is scary.
>
> dharana wrote:
>
>> look at this screenshot:
>>
>> http://dharana.net/memory_concerns.png
>>
>> It shows two screenshots of the same page of my webapp. The lower one
>> was shoot just after an apachectl start. The second one was shot after
>> 6 consecutive refreshes where memory usage increased every time by a
>> lot of Mbs.
>>
>> I use a lot of custom modules, from db calls to xslt transformations
>> (with libxml). But in the page shown in the screenshot it does little.
>> The strange thing is that after a few more page refreshes the memory
>> load goes back to 23Mb and then increases again.
>>
>> I use some light module-level caching (I cache the 3 queries executed
>> in the lower page). But I don't think this is causing the heavy memory
>> usage because after it goes back to 23Mb it does query again (ie: the
>> caching mechanism still has it's cache).
>>
>> I use apache prefork and Maxrequestperchild 0. After every page
>> refresh the unr obj var shown on the page doesn't changes. It's either
>> 63 or 0.
>>
>> mem, resident and stack come from this script:
>>
>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222
>>
>> unr obj is just the output of gc.collect() after everything else is
>> done (the debug print is printed just before return apache.OK.
>>
>> What I am asking is for some help or pointers in how to track down who
>> is causing this or where to look for reducing the memory usage.
>>
>>
>> Thanks,
>>
>
--
dharana
Re: [mod_python] Help with memory problems
Posted by dharana <dh...@dharana.net>.
I use apache pre-fork model. Definition of some refreshes was like 20 or 30
total page refreshes.
I switched to DbmSession and it was faster. Then I rebooted the server and now
FileSession works flawlessly again. I don't know what happened but it's gone.
Whenever I try to report some bug it disappears _after_ sending the email.Sorry
for bothering you.
Jim Gallacher wrote:
> dharana wrote:
>
>>
>> dharana wrote:
>>
>>> Uh oh, no it doesn't goes away. Check this (custom handler):
>>>
>>> ------ controller.py ---------------
>>> import os
>>> from time import time
>>> from mod_python import apache
>>> from mod_python import Session
>>>
>>> def handler(req):
>>> start_time = time()
>>> req.sess = Session.FileSession(req)
>>> req.content_type = 'text/html'
>>> end_time = time()
>>> totaltime = (end_time - start_time)*1000
>>> req.write('%.0f ms' % ((totaltime)))
>>> return apache.OK
>>> ------------------------------------
>>>
>>>
>>> This takes 75ms on a 1Ghz with 1Gb RAM server. After some page
>>> refreshes the memory load is over 70Mb. Is this right? (I'm asking
>>> for both the load time and memory usage)
>>>
>
> Using the above code I get 1ms response time on an Athlon 2500+ 512MB
> RAM, apache mpm-worker model. What is the defintion of some refreshes:
> 10, 1000, 1000000?
>
> Jim
>
>
>
>
--
dharana
Re: [mod_python] Help with memory problems
Posted by Jim Gallacher <jg...@sympatico.ca>.
dharana wrote:
>
> dharana wrote:
>
>> Uh oh, no it doesn't goes away. Check this (custom handler):
>>
>> ------ controller.py ---------------
>> import os
>> from time import time
>> from mod_python import apache
>> from mod_python import Session
>>
>> def handler(req):
>> start_time = time()
>> req.sess = Session.FileSession(req)
>> req.content_type = 'text/html'
>> end_time = time()
>> totaltime = (end_time - start_time)*1000
>> req.write('%.0f ms' % ((totaltime)))
>> return apache.OK
>> ------------------------------------
>>
>>
>> This takes 75ms on a 1Ghz with 1Gb RAM server. After some page
>> refreshes the memory load is over 70Mb. Is this right? (I'm asking for
>> both the load time and memory usage)
>>
Using the above code I get 1ms response time on an Athlon 2500+ 512MB
RAM, apache mpm-worker model. What is the defintion of some refreshes:
10, 1000, 1000000?
Jim
Re: [mod_python] Help with memory problems
Posted by dharana <dh...@dharana.net>.
dharana wrote:
> Uh oh, no it doesn't goes away. Check this (custom handler):
>
> ------ controller.py ---------------
> import os
> from time import time
> from mod_python import apache
> from mod_python import Session
>
> def handler(req):
> start_time = time()
> req.sess = Session.FileSession(req)
> req.content_type = 'text/html'
> end_time = time()
> totaltime = (end_time - start_time)*1000
> req.write('%.0f ms' % ((totaltime)))
> return apache.OK
> ------------------------------------
>
>
> This takes 75ms on a 1Ghz with 1Gb RAM server. After some page refreshes
> the memory load is over 70Mb. Is this right? (I'm asking for both the
> load time and memory usage)
>
(Memory load == memory usage of httpd/python child)
>
>
--
dharana