You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mod_python-dev@quetz.apache.org by "Laurent Blanquet (JIRA)" <ji...@apache.org> on 2006/06/10 20:50:30 UTC
[jira] Created: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
--------------------------------------------------------------------------
Key: MODPYTHON-172
URL: http://issues.apache.org/jira/browse/MODPYTHON-172
Project: mod_python
Type: Bug
Components: core
Versions: 3.2.8
Environment: Win32 XP SP1 / SP2
Apache 2.0.55 installed from binary (.MSI)
Python 2.4.2 or 2.4.3 installed from binary from www.python.org
Reporter: Laurent Blanquet
I encounter memory leaks [~ 16 K per request) using the configuration described below.
=============================
Python configuration from Httpd.conf:
=============================
Alias /python/ "d:/python24/B2B/"
<Directory "d:/python24/B2B">
AddHandler mod_python .py
PythonHandler pyHandlerHTTP
PythonDebug On
</Directory>
=============================
Test handler - pyHandlerHTTP.py :
=============================
import mod_python
from mod_python import util
def handler(req):
#Removing this line solves the problem.
F=util.FieldStorage( req )
return mod_python.apache.OK
=============================
HTTP Request (dump using TCPWATCH):
=============================
POST http://localhost:80/python/Alertes.py HTTP/1.0
Content-Type: multipart/form-data; boundary=--------061006144341906
Content-Length: 209
Proxy-Connection: keep-alive
Host: www.tx2-localhost
Accept: text/html, */*
User-Agent: Mozilla/3.0 (compatible; Indy Library)
Proxy-Authorization: Basic Og==
----------061006144341906
Content-Disposition: form-data; name="TYPE"
LAST_ALERTS
----------061006144341906
Content-Disposition: form-data; name="FILEAGE"
180
----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Harold Ship <ha...@giant-steps-networks.com>.
Hi Nicolas
Howcome I didn't see this post in JIRA? I added my comment there. I also didn't
get email. I just found it in Google.
Anyway, I have an application that was started over a year ago with 3.1.3. We
switched to 3.2.8 because we had a requirement to upload files and there was a
problem that was fixed in that version.
I observed a memory leak in our application under 3.2.8, and found MODPYTHON-172
on Google. I ran the test program in my first comment on 3.2.8, then tried going
back to 3.1.3 to see if the leak was there too.
This leak seems to be in both 3.1.3 and 3.2.8.
I have added another comment regarding _apachemodule.c, which I include below.
Thanks for your help,
Harold
### ============ Begin Code ==============
Excuse me if this is a dumb question, but I don't know anything about the
Python-C interface.
Is it possible that the code in line 319 of _apachemodule.c (from 3.2.8) is the
problem?
The code is:
if (key && val)
PyList_Append(pairs, Py_BuildValue("(O,O)", key, val));
Does the object returned by Py_BuildValue() need to be dereferenced?
Something like:
if (key && val) {
PyObject * list_elem = Py_BuildValue("(O,O)", key, val);
if (list_elem)
PyList_Append(pairs, list_elem);
Py_XDECREF(list_elem);
}
========================================
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Nicolas Lehuen <ni...@lehuen.com>.
Hi,
The subject of this thread is about mod_python 3.2.8, yet you report using
mod_python 3.1.3. Which version have you detected the memory leak in ? There
are a bunch of leaks that have been fixed since 3.1.3...
Regards,
Nicolas
2006/6/30, Harold Ship (JIRA) <ji...@apache.org>:
>
> [
> http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12418577]
>
> Harold Ship commented on MODPYTHON-172:
> ---------------------------------------
>
> I've been able to reproduce the problem with "mod_python.publisher".
>
> Using:
> Windows 2000 Professional
> apache 2.0.54
> python 2.3.5
> mod_python 3.1.3
>
> the following script:
> #foo.py
> def bar(req):
> a=req.form.getfirst('a')
> req.write('a=%s'%str(a))
> req.write('Ok')
>
> repeatedly sending requests to <addr>/foo/bar?a=b causes memory to rise
> continuously. removing the querystring does not. I used task manager to
> measure memory.
>
> I've also found that the following change to util.py corrects this
> behaviour:
>
> old util.py:
> parse_qs = _apache.parse_qs
> parse_qsl = _apache.parse_qsl
>
>
> workaround util.py:
> parse_qs = _apache.parse_qs
> #parse_qsl = _apache.parse_qsl
> from cgi import parse_qsl
>
>
> > Memory leak with util.fieldstorage using mod_python 3.2.8 on apache
> 2.0.55
> >
> --------------------------------------------------------------------------
> >
> > Key: MODPYTHON-172
> > URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> > Project: mod_python
> > Type: Bug
>
> > Components: core
> > Versions: 3.2.8
> > Environment: Win32 XP SP1 / SP2
> > Apache 2.0.55 installed from binary (.MSI)
> > Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> > Reporter: Laurent Blanquet
>
> >
> > I encounter memory leaks [~ 16 K per request) using the configuration
> described below.
> > =============================
> > Python configuration from Httpd.conf:
> > =============================
> > Alias /python/ "d:/python24/B2B/"
> > <Directory "d:/python24/B2B">
> > AddHandler mod_python .py
> > PythonHandler pyHandlerHTTP
> > PythonDebug On
> > </Directory>
> > =============================
> > Test handler - pyHandlerHTTP.py :
> > =============================
> > import mod_python
> > from mod_python import util
> > def handler(req):
> > #Removing this line solves the problem.
> > F=util.FieldStorage( req )
> > return mod_python.apache.OK
> > =============================
> > HTTP Request (dump using TCPWATCH):
> > =============================
> > POST http://localhost:80/python/Alertes.py HTTP/1.0
> > Content-Type: multipart/form-data; boundary=--------061006144341906
> > Content-Length: 209
> > Proxy-Connection: keep-alive
> > Host: www.tx2-localhost
> > Accept: text/html, */*
> > User-Agent: Mozilla/3.0 (compatible; Indy Library)
> > Proxy-Authorization: Basic Og==
> >
> > ----------061006144341906
> > Content-Disposition: form-data; name="TYPE"
> >
> > LAST_ALERTS
> > ----------061006144341906
> > Content-Disposition: form-data; name="FILEAGE"
> >
> > 180
> >
> > ----------061006144341906
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
>
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Harold Ship <ha...@giant-steps-networks.com>.
Nicolas Lehuen (JIRA <jira <at> apache.org> writes:
>
> [
http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419906 ]
>
> Nicolas Lehuen commented on MODPYTHON-172:
> ------------------------------------------
>
> I've just backported the fix into the 3.2 branch.
>
> > Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
I just wanted to say thanks to everyone for their help.
Harold
Re: *** PROBABLY SPAM *** RE: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Nicolas Lehuen <ni...@lehuen.com>.
Yes, I've done the same except that I've used Py_DECREF since the references
are guarded by previous non null tests.
https://svn.apache.org/repos/asf/httpd/mod_python/trunk/src/util.c
Regards,
Nicolas
2006/7/9, Harold J. Ship <Ha...@giant-steps-networks.com>:
>
> Just to be sure, this is the change I've made to util.c (marked with
> @HJS):
>
> while (dir) {
>
> PyObject *t = Py_BuildValue("(s, s)", dir->directive, dir->args);
> if (!t)
> return PyErr_NoMemory();
>
> PyList_Append(list, t);
> Py_XDECREF(t); // @HJS
>
> if (dir->first_child) {
>
> PyObject *child = cfgtree_walk(dir->first_child);
> if (!child)
> return PyErr_NoMemory();
>
> PyList_Append(list, child);
> Py_XDECREF(child); // @HJS
>
> }
>
> dir = dir->next;
> }
>
> ------------------------------
> *From:* nicolas.lehuen@gmail.com [mailto:nicolas.lehuen@gmail.com] *On
> Behalf Of *Nicolas Lehuen
> *Sent:* Sunday, July 09, 2006 11:46 AM
> *To:* Harold J. Ship
> *Cc:* python-dev@httpd.apache.org
> *Subject:* Re: Commented: (MODPYTHON-172) Memory leak with
> util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
>
> OK, I'm currently checking in the fixes you suggested on the trunk. Too
> bad we cannot write a unit test that checks for memory leaks.
>
> Jim, Graham, what shall we do for the 3.2.9 release ? Shall we keep on
> with the current branch or backport the fixes ?
>
> Regards,
> Nicolas
>
> 2006/7/9, Harold Ship <ha...@giant-steps-networks.com>:
> >
> >
> > I've made a test build based on 3.2.8 release, where I've added
> > Py_XDECREF()
> > calls in parse_qsl(), cfgtree_walk() TWICE (one on t, one on child), and
> > req_readlines().
> >
> > My foo/bar program doesn't leak, and I'm now testing my full
> > application. So
> > far, it seems to be ok.
> >
> >
> > Harold
> >
> >
> >
> >
>
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Graham Dumpleton wrote:
>
> On 09/07/2006, at 7:46 PM, Nicolas Lehuen wrote:
>
>> OK, I'm currently checking in the fixes you suggested on the trunk.
>> Too bad we cannot write a unit test that checks for memory leaks.
>>
>> Jim, Graham, what shall we do for the 3.2.9 release ? Shall we keep on
>> with the current branch or backport the fixes ?
>
> Were we anticipating doing another backport release in 3.2 branch prior to
> a 3.3 release if this memory leak issue hadn't come up? I have been off
> doing
> other stuff of late and haven't got around to finishing important bits
> for 3.3
> before it can be released with the new module importer. Thus, not sure when
> it might be ready. There certainly are other things that could be
> backported if
> we wanted to have another backport release. If we want to do another
> backport
> release anyway, is this memory leak serious enough to bail on 3.2.9 release
> and quickly produce a 3.2.10 with just this extra fix, or release 3.2.9
> and just
> add the memory leak fix to a future backport release for 3.2 branch.
>
> Graham
+1 skip the 3.2.9 release
+1 backport to 3.2.x
+1 release 3.1.10 asap
I'd suggested a couple of months ago that we shoot for a 3.3 release in
October / November, and these memory leaks fixes should not wait that long.
Jim
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On 09/07/2006, at 7:46 PM, Nicolas Lehuen wrote:
> OK, I'm currently checking in the fixes you suggested on the trunk.
> Too bad we cannot write a unit test that checks for memory leaks.
>
> Jim, Graham, what shall we do for the 3.2.9 release ? Shall we keep
> on with the current branch or backport the fixes ?
Were we anticipating doing another backport release in 3.2 branch
prior to
a 3.3 release if this memory leak issue hadn't come up? I have been
off doing
other stuff of late and haven't got around to finishing important
bits for 3.3
before it can be released with the new module importer. Thus, not
sure when
it might be ready. There certainly are other things that could be
backported if
we wanted to have another backport release. If we want to do another
backport
release anyway, is this memory leak serious enough to bail on 3.2.9
release
and quickly produce a 3.2.10 with just this extra fix, or release
3.2.9 and just
add the memory leak fix to a future backport release for 3.2 branch.
Graham
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Nicolas Lehuen wrote:
> OK, I'm currently checking in the fixes you suggested on the trunk. Too bad
> we cannot write a unit test that checks for memory leaks.
I'm working on a test harness - Linux only at this point. I don't think
it would be a good idea to incorporate it into the current unit tests
as it may take many hours to run the full suite. It uses top and
optionally pmap to get a detailed memory map so it should work on other
*nixes, but I'm not sure how hard it would be to make it truly cross
platform. I may drop the pmap option all together as it outputs more
information than is really necessary, and top does the trick.
Right now it does not use the unittest framework, but I think I'll
change it to do so. Once I've cleaned up the code a little I'll make it
publicly available. (Unless someone would like to play with the current
hack - but it isn't pretty ;) ).
Oh, and just in case there was any doubt, req_readlines, parse_qsl and
cfgtree_walk do indeed leak memory.
Jim
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Harold Ship <ha...@giant-steps-networks.com>.
If possible, I would like a backport release.
Nicolas Lehuen <nicolas <at> lehuen.com> writes:
> OK, I'm currently checking in the fixes you suggested on the trunk. Too bad we
cannot write a unit test that checks for memory leaks.Jim, Graham, what shall we
do for the 3.2.9 release ? Shall we keep on with the current branch or backport
the fixes ?
> Regards,Nicolas
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Nicolas Lehuen <ni...@lehuen.com>.
OK, I'm currently checking in the fixes you suggested on the trunk. Too bad
we cannot write a unit test that checks for memory leaks.
Jim, Graham, what shall we do for the 3.2.9 release ? Shall we keep on with
the current branch or backport the fixes ?
Regards,
Nicolas
2006/7/9, Harold Ship <ha...@giant-steps-networks.com>:
>
>
> I've made a test build based on 3.2.8 release, where I've added
> Py_XDECREF()
> calls in parse_qsl(), cfgtree_walk() TWICE (one on t, one on child), and
> req_readlines().
>
> My foo/bar program doesn't leak, and I'm now testing my full application.
> So
> far, it seems to be ok.
>
>
> Harold
>
>
>
>
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Harold Ship <ha...@giant-steps-networks.com>.
I've made a test build based on 3.2.8 release, where I've added Py_XDECREF()
calls in parse_qsl(), cfgtree_walk() TWICE (one on t, one on child), and
req_readlines().
My foo/bar program doesn't leak, and I'm now testing my full application. So
far, it seems to be ok.
Harold
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Graham Dumpleton wrote:
> On first look I would agree there is possibly a problem. There also
> would appear to be
> similar issues in other parts of mod_python. For example in
> cfgtree_walk() of util.c it has:
>
> PyObject *t = Py_BuildValue("(s, s)", dir->directive, dir->args);
> if (!t)
> return PyErr_NoMemory();
>
> PyList_Append(list, t);
>
> with "t" not being DECREF'd either.
>
> Also, in req_readlines() of requestobject.c, we have:
>
> line = req_readline(self, rlargs);
> while (line && (PyString_Size(line)>0)) {
> PyList_Append(result, line);
> size += PyString_Size(line);
> if ((sizehint != -1) && (size >= size))
> break;
> line = req_readline(self, args);
> }
>
> No DECREF for item added to list.
>
> I can't see any others in relation to dictionaries yet, but we probably
> need to do a good
> audit of all the code for such things as I have only check
> PyList_Append() and
> PyDict_SetItem() and not the other ways stuff can be added to such data
> structures.
>
> Anyway, still need to run some tests yet to confirm memory leak. :-)
I've been working through the PyList_Append instances and noticed the
cfgtree_walk and req_readlines issues as well. I've confirmed that
cfgtree_walk does indeed leak. I haven't tested req_readlines but I
can't see why it wouldn't leak.
Can you spot the other bug in req_readlines? Hint: (size >= size) will
always be true. ;) Once that is fixed I think we need to alter the docs
for req.readlines() as I'm not sure the description for sizehint
accurately reflects the way it works.
Jim
> Graham
>
> On 08/07/2006, at 1:05 AM, Harold Ship (JIRA) wrote:
>
>> [
>> http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419728
>> ]
>>
>> Harold Ship commented on MODPYTHON-172:
>> ---------------------------------------
>>
>> Please look at lines 202-205 of the same file, in parse_qs() :
>>
>> PyObject *list;
>> list = Py_BuildValue("[O]", val);
>> PyDict_SetItem(dict, key, list);
>> Py_DECREF(list);
>>
>>
>>
>>> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache
>>> 2.0.55
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Key: MODPYTHON-172
>>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>>> Project: mod_python
>>> Type: Bug
>>
>>> Components: core
>>> Versions: 3.2.8
>>> Environment: Win32 XP SP1 / SP2
>>> Apache 2.0.55 installed from binary (.MSI)
>>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>>> Reporter: Laurent Blanquet
>>
>>>
>>> I encounter memory leaks [~ 16 K per request) using the configuration
>>> described below.
>>> =============================
>>> Python configuration from Httpd.conf:
>>> =============================
>>> Alias /python/ "d:/python24/B2B/"
>>> <Directory "d:/python24/B2B">
>>> AddHandler mod_python .py
>>> PythonHandler pyHandlerHTTP
>>> PythonDebug On
>>> </Directory>
>>> =============================
>>> Test handler - pyHandlerHTTP.py :
>>> =============================
>>> import mod_python
>>> from mod_python import util
>>> def handler(req):
>>> #Removing this line solves the problem.
>>> F=util.FieldStorage( req )
>>> return mod_python.apache.OK
>>> =============================
>>> HTTP Request (dump using TCPWATCH):
>>> =============================
>>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>>> Content-Type: multipart/form-data; boundary=--------061006144341906
>>> Content-Length: 209
>>> Proxy-Connection: keep-alive
>>> Host: www.tx2-localhost
>>> Accept: text/html, */*
>>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>>> Proxy-Authorization: Basic Og==
>>>
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="TYPE"
>>>
>>> LAST_ALERTS
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="FILEAGE"
>>>
>>> 180
>>>
>>> ----------061006144341906
>>
>> --This message is automatically generated by JIRA.
>> -
>> If you think it was sent incorrectly contact one of the administrators:
>> http://issues.apache.org/jira/secure/Administrators.jspa
>> -
>> For more information on JIRA, see:
>> http://www.atlassian.com/software/jira
>
>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On first look I would agree there is possibly a problem. There also
would appear to be
similar issues in other parts of mod_python. For example in
cfgtree_walk() of util.c it has:
PyObject *t = Py_BuildValue("(s, s)", dir->directive, dir-
>args);
if (!t)
return PyErr_NoMemory();
PyList_Append(list, t);
with "t" not being DECREF'd either.
Also, in req_readlines() of requestobject.c, we have:
line = req_readline(self, rlargs);
while (line && (PyString_Size(line)>0)) {
PyList_Append(result, line);
size += PyString_Size(line);
if ((sizehint != -1) && (size >= size))
break;
line = req_readline(self, args);
}
No DECREF for item added to list.
I can't see any others in relation to dictionaries yet, but we
probably need to do a good
audit of all the code for such things as I have only check
PyList_Append() and
PyDict_SetItem() and not the other ways stuff can be added to such
data structures.
Anyway, still need to run some tests yet to confirm memory leak. :-)
Graham
On 08/07/2006, at 1:05 AM, Harold Ship (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/MODPYTHON-172?
> page=comments#action_12419728 ]
>
> Harold Ship commented on MODPYTHON-172:
> ---------------------------------------
>
> Please look at lines 202-205 of the same file, in parse_qs() :
>
> PyObject *list;
> list = Py_BuildValue("[O]", val);
> PyDict_SetItem(dict, key, list);
> Py_DECREF(list);
>
>
>
>> Memory leak with util.fieldstorage using mod_python 3.2.8 on
>> apache 2.0.55
>> ---------------------------------------------------------------------
>> -----
>>
>> Key: MODPYTHON-172
>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>> Project: mod_python
>> Type: Bug
>
>> Components: core
>> Versions: 3.2.8
>> Environment: Win32 XP SP1 / SP2
>> Apache 2.0.55 installed from binary (.MSI)
>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>> Reporter: Laurent Blanquet
>
>>
>> I encounter memory leaks [~ 16 K per request) using the
>> configuration described below.
>> =============================
>> Python configuration from Httpd.conf:
>> =============================
>> Alias /python/ "d:/python24/B2B/"
>> <Directory "d:/python24/B2B">
>> AddHandler mod_python .py
>> PythonHandler pyHandlerHTTP
>> PythonDebug On
>> </Directory>
>> =============================
>> Test handler - pyHandlerHTTP.py :
>> =============================
>> import mod_python
>> from mod_python import util
>> def handler(req):
>> #Removing this line solves the problem.
>> F=util.FieldStorage( req )
>> return mod_python.apache.OK
>> =============================
>> HTTP Request (dump using TCPWATCH):
>> =============================
>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>> Content-Type: multipart/form-data; boundary=--------061006144341906
>> Content-Length: 209
>> Proxy-Connection: keep-alive
>> Host: www.tx2-localhost
>> Accept: text/html, */*
>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>> Proxy-Authorization: Basic Og==
>>
>> ----------061006144341906
>> Content-Disposition: form-data; name="TYPE"
>>
>> LAST_ALERTS
>> ----------061006144341906
>> Content-Disposition: form-data; name="FILEAGE"
>>
>> 180
>>
>> ----------061006144341906
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the
> administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Nicolas Lehuen <ni...@lehuen.com>.
Yeah Harold, thanks for the time you are spending on this bug.
I'm not ignoring you either, I'm just on vacation, trying to disconnect from
my work in particular and the net in general. Not so easy as this mail tends
to show.
I'll have a look at this too when I come back to civilization.
Regards,
Nicolas
2006/7/7, Jim Gallacher <jp...@jgassociates.ca>:
>
> Hi Harold,
>
> I just wanted to let you know you are not being ignored. I just need
> some free time to dive into this - hopefully this weekend.
>
> Jim
>
> Harold Ship (JIRA) wrote:
> > [
> http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419728]
> >
> > Harold Ship commented on MODPYTHON-172:
> > ---------------------------------------
> >
> > Please look at lines 202-205 of the same file, in parse_qs() :
> >
> > PyObject *list;
> > list = Py_BuildValue("[O]", val);
> > PyDict_SetItem(dict, key, list);
> > Py_DECREF(list);
> >
> >
> >
> >> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache
> 2.0.55
> >>
> --------------------------------------------------------------------------
> >>
> >> Key: MODPYTHON-172
> >> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> >> Project: mod_python
> >> Type: Bug
> >
> >> Components: core
> >> Versions: 3.2.8
> >> Environment: Win32 XP SP1 / SP2
> >> Apache 2.0.55 installed from binary (.MSI)
> >> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> >> Reporter: Laurent Blanquet
> >
> >> I encounter memory leaks [~ 16 K per request) using the configuration
> described below.
> >> =============================
> >> Python configuration from Httpd.conf:
> >> =============================
> >> Alias /python/ "d:/python24/B2B/"
> >> <Directory "d:/python24/B2B">
> >> AddHandler mod_python .py
> >> PythonHandler pyHandlerHTTP
> >> PythonDebug On
> >> </Directory>
> >> =============================
> >> Test handler - pyHandlerHTTP.py :
> >> =============================
> >> import mod_python
> >> from mod_python import util
> >> def handler(req):
> >> #Removing this line solves the problem.
> >> F=util.FieldStorage( req )
> >> return mod_python.apache.OK
> >> =============================
> >> HTTP Request (dump using TCPWATCH):
> >> =============================
> >> POST http://localhost:80/python/Alertes.py HTTP/1.0
> >> Content-Type: multipart/form-data; boundary=--------061006144341906
> >> Content-Length: 209
> >> Proxy-Connection: keep-alive
> >> Host: www.tx2-localhost
> >> Accept: text/html, */*
> >> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> >> Proxy-Authorization: Basic Og==
> >>
> >> ----------061006144341906
> >> Content-Disposition: form-data; name="TYPE"
> >>
> >> LAST_ALERTS
> >> ----------061006144341906
> >> Content-Disposition: form-data; name="FILEAGE"
> >>
> >> 180
> >>
> >> ----------061006144341906
> >
>
>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Nicolas Lehuen <ni...@lehuen.com>.
Hi Harold,
With Visual Studio .NET 2003, it's quite easy, just cd into the dist
directory and launch build_installer.bat. You should eventually get an
installer into the dist/dist directory. Note that with Apache 2.2, you may
need to tweak the setup.py.in file manually a little bit.
Regards,
Nicolas
2006/7/8, Harold J. Ship <Ha...@giant-steps-networks.com>:
>
> Thanks all.
>
> I wasn't worried. I would also like to help, if possible. Can someone
> point out some instructions on building mod_python on Windows? I'm using
> Activestate Python 2.3.5, and have Visual Studio 6.0 and .NET 2003.
>
> Harold
>
>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Graham Dumpleton <gr...@dscpl.com.au>.
On 08/07/2006, at 3:06 AM, Jim Gallacher wrote:
> Hi Harold,
>
> I just wanted to let you know you are not being ignored. I just need
> some free time to dive into this - hopefully this weekend.
My excuse too. ;-)
I also was hoping to get to it this weekend. I have been back from
holidays
almost a month and managed to get zero down on mod_python since.
Graham
> Harold Ship (JIRA) wrote:
>> [ http://issues.apache.org/jira/browse/MODPYTHON-172?
>> page=comments#action_12419728 ]
>>
>> Harold Ship commented on MODPYTHON-172:
>> ---------------------------------------
>>
>> Please look at lines 202-205 of the same file, in parse_qs() :
>>
>> PyObject *list;
>> list = Py_BuildValue("[O]", val);
>> PyDict_SetItem(dict, key, list);
>> Py_DECREF(list);
>>
>>
>>
>>> Memory leak with util.fieldstorage using mod_python 3.2.8 on
>>> apache 2.0.55
>>> --------------------------------------------------------------------
>>> ------
>>>
>>> Key: MODPYTHON-172
>>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>>> Project: mod_python
>>> Type: Bug
>>
>>> Components: core
>>> Versions: 3.2.8
>>> Environment: Win32 XP SP1 / SP2
>>> Apache 2.0.55 installed from binary (.MSI)
>>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>>> Reporter: Laurent Blanquet
>>
>>> I encounter memory leaks [~ 16 K per request) using the
>>> configuration described below.
>>> =============================
>>> Python configuration from Httpd.conf:
>>> =============================
>>> Alias /python/ "d:/python24/B2B/"
>>> <Directory "d:/python24/B2B">
>>> AddHandler mod_python .py
>>> PythonHandler pyHandlerHTTP
>>> PythonDebug On
>>> </Directory>
>>> =============================
>>> Test handler - pyHandlerHTTP.py :
>>> =============================
>>> import mod_python
>>> from mod_python import util
>>> def handler(req):
>>> #Removing this line solves the problem.
>>> F=util.FieldStorage( req )
>>> return mod_python.apache.OK
>>> =============================
>>> HTTP Request (dump using TCPWATCH):
>>> =============================
>>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>>> Content-Type: multipart/form-data; boundary=--------061006144341906
>>> Content-Length: 209
>>> Proxy-Connection: keep-alive
>>> Host: www.tx2-localhost
>>> Accept: text/html, */*
>>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>>> Proxy-Authorization: Basic Og==
>>>
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="TYPE"
>>>
>>> LAST_ALERTS
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="FILEAGE"
>>>
>>> 180
>>>
>>> ----------061006144341906
>>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Hi Harold,
I just wanted to let you know you are not being ignored. I just need
some free time to dive into this - hopefully this weekend.
Jim
Harold Ship (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419728 ]
>
> Harold Ship commented on MODPYTHON-172:
> ---------------------------------------
>
> Please look at lines 202-205 of the same file, in parse_qs() :
>
> PyObject *list;
> list = Py_BuildValue("[O]", val);
> PyDict_SetItem(dict, key, list);
> Py_DECREF(list);
>
>
>
>> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
>> --------------------------------------------------------------------------
>>
>> Key: MODPYTHON-172
>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>> Project: mod_python
>> Type: Bug
>
>> Components: core
>> Versions: 3.2.8
>> Environment: Win32 XP SP1 / SP2
>> Apache 2.0.55 installed from binary (.MSI)
>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>> Reporter: Laurent Blanquet
>
>> I encounter memory leaks [~ 16 K per request) using the configuration described below.
>> =============================
>> Python configuration from Httpd.conf:
>> =============================
>> Alias /python/ "d:/python24/B2B/"
>> <Directory "d:/python24/B2B">
>> AddHandler mod_python .py
>> PythonHandler pyHandlerHTTP
>> PythonDebug On
>> </Directory>
>> =============================
>> Test handler - pyHandlerHTTP.py :
>> =============================
>> import mod_python
>> from mod_python import util
>> def handler(req):
>> #Removing this line solves the problem.
>> F=util.FieldStorage( req )
>> return mod_python.apache.OK
>> =============================
>> HTTP Request (dump using TCPWATCH):
>> =============================
>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>> Content-Type: multipart/form-data; boundary=--------061006144341906
>> Content-Length: 209
>> Proxy-Connection: keep-alive
>> Host: www.tx2-localhost
>> Accept: text/html, */*
>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>> Proxy-Authorization: Basic Og==
>>
>> ----------061006144341906
>> Content-Disposition: form-data; name="TYPE"
>>
>> LAST_ALERTS
>> ----------061006144341906
>> Content-Disposition: form-data; name="FILEAGE"
>>
>> 180
>>
>> ----------061006144341906
>
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Harold Ship (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419728 ]
Harold Ship commented on MODPYTHON-172:
---------------------------------------
Please look at lines 202-205 of the same file, in parse_qs() :
PyObject *list;
list = Py_BuildValue("[O]", val);
PyDict_SetItem(dict, key, list);
Py_DECREF(list);
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: memory leaks (was Re: Commented: (MODPYTHON-172) Memory leak
with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55)
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Harold Ship wrote:
> Jim Gallacher <jpg <at> jgassociates.ca> writes:
>
>> I've run some tests to evaluate the memory leaks. The tests are brute
>> force - make requests and watch the memory changes with top -b.
>>
>
> ...
>
>> First up - our leaky 3.2.9, and man does it leak! The readlines request
>> has a body of 1M bytes, and it fails in a hurry.
>>
>
> ...
>
>> Everything is looking much better, with readlines looking very stable.
>> Cfgtree_walk and parse_qsl are also looking much better but there may
>> still be a problem. To confirm this I ran the tests against 3.2.x build
>> 20060709 again, but with 100,000 requests.
>>
>
> Did you have a chance to re-run the tests?
I ran my test using FieldStorage but haven't had a chance to go further.
For 500k requests memory consumption increases from 1.1% to 10%
(mpm-worker, StartServers = 2) so it still leaks. This is still an
improvement as the machine would have crumpled far before the 500k
request mark, but a leak is a leak. I assume the main culprit in
FieldStorage is parse_qsl, but I don't see any obvious problem.
> There are some things I've found, although I don't think they cause a leak.
> For example, in parse_qs() there is:
>
> pair = PyString_FromStringAndSize(NULL, len);
> if (pair == NULL)
> return NULL;
>
> this should be:
>
> pair = PyString_FromStringAndSize(NULL, len);
> if (pair == NULL) {
> Py_DECREF(pairs);
> return NULL;
> }
>
> There are a few more like that in parse_qs() and parse_qsl(), and cfgtree_walk().
>
> In req_readlines(), line 810/811 the intention isn't clear of:
>
> if (result == NULL)
> return PyErr_NoMemory();
I agree it would be clearer if the result object creation was
immediately before the test. eg
PyObject *result = PyList_New(0);
...
result = PyList_New(0);
if (result == NULL)
return PyErr_NoMemory()
> While the exact same code on lines 814/815 are not correct.
> They should be:
>
> if (rlargs == NULL) { // note rlargs not result
> Py_DECREF(result);
> return PyErr_NoMemory();
> }
(Good catch on the rlargs thing).
I think these are mainly just coding inconsistencies, and agree that
they are should not be the cause of the current memory leaks. If one of
the object creation functions fails due to lack of memory, our failure
to decrement a previously created object is hardly going to matter.
Sloppy programming? - for sure.
The use of PyErr_NoMemory() in the above code fragments does concern me
a little though.
PyErr_NoMemory() is shorthand for PyErr_SetNone(PyExc_MemoryError) and
returns NULL. If a call such as PyList_New(0) fails, doesn't it have the
responsibility to set the exception, and our calling function should
simply return NULL? I had the idea that we should only call
PyErr_NoMemory if we are doing a malloc in our code and it fails. Is
there something unique about PyList_New and PyTuple_New that I'm missing?
By the way Harold, it would be helpful if when talking about specific
code lines of code if you were referencing the current development in
svn trunk. Generally we've been making changes in trunk, and backporting
to branches/3.2.x as required. Anyway, thanks for your continuing
efforts. :)
Jim
Re: memory leaks (was Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55)
Posted by Harold Ship <ha...@giant-steps-networks.com>.
Jim Gallacher <jpg <at> jgassociates.ca> writes:
>
> I've run some tests to evaluate the memory leaks. The tests are brute
> force - make requests and watch the memory changes with top -b.
>
...
> First up - our leaky 3.2.9, and man does it leak! The readlines request
> has a body of 1M bytes, and it fails in a hurry.
>
...
> Everything is looking much better, with readlines looking very stable.
> Cfgtree_walk and parse_qsl are also looking much better but there may
> still be a problem. To confirm this I ran the tests against 3.2.x build
> 20060709 again, but with 100,000 requests.
>
Did you have a chance to re-run the tests?
There are some things I've found, although I don't think they cause a leak.
For example, in parse_qs() there is:
pair = PyString_FromStringAndSize(NULL, len);
if (pair == NULL)
return NULL;
this should be:
pair = PyString_FromStringAndSize(NULL, len);
if (pair == NULL) {
Py_DECREF(pairs);
return NULL;
}
There are a few more like that in parse_qs() and parse_qsl(), and cfgtree_walk().
In req_readlines(), line 810/811 the intention isn't clear of:
if (result == NULL)
return PyErr_NoMemory();
While the exact same code on lines 814/815 are not correct.
They should be:
if (rlargs == NULL) { // note rlargs not result
Py_DECREF(result);
return PyErr_NoMemory();
}
memory leaks (was Re: [jira] Commented: (MODPYTHON-172) Memory leak
with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55)
Posted by Jim Gallacher <jp...@jgassociates.ca>.
I've run some tests to evaluate the memory leaks. The tests are brute
force - make requests and watch the memory changes with top -b.
These tests were run using Ubuntu 6.06, Apache 2.0.55 (mpm-worker with
StartServers = 2) and Python 2.4.3.
The baseline test is just a simple request to determine the initial
memory consumption and in theory should not leak. The memory footprint
is checked after each 1000 requests, and if greater than the memory
maximum the test is terminated and marked as FAIL. Tests with PASS
status may still be leaking, just not in a dramatic, fatal manner.
First up - our leaky 3.2.9, and man does it leak! The readlines request
has a body of 1M bytes, and it fails in a hurry.
============================================================
Mod_python Memory Leak Test
mod_python version: 3.2.9
requests attempted: 50000
memory maximum for failure condition: 10.0%
============================================================
Test Name Status Requests %Mem
------------------------------------------------------------
baseline PASS 50000 1.1
cfgtree_walk FAIL 5000 11.5
parse_qs PASS 50000 2.2
parse_qsl FAIL 15000 10.3
readlines FAIL 1000 14.3
The following is the lastest from branches/3.2.x (includes backported
MODPYTHON-172 changes.)
============================================================
Mod_python Memory Leak Test
mod_python version: 3.2.x build 20060709
requests attempted: 50000
memory maximum for failure condition: 10.0%
============================================================
Test Name Status Requests %Mem
------------------------------------------------------------
baseline PASS 50000 1.1
cfgtree_walk PASS 50000 2.0
parse_qs PASS 50000 2.2
parse_qsl PASS 50000 2.1
readlines PASS 50000 1.3
Everything is looking much better, with readlines looking very stable.
Cfgtree_walk and parse_qsl are also looking much better but there may
still be a problem. To confirm this I ran the tests against 3.2.x build
20060709 again, but with 100,000 requests.
============================================================
Mod_python Memory Leak Test
mod_python version: 3.2.x build 20060709
requests attempted: 100000
memory maximum for failure condition: 10.0%
============================================================
Test Name Status Requests %Mem
------------------------------------------------------------
baseline PASS 100000 1.2
cfgtree_walk PASS 100000 2.9
parse_qs PASS 100000 3.0
parse_qsl PASS 100000 3.1
readlines PASS 100000 1.3
I'm still suspicious of cfgtree_walk, parse_qs and parse_qsl as their
memory consumption is creeping up. I've taken a quick look at the code
but I don't see any obvious problem. I'll re-run the tests tonight, but
with a much larger number of requests.
The next step is to look at other methods we should include for testing.
I would be nice to get 100% coverage, but I'm not sure that is really
practical. We should include the most commonly called methods as a minimum.
Jim
Jim Gallacher wrote:
> I'm running my leaktest suite against this commit now. Results to follow.
>
> Jim
>
> Nicolas Lehuen (JIRA) wrote:
>> [ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419906 ]
>>
>> Nicolas Lehuen commented on MODPYTHON-172:
>> ------------------------------------------
>>
>> I've just backported the fix into the 3.2 branch.
>>
>>> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
>>> --------------------------------------------------------------------------
>>>
>>> Key: MODPYTHON-172
>>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>>> Project: mod_python
>>> Type: Bug
>>> Components: core
>>> Versions: 3.2.8
>>> Environment: Win32 XP SP1 / SP2
>>> Apache 2.0.55 installed from binary (.MSI)
>>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>>> Reporter: Laurent Blanquet
>>> Fix For: 3.3
>>> I encounter memory leaks [~ 16 K per request) using the configuration described below.
>>> =============================
>>> Python configuration from Httpd.conf:
>>> =============================
>>> Alias /python/ "d:/python24/B2B/"
>>> <Directory "d:/python24/B2B">
>>> AddHandler mod_python .py
>>> PythonHandler pyHandlerHTTP
>>> PythonDebug On
>>> </Directory>
>>> =============================
>>> Test handler - pyHandlerHTTP.py :
>>> =============================
>>> import mod_python
>>> from mod_python import util
>>> def handler(req):
>>> #Removing this line solves the problem.
>>> F=util.FieldStorage( req )
>>> return mod_python.apache.OK
>>> =============================
>>> HTTP Request (dump using TCPWATCH):
>>> =============================
>>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>>> Content-Type: multipart/form-data; boundary=--------061006144341906
>>> Content-Length: 209
>>> Proxy-Connection: keep-alive
>>> Host: www.tx2-localhost
>>> Accept: text/html, */*
>>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>>> Proxy-Authorization: Basic Og==
>>>
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="TYPE"
>>>
>>> LAST_ALERTS
>>> ----------061006144341906
>>> Content-Disposition: form-data; name="FILEAGE"
>>>
>>> 180
>>>
>>> ----------061006144341906
>
>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by Jim Gallacher <jp...@jgassociates.ca>.
I'm running my leaktest suite against this commit now. Results to follow.
Jim
Nicolas Lehuen (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419906 ]
>
> Nicolas Lehuen commented on MODPYTHON-172:
> ------------------------------------------
>
> I've just backported the fix into the 3.2 branch.
>
>> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
>> --------------------------------------------------------------------------
>>
>> Key: MODPYTHON-172
>> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
>> Project: mod_python
>> Type: Bug
>
>> Components: core
>> Versions: 3.2.8
>> Environment: Win32 XP SP1 / SP2
>> Apache 2.0.55 installed from binary (.MSI)
>> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
>> Reporter: Laurent Blanquet
>> Fix For: 3.3
>
>> I encounter memory leaks [~ 16 K per request) using the configuration described below.
>> =============================
>> Python configuration from Httpd.conf:
>> =============================
>> Alias /python/ "d:/python24/B2B/"
>> <Directory "d:/python24/B2B">
>> AddHandler mod_python .py
>> PythonHandler pyHandlerHTTP
>> PythonDebug On
>> </Directory>
>> =============================
>> Test handler - pyHandlerHTTP.py :
>> =============================
>> import mod_python
>> from mod_python import util
>> def handler(req):
>> #Removing this line solves the problem.
>> F=util.FieldStorage( req )
>> return mod_python.apache.OK
>> =============================
>> HTTP Request (dump using TCPWATCH):
>> =============================
>> POST http://localhost:80/python/Alertes.py HTTP/1.0
>> Content-Type: multipart/form-data; boundary=--------061006144341906
>> Content-Length: 209
>> Proxy-Connection: keep-alive
>> Host: www.tx2-localhost
>> Accept: text/html, */*
>> User-Agent: Mozilla/3.0 (compatible; Indy Library)
>> Proxy-Authorization: Basic Og==
>>
>> ----------061006144341906
>> Content-Disposition: form-data; name="TYPE"
>>
>> LAST_ALERTS
>> ----------061006144341906
>> Content-Disposition: form-data; name="FILEAGE"
>>
>> 180
>>
>> ----------061006144341906
>
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419906 ]
Nicolas Lehuen commented on MODPYTHON-172:
------------------------------------------
I've just backported the fix into the 3.2 branch.
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by "Graham Dumpleton (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=all ]
Graham Dumpleton updated MODPYTHON-172:
---------------------------------------
Fix Version/s: 3.2.10
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3, 3.2.10
>
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Harold Ship <ha...@giant-steps-networks.com>.
Are the memory leaks with the patch still 16K per request?
<snip>
>
> And I always see some memory leaks after having installed the patch :
>
Re: [jira] Commented: (MODPYTHON-172) Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by Laurent Blanquet <lb...@b2btechno.net>.
Hello,
my configuration is always :
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
I've tested with Handler :
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
import mod_python
from mod_python import util
def handler(req):
#Removing this line solves the problem.
F=util.FieldStorage( req )
return mod_python.apache.OK
And I always see some memory leaks after having installed the patch :
> old util.py:
> parse_qs = _apache.parse_qs
> parse_qsl = _apache.parse_qsl
>
>
> workaround util.py:
> parse_qs = _apache.parse_qs
> #parse_qsl = _apache.parse_qsl
> from cgi import parse_qsl
>
Laurent.
--
----- Original Message -----
From: "Harold Ship (JIRA)" <ji...@apache.org>
To: <py...@httpd.apache.org>
Sent: Friday, June 30, 2006 8:58 AM
Subject: [jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> [
http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12418577 ]
>
> Harold Ship commented on MODPYTHON-172:
> ---------------------------------------
>
> I've been able to reproduce the problem with "mod_python.publisher".
>
> Using:
> Windows 2000 Professional
> apache 2.0.54
> python 2.3.5
> mod_python 3.1.3
>
> the following script:
> #foo.py
> def bar(req):
> a=req.form.getfirst('a')
> req.write('a=%s'%str(a))
> req.write('Ok')
>
> repeatedly sending requests to <addr>/foo/bar?a=b causes memory to rise
continuously. removing the querystring does not. I used task manager to
measure memory.
>
> I've also found that the following change to util.py corrects this
behaviour:
>
> old util.py:
> parse_qs = _apache.parse_qs
> parse_qsl = _apache.parse_qsl
>
>
> workaround util.py:
> parse_qs = _apache.parse_qs
> #parse_qsl = _apache.parse_qsl
> from cgi import parse_qsl
>
>
> > Memory leak with util.fieldstorage using mod_python 3.2.8 on apache
2.0.55
>
> --------------------------------------------------------------------------
> >
> > Key: MODPYTHON-172
> > URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> > Project: mod_python
> > Type: Bug
>
> > Components: core
> > Versions: 3.2.8
> > Environment: Win32 XP SP1 / SP2
> > Apache 2.0.55 installed from binary (.MSI)
> > Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> > Reporter: Laurent Blanquet
>
> >
> > I encounter memory leaks [~ 16 K per request) using the configuration
described below.
> > =============================
> > Python configuration from Httpd.conf:
> > =============================
> > Alias /python/ "d:/python24/B2B/"
> > <Directory "d:/python24/B2B">
> > AddHandler mod_python .py
> > PythonHandler pyHandlerHTTP
> > PythonDebug On
> > </Directory>
> > =============================
> > Test handler - pyHandlerHTTP.py :
> > =============================
> > import mod_python
> > from mod_python import util
> > def handler(req):
> > #Removing this line solves the problem.
> > F=util.FieldStorage( req )
> > return mod_python.apache.OK
> > =============================
> > HTTP Request (dump using TCPWATCH):
> > =============================
> > POST http://localhost:80/python/Alertes.py HTTP/1.0
> > Content-Type: multipart/form-data; boundary=--------061006144341906
> > Content-Length: 209
> > Proxy-Connection: keep-alive
> > Host: www.tx2-localhost
> > Accept: text/html, */*
> > User-Agent: Mozilla/3.0 (compatible; Indy Library)
> > Proxy-Authorization: Basic Og==
> >
> > ----------061006144341906
> > Content-Disposition: form-data; name="TYPE"
> >
> > LAST_ALERTS
> > ----------061006144341906
> > Content-Disposition: form-data; name="FILEAGE"
> >
> > 180
> >
> > ----------061006144341906
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
>
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Harold Ship (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12418577 ]
Harold Ship commented on MODPYTHON-172:
---------------------------------------
I've been able to reproduce the problem with "mod_python.publisher".
Using:
Windows 2000 Professional
apache 2.0.54
python 2.3.5
mod_python 3.1.3
the following script:
#foo.py
def bar(req):
a=req.form.getfirst('a')
req.write('a=%s'%str(a))
req.write('Ok')
repeatedly sending requests to <addr>/foo/bar?a=b causes memory to rise continuously. removing the querystring does not. I used task manager to measure memory.
I've also found that the following change to util.py corrects this behaviour:
old util.py:
parse_qs = _apache.parse_qs
parse_qsl = _apache.parse_qsl
workaround util.py:
parse_qs = _apache.parse_qs
#parse_qsl = _apache.parse_qsl
from cgi import parse_qsl
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Jim Gallacher (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12447658 ]
Jim Gallacher commented on MODPYTHON-172:
-----------------------------------------
[[ Old comment, sent by email on Fri, 07 Jul 2006 21:26:01 -0400 ]]
I've been working through the PyList_Append instances and noticed the
cfgtree_walk and req_readlines issues as well. I've confirmed that
cfgtree_walk does indeed leak. I haven't tested req_readlines but I
can't see why it wouldn't leak.
Can you spot the other bug in req_readlines? Hint: (size >= size) will
always be true. ;) Once that is fixed I think we need to alter the docs
for req.readlines() as I'm not sure the description for sizehint
accurately reflects the way it works.
Jim
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3, 3.2.10
>
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Harold Ship (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12419534 ]
Harold Ship commented on MODPYTHON-172:
---------------------------------------
Excuse me if this is a dumb question, but I don't know anything about the Python-C interface.
Is it possible that the code in line 319 of _apachemodule.c (from 3.2.8) is the problem?
The code is:
if (key && val)
PyList_Append(pairs, Py_BuildValue("(O,O)", key, val));
Does the object returned by Py_BuildValue() need to be dereferenced? Something like:
if (key && val) {
PyObject * list_elem = Py_BuildValue("(O,O)", key, val);
if (list_elem)
PyList_Append(pairs, list_elem);
Py_XDECREF(list_elem);
}
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Laurent Blanquet (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12442162 ]
Laurent Blanquet commented on MODPYTHON-172:
--------------------------------------------
[[ Old comment, sent by email on Fri, 30 Jun 2006 15:03:22 +0200 ]]
Hello,
I've tested with Handler :
import mod_python
from mod_python import util
def handler(req):
#Removing this line solves the problem.
F=util.FieldStorage( req )
return mod_python.apache.OK
And I always see some memory leaks after having installed the patch :
Laurent.
--
----- Original Message -----
From: "Harold Ship (JIRA)" <ji...@apache.org>
To: <py...@httpd.apache.org>
Sent: Friday, June 30, 2006 8:58 AM
Subject: [jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12418577 ]
continuously. removing the querystring does not. I used task manager to
measure memory.
behaviour:
2.0.55
described below.
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3, 3.2.10
>
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by "Graham Dumpleton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MODPYTHON-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Graham Dumpleton closed MODPYTHON-172.
--------------------------------------
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: https://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3, 3.2.10
>
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MODPYTHON-172) Memory leak with
util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
Posted by "Graham Dumpleton (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=comments#action_12442163 ]
Graham Dumpleton commented on MODPYTHON-172:
--------------------------------------------
[[ Old comment, sent by email on Sat, 8 Jul 2006 10:27:48 +1000 ]]
On first look I would agree there is possibly a problem. There also
would appear to be
similar issues in other parts of mod_python. For example in
cfgtree_walk() of util.c it has:
PyObject *t = Py_BuildValue("(s, s)", dir->directive, dir-
>args);
if (!t)
return PyErr_NoMemory();
PyList_Append(list, t);
with "t" not being DECREF'd either.
Also, in req_readlines() of requestobject.c, we have:
line = req_readline(self, rlargs);
while (line && (PyString_Size(line)>0)) {
PyList_Append(result, line);
size += PyString_Size(line);
if ((sizehint != -1) && (size >= size))
break;
line = req_readline(self, args);
}
No DECREF for item added to list.
I can't see any others in relation to dictionaries yet, but we
probably need to do a good
audit of all the code for such things as I have only check
PyList_Append() and
PyDict_SetItem() and not the other ways stuff can be added to such
data structures.
Anyway, still need to run some tests yet to confirm memory leak. :-)
Graham
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3, 3.2.10
>
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (MODPYTHON-172) Memory leak with util.fieldstorage
using mod_python 3.2.8 on apache 2.0.55
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-172?page=all ]
Nicolas Lehuen resolved MODPYTHON-172:
--------------------------------------
Fix Version: 3.3
Resolution: Fixed
Fixed in the trunk, we need to apply changes from revision #420275 if we want to backport this into the 3.2 branch.
> Memory leak with util.fieldstorage using mod_python 3.2.8 on apache 2.0.55
> --------------------------------------------------------------------------
>
> Key: MODPYTHON-172
> URL: http://issues.apache.org/jira/browse/MODPYTHON-172
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.2.8
> Environment: Win32 XP SP1 / SP2
> Apache 2.0.55 installed from binary (.MSI)
> Python 2.4.2 or 2.4.3 installed from binary from www.python.org
> Reporter: Laurent Blanquet
> Fix For: 3.3
>
> I encounter memory leaks [~ 16 K per request) using the configuration described below.
> =============================
> Python configuration from Httpd.conf:
> =============================
> Alias /python/ "d:/python24/B2B/"
> <Directory "d:/python24/B2B">
> AddHandler mod_python .py
> PythonHandler pyHandlerHTTP
> PythonDebug On
> </Directory>
> =============================
> Test handler - pyHandlerHTTP.py :
> =============================
> import mod_python
> from mod_python import util
> def handler(req):
> #Removing this line solves the problem.
> F=util.FieldStorage( req )
> return mod_python.apache.OK
> =============================
> HTTP Request (dump using TCPWATCH):
> =============================
> POST http://localhost:80/python/Alertes.py HTTP/1.0
> Content-Type: multipart/form-data; boundary=--------061006144341906
> Content-Length: 209
> Proxy-Connection: keep-alive
> Host: www.tx2-localhost
> Accept: text/html, */*
> User-Agent: Mozilla/3.0 (compatible; Indy Library)
> Proxy-Authorization: Basic Og==
>
> ----------061006144341906
> Content-Disposition: form-data; name="TYPE"
>
> LAST_ALERTS
> ----------061006144341906
> Content-Disposition: form-data; name="FILEAGE"
>
> 180
>
> ----------061006144341906
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira