You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mod_python-dev@quetz.apache.org by "Nicolas Lehuen (JIRA)" <ji...@apache.org> on 2004/12/14 22:58:57 UTC
[jira] Created: (MODPYTHON-2) multiple/redundant interpreter creation
multiple/redundant interpreter creation
---------------------------------------
Key: MODPYTHON-2
URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
Project: mod_python
Type: Bug
Versions: 3.1.3
Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
Reporter: Nicolas Lehuen
Assigned to: Nicolas Lehuen
Fix For: 3.1.3
Attachments: mod_python.c.patch
A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Work started: (MODPYTHON-2) multiple/redundant interpreter creation
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=history ]
Work on MODPYTHON-2 started by Nicolas Lehuen
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.1.3
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Updated: (MODPYTHON-2) multiple/redundant interpreter creation
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-2?page=history ]
Nicolas Lehuen updated MODPYTHON-2:
-----------------------------------
Fix Version: 3.1.4
(was: 3.1.3)
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://issues.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.1.4
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Updated: (MODPYTHON-2) multiple/redundant interpreter creation
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=history ]
Nicolas Lehuen updated MODPYTHON-2:
-----------------------------------
Attachment: mod_python.c.patch
Graham's patch.
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.1.3
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Closed: (MODPYTHON-2) multiple/redundant interpreter
creation
Posted by "Graham Dumpleton (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/MODPYTHON-2?page=all ]
Graham Dumpleton closed MODPYTHON-2:
------------------------------------
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://issues.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.2.7
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (MODPYTHON-2) multiple/redundant interpreter creation
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=history ]
Nicolas Lehuen resolved MODPYTHON-2:
------------------------------------
Resolution: Fixed
This should be OK now, but I need other people to compile and test the fix on platforms other than Win32.
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.1.3
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-2) multiple/redundant interpreter creation
Posted by "Nicolas Lehuen (JIRA)" <ji...@apache.org>.
[ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=comments#action_56673 ]
Nicolas Lehuen commented on MODPYTHON-2:
----------------------------------------
The patch for mod_python.c solves a problem with mod_python creating multiple interpreters with the same name, but there is stil a problem in the mod_python.apache module, since it reloads the same publisher many times.
Indeed, having a look at apache.py, I didn't see any locking done to prevent the same module from being reloaded multiple times concurrently. What we need is to put a lock in the import_module method, so that two threads cannot check the freshness of the publisher simultaneously. This can be done in a rather brutal way by calling imp.acquire_lock() at the beginning of import_module, put a try/finally block around the code of the function and call imp.release_lock() in the finally block :
def import_module(module_name, autoreload=1, log=0, path=None):
"""
Get the module to handle the request. If
autoreload is on, then the module will be reloaded
if it has changed since the last import.
"""
imp.acquire_lock()
try:
# previous code of apache.import_module
finally:
imp.release_lock()
I tested this, and it works as expected. BTW, maybe it wouldn't work without the patch to mod_python.c, since I could suddenly see two interpreters with their own copy of the mod_python.apache module. But the two fixes are required for it to work properly.
Why did I wrote "in a rather brutal way" above ? Because this locking scheme is indeed brutal. It is a global lock, so each and every thread who want to get its handler must acquire it for a brief amount of time. The problem is that if a handler module is reloaded, it locks each and every other thread during its reloading, even if they should not be handled by the same module. A smarter way to lock would be to use a two-levels locking scheme, as I described in my caching recipe in the Python Cookbook. I can implement it there if it's ok for everybody.
I suggest we solve this promptly with a big lock and try to have a finer locking scheme later on.
> multiple/redundant interpreter creation
> ---------------------------------------
>
> Key: MODPYTHON-2
> URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
> Project: mod_python
> Type: Bug
> Versions: 3.1.3
> Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
> Reporter: Nicolas Lehuen
> Assignee: Nicolas Lehuen
> Fix For: 3.1.3
> Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira