[jira] Commented: (MODPYTHON-2) multiple/redundant interpreter creation

["Nicolas Lehuen (JIRA)" <ji...@apache.org>]

     [ http://nagoya.apache.org/jira/browse/MODPYTHON-2?page=comments#action_56673 ]
     
Nicolas Lehuen commented on MODPYTHON-2:
----------------------------------------

The patch for mod_python.c solves a problem with mod_python creating multiple interpreters with the same name, but there is stil a problem in the mod_python.apache module, since it reloads the same publisher many times.

Indeed, having a look at apache.py, I didn't see any locking done to prevent the same module from being reloaded multiple times concurrently. What we need is to put a lock in the import_module method, so that two threads cannot check the freshness of the publisher simultaneously. This can be done in a rather brutal way by calling imp.acquire_lock() at the beginning of import_module, put a try/finally block around the code of the function and call imp.release_lock() in the finally block : 

def import_module(module_name, autoreload=1, log=0, path=None):
    """
    Get the module to handle the request. If
    autoreload is on, then the module will be reloaded
    if it has changed since the last import.
    """

    imp.acquire_lock()
    try:
	# previous code of apache.import_module
    finally:
        imp.release_lock()

I tested this, and it works as expected. BTW, maybe it wouldn't work without the patch to mod_python.c, since I could suddenly see two interpreters with their own copy of the mod_python.apache module. But the two fixes are required for it to work properly.

Why did I wrote "in a rather brutal way" above ? Because this locking scheme is indeed brutal. It is a global lock, so each and every thread who want to get its handler must acquire it for a brief amount of time. The problem is that if a handler module is reloaded, it locks each and every other thread during its reloading, even if they should not be handled by the same module. A smarter way to lock would be to use a two-levels locking scheme, as I described in my caching recipe in the Python Cookbook. I can implement it there if it's ok for everybody.

I suggest we solve this promptly with a big lock and try to have a finer locking scheme later on.

> multiple/redundant interpreter creation
> ---------------------------------------
>
>          Key: MODPYTHON-2
>          URL: http://nagoya.apache.org/jira/browse/MODPYTHON-2
>      Project: mod_python
>         Type: Bug
>     Versions: 3.1.3
>  Environment: mod_python 3.1.3 + a threaded MPM (observed on Win32 and Mac OS X)
>     Reporter: Nicolas Lehuen
>     Assignee: Nicolas Lehuen
>      Fix For: 3.1.3
>  Attachments: mod_python.c.patch
>
> A small bug in mod_python.c allows the creation of many Python interpreters, where there should be only one. As a result, modules can be loaded multiple times (once per interpreter) and some higher level bugs can occur (beginning with higher memory usage). Graham Dumpleton found the bug in mod_python.c, and I completed the fix with a patch to apache.py (in the import_module) function.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://nagoya.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
   http://www.atlassian.com/software/jira