[jira] Closed: (MODPYTHON-77) The multiple interpreter concept of mod_python is broken for Python extension modules since Python 2.3

["Graham Dumpleton (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MODPYTHON-77?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Graham Dumpleton closed MODPYTHON-77.
-------------------------------------


> The multiple interpreter concept of mod_python is broken for Python extension modules since Python 2.3
> ------------------------------------------------------------------------------------------------------
>
>                 Key: MODPYTHON-77
>                 URL: https://issues.apache.org/jira/browse/MODPYTHON-77
>             Project: mod_python
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 3.1.4
>         Environment: Python >= 2.3
>            Reporter: Boyan Boyadjiev
>         Assigned To: Graham Dumpleton
>             Fix For: 3.3
>
>         Attachments: diff.txt, diff2.txt, diff3.txt, gil_test.c, gilstate.tar.gz, grahamd_20051105.tar.gz, mod_python.c, mod_python.c.diff, mod_python.h.diff, src.zip
>
>
> The multiple interpreter concept of mod_python is broken for Python extension modules since Python 2.3 because of the PEP 311 (Simplified Global Interpreter Lock Acquisition for Extensions):
> ...
> Limitations and Exclusions
>     This proposal identifies a solution for extension authors with
>     complex multi-threaded requirements, but that only require a
>     single "PyInterpreterState".  There is no attempt to cater for
>     extensions that require multiple interpreter states.  At the time
>     of writing, no extension has been identified that requires
>     multiple PyInterpreterStates, and indeed it is not clear if that
>     facility works correctly in Python itself.
> ...
> For mod_python this means, that complex Python extensions won't work any more with Python >= 2.3, because they are supposed to work only with the first interpreter state initialized for the current process (a problem we experienced). The first interpreter state is not used by mod_python after the python_init is called. 
> One solution, which works fine for me, is to save the first interpreter state into the "interpreters" dictionary in the function python_init (MAIN_INTERPRETER is used as a key):
> static int python_init(apr_pool_t *p, apr_pool_t *ptemp,
>                        apr_pool_t *plog, server_rec *s)
> {
>     ...
>     /* initialize global Python interpreter if necessary */
>     if (! Py_IsInitialized())
>     {
>         /* initialze the interpreter */
>         Py_Initialize();
> #ifdef WITH_THREAD
>         /* create and acquire the interpreter lock */
>         PyEval_InitThreads();
> #endif
>         /* create the obCallBack dictionary */
>         interpreters = PyDict_New();
>         if (! interpreters) {
>             ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s,
>                          "python_init: PyDict_New() failed! No more memory?");
>             exit(1);
>         }
>         {   
>             /*
>             Workaround PEP 311 - Simplified Global Interpreter Lock Acquisition for Extensions
>             BEGIN
>             */
>             PyObject *p = 0;
>             interpreterdata * idata = (interpreterdata *)malloc(sizeof(interpreterdata));
>             PyThreadState* currentThreadState = PyThreadState_Get();
>             PyInterpreterState *istate = currentThreadState->interp;
>             idata->istate = istate;
>             /* obcallback will be created on first use */
>             idata->obcallback = NULL;
>             p = PyCObject_FromVoidPtr((void ) idata, NULL); /*p->refcout = 1*/
>             PyDict_SetItemString(interpreters, MAIN_INTERPRETER, p); /*p->refcout = 2*/
>             Py_DECREF(p); /*p->refcout = 1*/
>             /*
>             END
>             Workaround PEP 311 - Simplified Global Interpreter Lock Acquisition for Extensions
>             */
>         }
>         /* Release the thread state because we will never use
>          * the main interpreter, only sub interpreters created later. */
>         PyThreadState_Swap(NULL);
> #ifdef WITH_THREAD
>         /* release the lock; now other threads can run */
>         PyEval_ReleaseLock();
> #endif
>     }
>     return OK;
> }
> Another change I've made in the attached file is to Py_DECREF(p) in get_interpreter, which will remove leaky reference to the PyCObject with the interpreter data. This was not a real problem, but now I see fewer leaks in BoundsChecker :-).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.