[jira] Commented: (MODPYTHON-109) Signal handler calling Py_Finalize() when child processes being killed.

["Graham Dumpleton (JIRA)" <ji...@apache.org>]

    [ http://issues.apache.org/jira/browse/MODPYTHON-109?page=comments#action_12440731 ] 
            
Graham Dumpleton commented on MODPYTHON-109:
--------------------------------------------

After more digging on this issue my opinion is now that we should just leave the register_cleanup() functions as is. That is don't allow Py_Finalize() to be called, but with the register_cleanup() functions being left in place. If it is shown somehow that the area is definitely a problem, we can revisit it then, but right now all I can say is that I can't set up test cases which show an issue.

The implications of not calling Py_Finalize() are that any functions registered with the Python atexit module will not be called and Python objects will not be destroyed. The atexit limitation isn't a big deal as only functions which were registered in the main_interpreter (after MODPYTHON-77 changes) were called anyway, and the register_cleanup() functions should be used instead.

More importantly, by not calling Py_Finalize() you don't start destroying all your interpreters while there are potentially user created threads still running. If one does call it and threads are still running, they can start crashing with Python exceptions. These wouldn't probably be noticed as stderr at that point probably isn't going to go anywhere, but when you catch exceptions and log them to the Apache error log you do see them. Thus, probably better to just let the threads run with a full environment up to the very point that the process it exited.

Consequently, am going to mark this as resolved.

> Signal handler calling Py_Finalize() when child processes being killed.
> -----------------------------------------------------------------------
>
>                 Key: MODPYTHON-109
>                 URL: http://issues.apache.org/jira/browse/MODPYTHON-109
>             Project: mod_python
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 3.2.7
>            Reporter: Graham Dumpleton
>         Assigned To: Graham Dumpleton
>             Fix For: 3.3
>
>         Attachments: MP109_20060308_grahamd_1.diff
>
>
> When Apache is killing off child processes as part of actions taken when the "apachectl restart" or "apachectl graceful" command is run, it sends a SIGTERM signal to the child processes. This causes a signal handler registered by Apache to be run. That signal handler destroys the main child memory pool. That memory pool has though a cleanup handler associated with it which was registered by mod_python. That cleanup handler ultimately calls Py_Finalize().
> The problem with this is that Py_Finalize() isn't safe to be called from a signal handler and if a handler is still executing or there is a separate thread running in the context of Python, a deadlock will likely ensue. This will prevent the child process exiting due to the SIGTERM causing the Apache parent process to send it a SIGKILL to really kill it.
> For a more detailed assessment of the problem and what lead to this conclusion see:
>   http://www.modpython.org/pipermail/mod_python/2006-January/019865.html
>   http://www.modpython.org/pipermail/mod_python/2006-January/019866.html
>   http://www.modpython.org/pipermail/mod_python/2006-January/019870.html
> To avoid the problem, the only choice seems to be avoid calling Py_Finalize() from the signal handler. The simplistic way of doing this seems to be to add:
>      if (child_init_pool)
>          return APR_SUCCESS;
> at the start of python_finalize(). This will mean that Py_Finalize() is never called in child processes. The full consequences of this is unknown, but on face value it would seem that it might be a reasonable thing to do. More research may be required.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira