You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@logging.apache.org by "Tobias Frost (Jira)" <lo...@logging.apache.org> on 2021/12/11 07:37:00 UTC
[jira] [Comment Edited] (LOGCXX-322) Crashes on exit from multithreaded program using log4cxx
[ https://issues.apache.org/jira/browse/LOGCXX-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457553#comment-17457553 ]
Tobias Frost edited comment on LOGCXX-322 at 12/11/21, 7:36 AM:
----------------------------------------------------------------
At version 12.1, I'm seeing crashes (segfaults) in the multithreadtest test; according to the comment, this test is associated to this issue. (Posting here, as this issue is marked resolved.)
The crash does not happen everytime, but something like in average every 10 times the test is executed.
Here is a sample backtrace of a core dump generated by this:
{noformat}
gdb src/test/cpp/multithreadtest --core core
GNU gdb (Debian 10.1-2) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from src/test/cpp/multithreadtest...
[New LWP 665786]
[New LWP 665782]
[New LWP 665787]
[New LWP 665783]
[New LWP 665781]
[New LWP 665785]
[New LWP 665784]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./src/test/cpp/multithreadtest'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f2e3d547ac8 in __cxa_finalize (d=0x7f2e3db7c000) at cxa_finalize.c:41
41 cxa_finalize.c: No such file or directory.
[Current thread is 1 (Thread 0x7f2e3b2b9640 (LWP 665786))]
(gdb) bt
#0 0x00007f2e3d547ac8 in __cxa_finalize (d=0x7f2e3db7c000) at cxa_finalize.c:41
#1 0x00007f2e3da6a5d7 in __do_global_dtors_aux ()
from /home/tobi/workspace/deb/packages/log4cxx_edoras/log4cxx/obj-x86_64-linux-gnu/src/main/cpp/liblog4cxx.so.12
#2 0x00007f2e3b2b6560 in ?? ()
#3 0x00007f2e3db904b3 in _dl_fini () at dl-fini.c:139
Backtrace stopped: frame did not save the PC
(gdb)
{noformat}
was (Author: coldtobi):
At version 12.1, I'm seeing crashes in the multithreadtest test; according to the comment, this test is associated to this issue. (Posting here, as this issue is marked resolved.)
The crash does not happen everytime, but something like in average every 10 times the test is executed.
Here is a sample backtrace of a core dump generated by this:
{noformat}
gdb src/test/cpp/multithreadtest --core core
GNU gdb (Debian 10.1-2) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from src/test/cpp/multithreadtest...
[New LWP 665786]
[New LWP 665782]
[New LWP 665787]
[New LWP 665783]
[New LWP 665781]
[New LWP 665785]
[New LWP 665784]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./src/test/cpp/multithreadtest'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f2e3d547ac8 in __cxa_finalize (d=0x7f2e3db7c000) at cxa_finalize.c:41
41 cxa_finalize.c: No such file or directory.
[Current thread is 1 (Thread 0x7f2e3b2b9640 (LWP 665786))]
(gdb) bt
#0 0x00007f2e3d547ac8 in __cxa_finalize (d=0x7f2e3db7c000) at cxa_finalize.c:41
#1 0x00007f2e3da6a5d7 in __do_global_dtors_aux ()
from /home/tobi/workspace/deb/packages/log4cxx_edoras/log4cxx/obj-x86_64-linux-gnu/src/main/cpp/liblog4cxx.so.12
#2 0x00007f2e3b2b6560 in ?? ()
#3 0x00007f2e3db904b3 in _dl_fini () at dl-fini.c:139
Backtrace stopped: frame did not save the PC
(gdb)
{noformat}
> Crashes on exit from multithreaded program using log4cxx
> --------------------------------------------------------
>
> Key: LOGCXX-322
> URL: https://issues.apache.org/jira/browse/LOGCXX-322
> Project: Log4cxx
> Issue Type: Bug
> Affects Versions: 0.10.0
> Environment: Linux platform server product, compiled with g++ 4.3.x, running on multi-core IA32 and IA64 hardware
> Reporter: Ben Hockley
> Assignee: Curt Arnold
> Priority: Major
> Fix For: 0.12.0
>
> Attachments: 130-bugfix-LOGCXX-322.dpatch
>
>
> On exit(0); (attempt at graceful process exit) we commonly see segmentation faults in:
> * log4cxx::helpers::ObjectPtrBase()
> * log4cxx::LogManager::getLoggerLS()
> The cause
> =======
> We eventually traced this issue to the pervasive pattern of usage of singletons in the log4cxx code.
> The log4cxx library uses the "Meyers" singleton pattern, first popularised in Scott Meyers "Effective C++" (Item 47 in the 2nd edition of that book):
> Thing & getThingSingleton()
> {
> static Thing t;
> return t;
> }
> For many years, the above pattern was considered "best practice" for using Singletons in C++ - and was generally safe for most popular compiler implementations and most applications.
> Unfortunately, this recommendation is not actually guaranteed to be thread-safe for construction or destruction - something which is alluded to on Scott Meyers' own "Errata List for Effective C++, Second Edition"
> as described here: http://www.aristeia.com/BookErrata/ec++2e-errata.html
> The nub of the problem is that when a process calls "exit(0);" or similar, one thread will start running, in order, any user-registered "atexit" functions.
> Along with these, the compiler will execute the (conceptually similar) compiler-registered functions which invoke the destructors of any static file or function scope objects (also in order - the reverse order to static object construction).
> Unfortunately, other threads may still be in the process of running -and logging, perhaps using the static objects - during or after the execution of their destructors - thus opening the door to a bunch of potential SEGFAULTs.
> Solutions
> ======
> The solution is to introduce an (optional) new singleton lifetime management policy. Specifically, we need to make sure that any Singletons required for log4cxx logging to operate correctly (including, but not limited to, APRInitializer) are always available when required by any thread - including during process shutdown.
> Andrei Alexandrescu goes into a lot of detail about this C++ design problem in chapter 6 of "Modern C++ design" and proposes two elegant solutions - a "Phoenix Singleton" or SingletonHolder class.
> For complete thread-safety, we should also consider the startup/initialization race for each singleton (for that we'd need a multiple-locked singleton initialization pattern everywhere log4cxx creates a singleton).
> We will need to address the singletons in:
> * aprinitializer.cpp
> * level.cpp
> * logmanager.cpp
> and perhaps elsewhere too.
> An attempt at patching the above files to simply "leak" the crucial singleton objects seemed to prevent the crashes (but could, of course, cause unacceptable behaviour in certain embedded environments - or in environments using memory leak checkers like Boundshecker, Purify, Valgrind etc.)
> Curt Arnold suggested (on the mailing list) "Probably the best approach is to try to isolate the singleton pattern into a preprocessor macro and then allow the user to select what singleton pattern they'd like to use." - something which sounds like a very wise idea.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)