You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4cxx-user@logging.apache.org by Wyles Eric - ewyles <Er...@acxiom.com> on 2006/03/21 21:06:46 UTC
segmentation faults
Hello,
This is a lengthy explanation but please stay with me. I'm hoping
someone can help me track down some segmentation faults we have been
experiencing with log4cxx.
We are using one of the daily tar files from littletux.org. The file we
pulled was from sometime around Oct 25, 2005. Here is the version info
from the build.xml:
build.xml:<property name="version" value="0.9.8"/>
build.xml:<property name="apr.version" value="1.2.2"/>
build.xml:<property name="aprutil.version" value="1.2.2"/>
build.xml:<property name="cppunit.version" value="1.10.2"/>
We have made some slight modifications to the source to get
a SocketAppender up and running. We just had to implement the
LoggingEvent::write() method so that it would write to the output stream
using the format that our logging server expects.
This is a multi-threaded application running on RHEL 3 compiled under
gcc. Here is the output of gcc -v:
[ewyles@host ewyles]$ gcc -v
Reading specs from
/usr/lib/gcc-lib/i386-redhat-linux/3.2.3/specs
Configured with: ../configure --prefix=/usr
--mandir=/usr/share/man -- infodir=/usr/share/info
--enable-shared
--enable-threads=posix -- disable-checking --with-system-zlib
--enable-__cxa_atexit --host=i386- redhat-linux
Thread model: posix
gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-47)
In our code, we declare loggers in 3 different ways.
1 - We have a global logger in a file called 'logger.cpp':
log4cxx::LoggerPtr
logger(log4cxx::Logger::getLogger("<application_name>"));
We have a header file called 'logging.h' that is included by all
other source files in our project:
extern LoggerPtr logger;
2 - Each class in our project has a logger as a private member variable.
In the class header file:
class <class_name>
{
private:
static LoggerPtr logger;
}
And then in the .cpp file for the class we do this:
LoggerPtr
<class_name>::logger(Logger::getLogger("<application_name>.<class_name>"
));
3 - Each method of each class has a locally declared logger. In the .cpp
file for the class, each method looks like this:
short <class_name>::<method_name>()
{
static LoggerPtr
logger(Logger::getLogger("<application_name>.<class_name>.<method_name>"
));
//logic for the class goes here
}
The idea is that each method in our application will have a logger
called application.class.method and that we can set the logging level
for the application as a whole or fine tune it at the class and/or
method levels for debugging.
Now for the actual problem. Once in a while, our application crashes
with a segmentation fault. The stack trace looks like this:
Received signal 11 (Segmentation fault) from process [4]
Stack Trace:
0 :
<application_name>(_ZN14clsApplication18FatalSignalHandlerEiP7siginfoPv+
0x27e) [0x80fc050]
1 : /lib/tls/libpthread.so.0 [0xf93e50]
2 :
/usr/local/hive/i386/ags/lib/liblog4cxx.so(_ZN7log4cxx9Hierarchy9getLogg
erERKSsRKNS_7helpers10ObjectPtrTINS_3spi13LoggerFactoryEEE+0x251)
[0x608a09]
3 :
/usr/local/hive/i386/ags/lib/liblog4cxx.so(_ZN7log4cxx9Hierarchy9getLogg
erERKSs+0x39) [0x6087ab]
4 :
/usr/local/hive/i386/ags/lib/liblog4cxx.so(_ZN7log4cxx10LogManager9getLo
ggerERKSs+0x3c) [0x57528a]
5 :
/usr/local/hive/i386/ags/lib/liblog4cxx.so(_ZN7log4cxx6Logger9getLoggerE
PKc+0x53) [0x6063b9]
6-14 : <various classes specific to our application>
15 : /lib/tls/libpthread.so.0 [0xf8ddec]
16 : /lib/tls/libc.so.6(__clone+0x5a) [0xb9da2a]
There are 2 times we have seen this occur. Most frequently, it happens
when the application is attempting to start up. We create several thread
on startup and once in a while we will get this segmentation fault. The
other case where we have seen this is after the application has been
running for a while, but needs to create more threads. Once in a while
if we are creating many threads for some purpose, we will get this
segmentation fault. It could be coincidence, but it seems that in both
cases where we run into this error it is during times when we are
creating threads.
Does anyone have any ideas or suggestions? I have a gcc disassembly of
our
_ZN7log4cxx9Hierarchy9getLoggerERKSsRKNS_7helpers10ObjectPtrTINS_3spi13L
oggerFactoryEEE method if that would be useful to anyone.
Log4cxx is working great for us, we just need to figure out how to get
around this segmentation fault!
*************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be
legally privileged.
If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.
If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.
Thank you.
*************************************************************************
Re: segmentation faults
Posted by Curt Arnold <ca...@apache.org>.
On Mar 21, 2006, at 2:06 PM, Wyles Eric - ewyles wrote:
> Now for the actual problem. Once in a while, our application
> crashes with a segmentation fault. The stack trace looks like this:
>
> Received signal 11 (Segmentation fault) from process [4]
>
> Stack Trace:
>
> 0 : <application_name>
> (_ZN14clsApplication18FatalSignalHandlerEiP7siginfoPv+0x27e)
> [0x80fc050]
>
> 1 : /lib/tls/libpthread.so.0 [0xf93e50]
>
> 2 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx9Hierarchy9getLoggerERKSsRKNS_7helpers10ObjectPtrTINS_3spi
> 13LoggerFactoryEEE+0x251) [0x608a09]
>
> 3 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx9Hierarchy9getLoggerERKSs+0x39) [0x6087ab]
>
> 4 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx10LogManager9getLoggerERKSs+0x3c) [0x57528a]
>
> 5 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx6Logger9getLoggerEPKc+0x53) [0x6063b9]
>
> 6-14 : <various classes specific to our application>
>
> 15 : /lib/tls/libpthread.so.0 [0xf8ddec]
>
> 16 : /lib/tls/libc.so.6(__clone+0x5a) [0xb9da2a]
>
>
....
> Does anyone have any ideas or suggestions? I have a gcc disassembly
> of our
> _ZN7log4cxx9Hierarchy9getLoggerERKSsRKNS_7helpers10ObjectPtrTINS_3spi1
> 3LoggerFactoryEEE method if that would be useful to anyone.
>
>
Interesting stack trace. I would expect that pthread would be
eventually be called from the synchronize(mutex) line or the
corresponding destructor, but I would have expected to see
synchronized::synchronized() and apr_thread_mutex_lock or
synchronized::~synchronized() and apr_thread_mutex_unlock calls to
appear in the stack trace.
Since it occurs after start-up, it must be the getLogger() call in
the method body since they are called the first time through the
method, not the class level loggers.
Segmentation faults have been observed when APR calls precede the
call to apr_initialize in the APRInitalizator constructor or after
apr_terminate in the corresponding destructor, however I would not
think that would be the case here since it appears in the middle of
the apps lifetime.
Is there anything unusual about seeing /lib/tls/pthread? Are you
doing anything funky with thread local storage?
Re: segmentation faults
Posted by Curt Arnold <ca...@apache.org>.
On Mar 21, 2006, at 2:06 PM, Wyles Eric - ewyles wrote:
>
>
> Now for the actual problem. Once in a while, our application
> crashes with a segmentation fault. The stack trace looks like this:
>
>
>
> Received signal 11 (Segmentation fault) from process [4]
>
> Stack Trace:
>
> 0 : <application_name>
> (_ZN14clsApplication18FatalSignalHandlerEiP7siginfoPv+0x27e)
> [0x80fc050]
>
> 1 : /lib/tls/libpthread.so.0 [0xf93e50]
>
> 2 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx9Hierarchy9getLoggerERKSsRKNS_7helpers10ObjectPtrTINS_3spi
> 13LoggerFactoryEEE+0x251) [0x608a09]
>
> 3 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx9Hierarchy9getLoggerERKSs+0x39) [0x6087ab]
>
> 4 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx10LogManager9getLoggerERKSs+0x3c) [0x57528a]
>
> 5 : /usr/local/hive/i386/ags/lib/liblog4cxx.so
> (_ZN7log4cxx6Logger9getLoggerEPKc+0x53) [0x6063b9]
>
> 6-14 : <various classes specific to our application>
>
> 15 : /lib/tls/libpthread.so.0 [0xf8ddec]
>
> 16 : /lib/tls/libc.so.6(__clone+0x5a) [0xb9da2a]
>
>
Is the application-specific part of the calling stack consistent? Is
the method that calls getLogger() (level 6) consistent? If so, if
you comment out the method-local logger (which should result in the
class logger being used), do you still observe the segmentation
faults? If so, is there anything peculiar about that method.