Debugging Apache

[Bojan Smojver <bo...@binarix.com>]

I've encountered some segmentation faults while using mod_jk version
1.2.0 with Apache 1.3.20 (second and thereafter graceful restart tends
to be the problem).

After talking about debugging options with mod_jk developers, they've
pointed me to this list to ask for the best way to produce useful
debugging output from Apache when the segfault of the child process
happens and the core is not dumped. I've seen the undocumented -S option
for truss, but Linux uses strace. In all honesty, I'm not really
proficient in any of this, so that's probably the biggest challenge
you're facing ;-)

Platform is RedHat Linux 7.0 on Intel (AMD actually :-), Apache is
statically linked.

Bojan

Re: Debugging Apache

[jean-frederic clere <jf...@fujitsu-siemens.com>]

Marc Slemko wrote:
> 
> On Thu, 26 Jul 2001, Bojan Smojver wrote:
> 
> > scheme due to the fact I've been upgrading from older versions). This
> > directory is owned by root:root and only root user has write permissions
> > on it. Apache runs as apache:apache on my system and since child
> > processes are executed as apache, which is someone that doesn't have the
> > privilege to write here, nothing gets written, right? Could that be it?
> 
> The "normal" reason for Apache not dump core is that it normally
> starts as root then setuid()s to another user.  So most unixes, by
> default, do not allow a core dump since it has setuid() so there
> may still be "privileged" information in memory from pre-setuid()
> times.

That was my problem... With httpd-1.3. With httpd-2.0 there must be something
else because I do not get a core when kill -11 pid. (Does it make sense to kill
-11 pid when multithreaded httpd?).

> 
> If you set it up so it isn't started by root (normally requiring it be
> on a port >1024) then you can avoid this.
> 
> > When I was experimenting with compilers, Apache dumped the core in / and
> > that was the parent process (which runs as root) that did it. Not clear
> > why its core ended up in /...
> 
> Probably just because of where the problem happened in the process
> of starting up...

Re: Debugging Apache

[Bojan Smojver <bo...@binarix.com>]

Marc Slemko wrote:
> 
> On Thu, 26 Jul 2001, Bojan Smojver wrote:
> 
> > scheme due to the fact I've been upgrading from older versions). This
> > directory is owned by root:root and only root user has write permissions
> > on it. Apache runs as apache:apache on my system and since child
> > processes are executed as apache, which is someone that doesn't have the
> > privilege to write here, nothing gets written, right? Could that be it?
> 
> The "normal" reason for Apache not dump core is that it normally
> starts as root then setuid()s to another user.  So most unixes, by
> default, do not allow a core dump since it has setuid() so there
> may still be "privileged" information in memory from pre-setuid()
> times.
> 
> If you set it up so it isn't started by root (normally requiring it be
> on a port >1024) then you can avoid this.

Thanks. I'll try that.

> > When I was experimenting with compilers, Apache dumped the core in / and
> > that was the parent process (which runs as root) that did it. Not clear
> > why its core ended up in /...
> 
> Probably just because of where the problem happened in the process
> of starting up...

OK. Makes sense.

Bojan

Re: Debugging Apache

[Marc Slemko <ma...@znep.com>]

On Thu, 26 Jul 2001, Bojan Smojver wrote:

> scheme due to the fact I've been upgrading from older versions). This
> directory is owned by root:root and only root user has write permissions
> on it. Apache runs as apache:apache on my system and since child
> processes are executed as apache, which is someone that doesn't have the
> privilege to write here, nothing gets written, right? Could that be it?

The "normal" reason for Apache not dump core is that it normally
starts as root then setuid()s to another user.  So most unixes, by
default, do not allow a core dump since it has setuid() so there
may still be "privileged" information in memory from pre-setuid()
times.

If you set it up so it isn't started by root (normally requiring it be
on a port >1024) then you can avoid this.

> When I was experimenting with compilers, Apache dumped the core in / and
> that was the parent process (which runs as root) that did it. Not clear
> why its core ended up in /...

Probably just because of where the problem happened in the process
of starting up...

Re: Debugging Apache

[Bojan Smojver <bo...@binarix.com>]

Justin Erenkrantz wrote:
> 
> On Wed, Jul 25, 2001 at 06:54:52PM +1000, Bojan Smojver wrote:
> > I've encountered some segmentation faults while using mod_jk version
> > 1.2.0 with Apache 1.3.20 (second and thereafter graceful restart tends
> > to be the problem).
> >
> > After talking about debugging options with mod_jk developers, they've
> > pointed me to this list to ask for the best way to produce useful
> > debugging output from Apache when the segfault of the child process
> > happens and the core is not dumped. I've seen the undocumented -S option
> > for truss, but Linux uses strace. In all honesty, I'm not really
> > proficient in any of this, so that's probably the biggest challenge
> > you're facing ;-)
> >
> > Platform is RedHat Linux 7.0 on Intel (AMD actually :-), Apache is
> > statically linked.
> 
> As long as you have "ulimit -c" set high enough, it should dump the core
> on Linux (it certainly works fine with Apache 2.0).  Depending upon
> your shell, you can specify "ulimit -c 100m" which will allow 100MB
> core files to be created.  Most likely, it'll dump the cores in your
> apache-1.3 directory.

Aha, my Apache directory is /etc/httpd (ServerRoot=/etc/httpd, use old
scheme due to the fact I've been upgrading from older versions). This
directory is owned by root:root and only root user has write permissions
on it. Apache runs as apache:apache on my system and since child
processes are executed as apache, which is someone that doesn't have the
privilege to write here, nothing gets written, right? Could that be it?

When I was experimenting with compilers, Apache dumped the core in / and
that was the parent process (which runs as root) that did it. Not clear
why its core ended up in /...

BTW, my ulimit says it's 'unlimited', so I guess that shouldn't be an
issue.

> You know that seems to be an obvious omission from the debugging
> page.  Hmph.

If this in any way helps anyone else, I'll be more than happy.

Bojan

Re: Debugging Apache

[Justin Erenkrantz <je...@ebuilt.com>]

On Wed, Jul 25, 2001 at 06:54:52PM +1000, Bojan Smojver wrote:
> I've encountered some segmentation faults while using mod_jk version
> 1.2.0 with Apache 1.3.20 (second and thereafter graceful restart tends
> to be the problem).
> 
> After talking about debugging options with mod_jk developers, they've
> pointed me to this list to ask for the best way to produce useful
> debugging output from Apache when the segfault of the child process
> happens and the core is not dumped. I've seen the undocumented -S option
> for truss, but Linux uses strace. In all honesty, I'm not really
> proficient in any of this, so that's probably the biggest challenge
> you're facing ;-)
> 
> Platform is RedHat Linux 7.0 on Intel (AMD actually :-), Apache is
> statically linked.

As long as you have "ulimit -c" set high enough, it should dump the core
on Linux (it certainly works fine with Apache 2.0).  Depending upon 
your shell, you can specify "ulimit -c 100m" which will allow 100MB 
core files to be created.  Most likely, it'll dump the cores in your
apache-1.3 directory.  

You know that seems to be an obvious omission from the debugging
page.  Hmph.  

HTH.  -- justin

Re: Debugging Apache

[jean-frederic clere <jf...@fujitsu-siemens.com>]

Bojan Smojver wrote:
> 
> I've encountered some segmentation faults while using mod_jk version
> 1.2.0 with Apache 1.3.20 (second and thereafter graceful restart tends
> to be the problem).
> 
> After talking about debugging options with mod_jk developers, they've
> pointed me to this list to ask for the best way to produce useful
> debugging output from Apache when the segfault of the child process
> happens and the core is not dumped. I've seen the undocumented -S option
> for truss, but Linux uses strace. In all honesty, I'm not really
> proficient in any of this, so that's probably the biggest challenge
> you're facing ;-)
> 
> Platform is RedHat Linux 7.0 on Intel (AMD actually :-), Apache is
> statically linked.
> 
> Bojan

By the way I have noted kill -11 pid does not make a core on Linux, has it
something to do with the fact we are catching the signal in sig_coredump()?