You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Jeff Trawick <tr...@attglobal.net> on 2004/01/20 16:21:24 UTC
[1.3 PATCH] a different take on forensics
See http://www.apache.org/~trawick/exception_hook_13.html
There is a small patch to Apache 1.3 required to make the sample modules
work. This is analogous to the toys using the Apache 2.1 exception hook
which are described at http://www.apache.org/~trawick/exception_hook.html.
Re: [1.3 PATCH] a different take on forensics
Posted by gr...@apache.org.
Jeff Trawick wrote:
> See http://www.apache.org/~trawick/exception_hook_13.html
>
> There is a small patch to Apache 1.3 required to make the sample modules
> work. This is analogous to the toys using the Apache 2.1 exception hook
> which are described at http://www.apache.org/~trawick/exception_hook.html.
++1 in concept (haven't tested it yet). mod_whatkilledus_13 is similar in
concept to the save_input.patch we used to run with on daedalus which was a big
help in resolving bugs early in 2.0. This is nicer though because you don't
have to dig the information out of a coredump :-)
Greg
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Bill Stoddard wrote:
> What is the purpose of the geteuid() call in run_fatal_exception_hook
> and when might it return 0?
geteuid() returns 0 when running with root privileges
the purpose of not running the hook as root is a nod to the possibility
that the hook could be hijacked (dunno how since we'd have to be in the
parent to be running with root privileges but still...) and figuring
that it is safer to have some limit on how much damage could be done
feature-wise the limitation doesn't seem to be harmful, as I'm not aware
of any meaningful number of 1.3 crashes in the parent
Re: [1.3 PATCH] a different take on forensics
Posted by Bill Stoddard <bi...@wstoddard.com>.
Jeff Trawick wrote:
> Ben Laurie wrote:
>
>> Jeff Trawick wrote:
>>
>>> See http://www.apache.org/~trawick/exception_hook_13.html
>>
>>
>> You should make the logged strings safe, like mod_log_forensic does,
>> and I think the format should be compatible (which means no space
>> after the colon).
>
>
> Thanks for taking a look!
What is the purpose of the geteuid() call in run_fatal_exception_hook and when might it return 0?
Bill
Re: [1.3 PATCH] a different take on forensics
Posted by Bill Stoddard <bi...@wstoddard.com>.
> if the code is always built into the server (with/without
> EnableExceptionHook setting), perhaps we should add the #define
> AP_ENABLE_EXCEPTION_HOOK in ap_config.h for platforms where we know it
> will work
+1
Bill
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Mads Toftum wrote:
> On Thu, Jan 22, 2004 at 06:48:46AM -0500, Jeff Trawick wrote:
>
>>anyone else lurking?
>
>
> A very nice idea - seen with my #apache hat on, it'd be really
> nice to get this when people turn up complaining about segfaults.
> With my old admin and security paranoid hats on, it would
> certainly be nice too - but runtime configurable would certainly
> be nice.
perhaps EnableExceptionHook {On|Off}?
but this brings up the question: on what platforms does it actually
work? probably all unix-ish platforms
if it is only enabled via special build flag, Jeff doesn't have to think
about where it works :)
if the code is always built into the server (with/without
EnableExceptionHook setting), perhaps we should add the #define
AP_ENABLE_EXCEPTION_HOOK in ap_config.h for platforms where we know it
will work and the use of it will not cause additional problems? users
can always define AP_ENABLE_EXCEPTION_HOOK manually if they wish to test
suitability on their box...
Re: [1.3 PATCH] a different take on forensics
Posted by Mads Toftum <ma...@toftum.dk>.
On Thu, Jan 22, 2004 at 06:48:46AM -0500, Jeff Trawick wrote:
> anyone else lurking?
A very nice idea - seen with my #apache hat on, it'd be really
nice to get this when people turn up complaining about segfaults.
With my old admin and security paranoid hats on, it would
certainly be nice too - but runtime configurable would certainly
be nice.
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Ben Laurie wrote:
> Jeff Trawick wrote:
>
>> Ben Laurie wrote:
>>
>>> You should make the logged strings safe, like mod_log_forensic does,
>>> and I think the format should be compatible (which means no space
>>> after the colon).
>>
>>
>>
>> Thanks for taking a look!
>>
>> I removed the space after the colon, but at present am not too eager
>> about the escaping of the strings.
>
>
> Why not?
laziness^H^H^H^H^H^H^H^Hshort term priorities (for now I'd rather spend
some time in the short term adding footprints to quickly see how far we
got (type checker, fixups, handler, logger?))... as implied before, no
qualms here on escaping if I see the expected set of users expand beyond
me or people I give direct assistance to
> Anyway +1 (untested) for the core patch.
anyone else lurking?
Re: [1.3 PATCH] a different take on forensics
Posted by Ben Laurie <be...@algroup.co.uk>.
Jeff Trawick wrote:
> Ben Laurie wrote:
>
>> Jeff Trawick wrote:
>>
>>> See http://www.apache.org/~trawick/exception_hook_13.html
>>
>>
>> You should make the logged strings safe, like mod_log_forensic does,
>> and I think the format should be compatible (which means no space
>> after the colon).
>
>
> Thanks for taking a look!
>
> I removed the space after the colon, but at present am not too eager
> about the escaping of the strings.
Why not?
> If for some reason the module would
> be seriously considered for committing to cvs, that is a different
> ballgame, of course. Same if for some other reason I get the idea that
> folks would actually use it.
>
> What I'd really like to see happen is have the patch to core committed.
> If people are truly interested in having the module(s) put in
> experimental, that is cool too though I'm not sure I think that's such a
> good idea until/unless it is clear that it will be used by more than a
> handful of people (bloat).
Right - I think they would be great inclusions, though.
Anyway +1 (untested) for the core patch.
> BTW, I changed the storage allocation of the "hook" record to use pconf
> and to clear the list via a cleanup registered on pconf
> (http://www.apache.org/~trawick/13_fatal_exception_patch).
Sounds good.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html http://www.thebunker.net/
"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Ben Laurie wrote:
> Jeff Trawick wrote:
>
>> See http://www.apache.org/~trawick/exception_hook_13.html
>
> You should make the logged strings safe, like mod_log_forensic does, and
> I think the format should be compatible (which means no space after the
> colon).
Thanks for taking a look!
I removed the space after the colon, but at present am not too eager
about the escaping of the strings. If for some reason the module would
be seriously considered for committing to cvs, that is a different
ballgame, of course. Same if for some other reason I get the idea that
folks would actually use it.
What I'd really like to see happen is have the patch to core committed.
If people are truly interested in having the module(s) put in
experimental, that is cool too though I'm not sure I think that's such a
good idea until/unless it is clear that it will be used by more than a
handful of people (bloat).
BTW, I changed the storage allocation of the "hook" record to use pconf
and to clear the list via a cleanup registered on pconf
(http://www.apache.org/~trawick/13_fatal_exception_patch).
Re: [1.3 PATCH] a different take on forensics
Posted by Ben Laurie <be...@algroup.co.uk>.
Jeff Trawick wrote:
> See http://www.apache.org/~trawick/exception_hook_13.html
>
> There is a small patch to Apache 1.3 required to make the sample modules
> work. This is analogous to the toys using the Apache 2.1 exception hook
> which are described at http://www.apache.org/~trawick/exception_hook.html.
You should make the logged strings safe, like mod_log_forensic does, and
I think the format should be compatible (which means no space after the
colon).
Nice idea.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html http://www.thebunker.net/
"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Mads Toftum wrote:
> On Sat, Jan 24, 2004 at 12:58:40PM -0500, Jeff Trawick wrote:
>
>>groan :( the problem was in the proof-of-concept modules; they neglected
>>to add a null entry at the end of the command table, as shown below:
>>
>
> Much better now. Once I remembered EnableExceptionHook it was smooth
> sailing. I'm still +1 after testing - my only reservation is that
> BTFile and WKUFile could probably use more verbose naming.
perhaps these should be
BacktraceLog filename
WhatKilledUsLog filename
also:
the config logic in the modules needs to support serverroot-relative names for
the log files
as pointed out by Ben, the logging of the request line and the request header
fields as performed by mod_whatkilledus needs to escape any unprintables
Re: [1.3 PATCH] a different take on forensics
Posted by Mads Toftum <ma...@toftum.dk>.
On Sat, Jan 24, 2004 at 12:58:40PM -0500, Jeff Trawick wrote:
> groan :( the problem was in the proof-of-concept modules; they neglected
> to add a null entry at the end of the command table, as shown below:
>
Much better now. Once I remembered EnableExceptionHook it was smooth
sailing. I'm still +1 after testing - my only reservation is that
BTFile and WKUFile could probably use more verbose naming.
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Mads Toftum wrote:
> On Sat, Jan 24, 2004 at 07:23:24AM -0500, Jeff Trawick wrote:
>
>>url above now points to patch which implements these changes
>>vote added to apache-1.3/STATUS
>>
>
> Segfaults here (linux-2.4.23_pre8 - --enable-module=most
> --enable-shared=max)
>
> With mod_backtrace_13 enabled:
>
> #0 0x401291fc in strcasecmp () from /lib/libc.so.6
> #1 0x08055240 in ap_find_command (name=0x8099a74 "ClearModuleList", cmds=0x40262020) at http_config.c:964
> #2 0x08055294 in ap_find_command_in_modules (cmd_name=0x8099a74 "ClearModuleList", mod=0xbfffc570) at http_config.c:978
> #3 0x0805540d in ap_handle_command (parms=0xbfffe650, config=0x8095c9c, l=0xbfffc5b0 "ClearModuleList") at http_config.c:1021
> #4 0x0805551d in ap_srm_command_loop (parms=0xbfffe650, config=0x8095c9c) at http_config.c:1044
> #5 0x08055d11 in ap_process_resource_config (s=0x809575c, fname=0x8095e3c "/opt/run/trawick-1.3/conf/httpd.conf", p=0x8095724, ptemp=0x8099744) at http_config.c:1332
> #6 0x08056639 in ap_read_config (p=0x8095724, ptemp=0x8099744, confname=0x808e220 "conf/httpd.conf") at http_config.c:1616
> #7 0x080620a6 in main (argc=1, argv=0xbfffe7a4) at http_main.c:5528
> #8 0x400c7d8c in __libc_start_main () from /lib/libc.so.6
>
> the trace looks similar with mod_whatkilledus_13. Without either of
> them it looks fine.
groan :( the problem was in the proof-of-concept modules; they neglected
to add a null entry at the end of the command table, as shown below:
static const command_rec command_table[] = {
{
"WKUFile", cmd_file, NULL, RSRC_CONF, TAKE1,
"the filename of the mod_whatkilledus_13 logfile"
}
, <-
{ <-
NULL <-
} <-
};
sample modules updated, core patch unchanged, and of course THANKS :)
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Mads Toftum wrote:
> On Sat, Jan 24, 2004 at 07:23:24AM -0500, Jeff Trawick wrote:
>
>>url above now points to patch which implements these changes
>>vote added to apache-1.3/STATUS
>>
>
> Segfaults here (linux-2.4.23_pre8 - --enable-module=most
> --enable-shared=max)
>
> With mod_backtrace_13 enabled:
>
> #0 0x401291fc in strcasecmp () from /lib/libc.so.6
> #1 0x08055240 in ap_find_command (name=0x8099a74 "ClearModuleList", cmds=0x40262020) at http_config.c:964
> #2 0x08055294 in ap_find_command_in_modules (cmd_name=0x8099a74 "ClearModuleList", mod=0xbfffc570) at http_config.c:978
> #3 0x0805540d in ap_handle_command (parms=0xbfffe650, config=0x8095c9c, l=0xbfffc5b0 "ClearModuleList") at http_config.c:1021
> #4 0x0805551d in ap_srm_command_loop (parms=0xbfffe650, config=0x8095c9c) at http_config.c:1044
> #5 0x08055d11 in ap_process_resource_config (s=0x809575c, fname=0x8095e3c "/opt/run/trawick-1.3/conf/httpd.conf", p=0x8095724, ptemp=0x8099744) at http_config.c:1332
> #6 0x08056639 in ap_read_config (p=0x8095724, ptemp=0x8099744, confname=0x808e220 "conf/httpd.conf") at http_config.c:1616
> #7 0x080620a6 in main (argc=1, argv=0xbfffe7a4) at http_main.c:5528
> #8 0x400c7d8c in __libc_start_main () from /lib/libc.so.6
>
> the trace looks similar with mod_whatkilledus_13. Without either of
> them it looks fine.
the good news is that I can recreate it (Fedora Core 1)... no fix yet
though...
Re: [1.3 PATCH] a different take on forensics
Posted by Mads Toftum <ma...@toftum.dk>.
On Sat, Jan 24, 2004 at 07:23:24AM -0500, Jeff Trawick wrote:
>
> url above now points to patch which implements these changes
> vote added to apache-1.3/STATUS
>
Segfaults here (linux-2.4.23_pre8 - --enable-module=most
--enable-shared=max)
With mod_backtrace_13 enabled:
#0 0x401291fc in strcasecmp () from /lib/libc.so.6
#1 0x08055240 in ap_find_command (name=0x8099a74 "ClearModuleList", cmds=0x40262020) at http_config.c:964
#2 0x08055294 in ap_find_command_in_modules (cmd_name=0x8099a74 "ClearModuleList", mod=0xbfffc570) at http_config.c:978
#3 0x0805540d in ap_handle_command (parms=0xbfffe650, config=0x8095c9c, l=0xbfffc5b0 "ClearModuleList") at http_config.c:1021
#4 0x0805551d in ap_srm_command_loop (parms=0xbfffe650, config=0x8095c9c) at http_config.c:1044
#5 0x08055d11 in ap_process_resource_config (s=0x809575c, fname=0x8095e3c "/opt/run/trawick-1.3/conf/httpd.conf", p=0x8095724, ptemp=0x8099744) at http_config.c:1332
#6 0x08056639 in ap_read_config (p=0x8095724, ptemp=0x8099744, confname=0x808e220 "conf/httpd.conf") at http_config.c:1616
#7 0x080620a6 in main (argc=1, argv=0xbfffe7a4) at http_main.c:5528
#8 0x400c7d8c in __libc_start_main () from /lib/libc.so.6
the trace looks similar with mod_whatkilledus_13. Without either of
them it looks fine.
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Jeff Trawick wrote:
> Jeff Trawick wrote:
>
>> See http://www.apache.org/~trawick/exception_hook_13.html
>>
> 1) automatically define AP_ENABLE_EXCEPTION_HOOK in ap_config.h for
> platforms that have been tested (AIX, Linux, Solaris already and I'll
> test HP-UX before long)
>
> 2) user can still enable it for other platforms by defining
> AP_ENABLE_EXCEPTION_HOOK for the configure step
>
> 3) when AP_ENABLE_EXCEPTION_HOOK is defined, a new directive
> EnableExceptionHook {on|off}
> which defaults to "off" must be used to tell sig_coredump() to call such
> hooks.
url above now points to patch which implements these changes
vote added to apache-1.3/STATUS
thanks everyone for your comments so far!
Re: [1.3 PATCH] a different take on forensics
Posted by Mads Toftum <ma...@toftum.dk>.
On Fri, Jan 23, 2004 at 09:45:46AM -0500, Jeff Trawick wrote:
> (trying to make progress without spinning wheels in mud :) )
>
> Here is a change I propose for the core patch to address some of the
> concerns (when to enable it):
>
> basically the patch listed above, but:
>
> 1) automatically define AP_ENABLE_EXCEPTION_HOOK in ap_config.h for
> platforms that have been tested (AIX, Linux, Solaris already and I'll
> test HP-UX before long)
>
> 2) user can still enable it for other platforms by defining
> AP_ENABLE_EXCEPTION_HOOK for the configure step
>
> 3) when AP_ENABLE_EXCEPTION_HOOK is defined, a new directive
> EnableExceptionHook {on|off}
> which defaults to "off" must be used to tell sig_coredump() to call such
> hooks.
>
> Any objections?
No objections at all.
vh
Mads Toftum
--
`Darn it, who spiked my coffee with water?!' - lwall
Re: [1.3 PATCH] a different take on forensics
Posted by Jeff Trawick <tr...@attglobal.net>.
Jeff Trawick wrote:
> See http://www.apache.org/~trawick/exception_hook_13.html
>
> There is a small patch to Apache 1.3 required to make the sample modules
> work. This is analogous to the toys using the Apache 2.1 exception hook
> which are described at http://www.apache.org/~trawick/exception_hook.html.
(trying to make progress without spinning wheels in mud :) )
Here is a change I propose for the core patch to address some of the
concerns (when to enable it):
basically the patch listed above, but:
1) automatically define AP_ENABLE_EXCEPTION_HOOK in ap_config.h for
platforms that have been tested (AIX, Linux, Solaris already and I'll
test HP-UX before long)
2) user can still enable it for other platforms by defining
AP_ENABLE_EXCEPTION_HOOK for the configure step
3) when AP_ENABLE_EXCEPTION_HOOK is defined, a new directive
EnableExceptionHook {on|off}
which defaults to "off" must be used to tell sig_coredump() to call such
hooks.
Any objections?