You are viewing a plain text version of this content. The canonical link for it is here.
Posted to rivet-dev@tcl.apache.org by Massimo Manghi <ma...@biol.unipr.it> on 2006/07/17 18:57:55 UTC

apache child process crashes

Hi everybody, I'm coping with a nasty problem for which
maybe Rivet has to be charged.

Apache children crash while executing rivet code in a quite
unpredictable way. I mean that displaying the same page or
sending the same data of a form causes the error to occur
randomly, but with a fairly high chance to happen.
It is not related to a specific script. The part of the backtrace
that pertains the Tcl stuff is

#0  0x4104183b in raise () from /lib/tls/libc.so.6
#1  0x41042fa2 in abort () from /lib/tls/libc.so.6
#2  0x407b4c4b in Rivet_ParseExecFile () from 
/usr/lib/apache/1.3/mod_rivet.so
#3  0x4093af70 in Tcl_PanicVA () from /usr/lib/libtcl8.4.so.0
#4  0x4093b006 in Tcl_Panic () from /usr/lib/libtcl8.4.so.0
#5  0x40949312 in Tcl_GetMemoryInfo () from /usr/lib/libtcl8.4.so.0
#6  0x40948d18 in TclpFree () from /usr/lib/libtcl8.4.so.0
#7  0x408ee12f in Tcl_Free () from /usr/lib/libtcl8.4.so.0
#8  0x40935124 in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
#9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1
#10 0x40934fad in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
#11 0x40934da8 in Tcl_DeleteNamespace () from /usr/lib/libtcl8.4.so.0
#12 0x40934eab in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
#13 0x40934da8 in Tcl_DeleteNamespace () from /usr/lib/libtcl8.4.so.0
#14 0x40937155 in Tcl_NamespaceObjCmd () from /usr/lib/libtcl8.4.so.0
#15 0x409369a3 in Tcl_NamespaceObjCmd () from /usr/lib/libtcl8.4.so.0
#16 0x408e96a4 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
#17 0x409109c8 in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
#18 0x4090ff0b in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
#19 0x40943147 in TclObjInterpProc () from /usr/lib/libtcl8.4.so.0
#20 0x408e96a4 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
#21 0x409109c8 in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
#22 0x4090ff0b in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
#23 0x408ea8f6 in Tcl_EvalObjEx () from /usr/lib/libtcl8.4.so.0
#24 0x407b44b0 in Rivet_ParseExecFile () from 
/usr/lib/apache/1.3/mod_rivet.so

and the error log entries are

/var/log/apache/error.log:541:[Mon Jul 17 16:29:49 2006] [crit] Critical 
error in request: /~manghi/webdev/template.rvt?func=login
/var/log/apache/error.log:542:[Mon Jul 17 16:29:49 2006] [crit] alloc: 
invalid block: bogus bogus %p84733c0 a0 8\n
/var/log/apache/error.log:543:[Mon Jul 17 16:29:49 2006] [notice] child 
pid 32379 exit signal Aborted (6)

(I'm using apache 1.3 and rivet 0.5.0-3 with the session and
dio packages taken from the svn repository, with the add-ons
Arnulf Widemann  made for Mysql)

-- 
-- Massimo Manghi 

-- Dipartimento di Biologia Evolutiva e Funzionale
-- Università degli Studi di Parma
-- Parco Area delle Scienze 11A - 43100 Parma


---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by David Welton <da...@apache.org>.
> I will try then. David, is there any facility for writing lines in
> the apache error log? Just to facilitate the synchronization between
> the session and the apache log files.

I think a simple puts stderr will go to the error logs...

-- 
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by Massimo Manghi <ma...@biol.unipr.it>.
A natural consenquence of what I found yesterday was
that I had to try to delete the 'class' definition from the
namespace. Adding this line

::itcl::delete class $fun(function_class)

seems to prevent the process from crashing and
therefore might be a workaround. I'm testing
the code right now refreshing the page every 20 seconds,
and I haven't got any crash in 30
minutes, whereas prior to that crashes were
quite easy to get. It doesn't provide for an explanation but
at least one knows where the thin ice is and can avoid
to walk over it.

David Welton wrote:

>> Here is what I found (I hope I don't get too specific)
>> The line that screws everything up is in a method where various
>> parts of a page are dynamically generated. The 'modules'
>> that generate these sections of the page are implemented through
>> classes which in turn are sourced by a method invoked by the
>> main template.
>>
>> I verified that the 'sourcing' of these classes' files
>> breaks the rivet process (although the code 'per se'
>> is functioning!).
>
>
> So if you don't source them, no crash?


no crash. Sourcing is enough for getting a crash, I don't
need to instantiate or execute methods.

>
>
> Hrm...  is it possible to replicate the crash in a 'clean' environment
> that is as simple as possible?
>
no, I tried but I failed. I created a class that sources in another class
definition, makes an object, runs some code and the cleans the object
up (not the class). It seems that same case, but it doesn't crash.
I can provide the code, its about 40 lines, empty lines included

>>
>
> It should be possible to build the '1.0' branch on debian stable, I 
> think.
>
I will try then. David, is there any facility for writing lines in
the apache error log? Just to facilitate the synchronization between
the session and the apache log files.

-- 
-- Massimo Manghi 

-- Dipartimento di Biologia Evolutiva e Funzionale
-- Università degli Studi di Parma
-- Parco Area delle Scienze 11A - 43100 Parma


---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by David Welton <da...@apache.org>.
> Here is what I found (I hope I don't get too specific)
> The line that screws everything up is in a method where various
> parts of a page are dynamically generated. The 'modules'
> that generate these sections of the page are implemented through
> classes which in turn are sourced by a method invoked by the
> main template.
>
> I verified that the 'sourcing' of these classes' files
> breaks the rivet process (although the code 'per se'
> is functioning!).

So if you don't source them, no crash?

> Let me show you the code snippet.
> Commenting out the whole code block makes the child
> processes robust as they can get. Commenting the instantiation,
> execution and destruction lines breaks apache again.
>
>     source        [file join $::request::site_root
> ${fun(filename)}.tcl]
>     set module  [$fun(function_class) #auto $dioObject $sessionStatus]
>     $module      displayModule
>     $module      destroy
>
> It seems that Itcl classes namespace definitions are not properly cleaned up
> in this case. I'm trying to see what happens if I force the class definition
> to happen in the ::request namespace, not in my class' namespace.

Hrm...  is it possible to replicate the crash in a 'clean' environment
that is as simple as possible?

> >> Should the following line imply that Itcl stuff is involved
> >> (but I could not figure out where...)
> >>
> >> #9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1
> >
> >
> > Could well be...  What OS and version are you running on?  Debian stable?
>
> yes, precisely.
>
> >
> >> > One thing you could do to help track this down is check this out of
> >> > subversion and use it to create your Apache module:
> >> >
> >> > https://svn.apache.org/repos/asf/tcl/rivet/branches/1_0
> >> >
> >>
> >> I'm afraid that my knowledge of svn goes as far as its commands
> >> have the same switches and syntax as cvs. Using cvs syntax for
> >> svn would do the work in this case? And Rivet doesn't compile
> >> on Debian using the autotools because of a macro problem...(I know
> >> you are laden by this task too, I'm sorry, I did not intend
> >> to overload you with too many things to think of)
> >
> >
> > You want to do:
> >
> > svn checkout https......   local-rivet-directory-name
> >
> > Is this Debian unstable/testing then?
>
>
> no, it's stable. The problem is similar on the laptop running
> 'testing' but the messages in the log complain about the
> child exiting upon a segfault.

It should be possible to build the '1.0' branch on debian stable, I think.

-- 
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by Massimo Manghi <ma...@biol.unipr.it>.
David Welton wrote:

> On 7/18/06, Massimo Manghi <ma...@biol.unipr.it> wrote:
>
>
> Some more things to try:
>
> Launch apache with the -X flag to put it in single process mode.
>
I obtained the backtrace running in single process mode.

> Keep eliminating code to see if you can get it to crash with less and
> less code (and thus an easier test case for me to replicate).
>
I'm getting close to the target. First of all: I checked
the code in the template is executed from the beginning
up to the last line. The last line in the template "puts" a message
in the process logfile. This message appears in the log of
every crashed process.

Here is what I found (I hope I don't get too specific)
The line that screws everything up is in a method where various
parts of a page are dynamically generated. The 'modules'
that generate these sections of the page are implemented through
classes which in turn are sourced by a method invoked by the
main template.

I verified that the 'sourcing' of these classes' files
breaks the rivet process (although the code 'per se'
is functioning!).

Let me show you the code snippet.
Commenting out the whole code block makes the child
processes robust as they can get. Commenting the instantiation,
execution and destruction lines breaks apache again.

    source        [file join $::request::site_root 
${fun(filename)}.tcl]        
    set module  [$fun(function_class) #auto $dioObject $sessionStatus]
    $module      displayModule
    $module      destroy
 
It seems that Itcl classes namespace definitions are not properly cleaned up
in this case. I'm trying to see what happens if I force the class definition
to happen in the ::request namespace, not in my class' namespace.


>> Should the following line imply that Itcl stuff is involved
>> (but I could not figure out where...)
>>
>> #9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1
>
>
> Could well be...  What OS and version are you running on?  Debian stable?

yes, precisely.

>
>> > One thing you could do to help track this down is check this out of
>> > subversion and use it to create your Apache module:
>> >
>> > https://svn.apache.org/repos/asf/tcl/rivet/branches/1_0
>> >
>>
>> I'm afraid that my knowledge of svn goes as far as its commands
>> have the same switches and syntax as cvs. Using cvs syntax for
>> svn would do the work in this case? And Rivet doesn't compile
>> on Debian using the autotools because of a macro problem...(I know
>> you are laden by this task too, I'm sorry, I did not intend
>> to overload you with too many things to think of)
>
>
> You want to do:
>
> svn checkout https......   local-rivet-directory-name
>
> Is this Debian unstable/testing then?


no, it's stable. The problem is similar on the laptop running
'testing' but the messages in the log complain about the
child exiting upon a segfault.

-- 
-- Massimo Manghi 

-- Dipartimento di Biologia Evolutiva e Funzionale
-- Università degli Studi di Parma
-- Parco Area delle Scienze 11A - 43100 Parma


---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by David Welton <da...@apache.org>.
On 7/18/06, Massimo Manghi <ma...@biol.unipr.it> wrote:

> > Odd... looks like something to do with removing the namespace,
> > although that could be a false lead.

> I tried to isolate every possible point where Itcl class
> deletion is invoked but found no offending object destruction.
> I put various marker messages in
> the template: I can only say that the template is parsed
> to the end and the code in it gets executed, but the child
> exit procedure gets never invoked by the server process.
> Again: might it be a sign of an error in the ::request
> namespace deletion?

Some more things to try:

Launch apache with the -X flag to put it in single process mode.

Keep eliminating code to see if you can get it to crash with less and
less code (and thus an easier test case for me to replicate).

> Should the following line imply that Itcl stuff is involved
> (but I could not figure out where...)
>
> #9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1

Could well be...  What OS and version are you running on?  Debian stable?

> > One thing you could do to help track this down is check this out of
> > subversion and use it to create your Apache module:
> >
> > https://svn.apache.org/repos/asf/tcl/rivet/branches/1_0
> >
>
> I'm afraid that my knowledge of svn goes as far as its commands
> have the same switches and syntax as cvs. Using cvs syntax for
> svn would do the work in this case? And Rivet doesn't compile
> on Debian using the autotools because of a macro problem...(I know
> you are laden by this task too, I'm sorry, I did not intend
> to overload you with too many things to think of)

You want to do:

svn checkout https......   local-rivet-directory-name

Is this Debian unstable/testing then?

> > That's the last 'good' version of Rivet before I added Eckhard
> > Lehmann's Apache 2.0 stuff.  Looks like the recent DIO/other stuff
> > changes haven't been back ported though...  More work I don't have
> > time for:-(
> >
>
> Does the Rivet package for Debian 'unstable' include this 2.0 things?

Nope.

-- 
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by Massimo Manghi <ma...@biol.unipr.it>.
Quoting David Welton <da...@apache.org>:


> Odd... looks like something to do with removing the namespace,
> although that could be a false lead.

I tried to isolate every possible point where Itcl class
deletion is invoked but found no offending object destruction.
I put various marker messages in
the template: I can only say that the template is parsed
to the end and the code in it gets executed, but the child
exit procedure gets never invoked by the server process.
Again: might it be a sign of an error in the ::request
namespace deletion?

Should the following line imply that Itcl stuff is involved
(but I could not figure out where...)

#9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1


>
> One thing you could do to help track this down is check this out of
> subversion and use it to create your Apache module:
>
> https://svn.apache.org/repos/asf/tcl/rivet/branches/1_0
>

I'm afraid that my knowledge of svn goes as far as its commands
have the same switches and syntax as cvs. Using cvs syntax for
svn would do the work in this case? And Rivet doesn't compile
on Debian using the autotools because of a macro problem...(I know
you are laden by this task too, I'm sorry, I did not intend
to overload you with too many things to think of)

> That's the last 'good' version of Rivet before I added Eckhard
> Lehmann's Apache 2.0 stuff.  Looks like the recent DIO/other stuff
> changes haven't been back ported though...  More work I don't have
> time for:-(
>

Does the Rivet package for Debian 'unstable' include this 2.0 things?

-- Massimo Manghi


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
----------------------------------------------------------------
Imp3 - Webmail del Dipartimento di Biologia Evolutiva e Funzionale

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org


Re: apache child process crashes

Posted by David Welton <da...@apache.org>.
> Apache children crash while executing rivet code in a quite
> unpredictable way. I mean that displaying the same page or
> sending the same data of a form causes the error to occur
> randomly, but with a fairly high chance to happen.
> It is not related to a specific script. The part of the backtrace
> that pertains the Tcl stuff is

> #0  0x4104183b in raise () from /lib/tls/libc.so.6
> #1  0x41042fa2 in abort () from /lib/tls/libc.so.6
> #2  0x407b4c4b in Rivet_ParseExecFile () from
> /usr/lib/apache/1.3/mod_rivet.so
> #3  0x4093af70 in Tcl_PanicVA () from /usr/lib/libtcl8.4.so.0
> #4  0x4093b006 in Tcl_Panic () from /usr/lib/libtcl8.4.so.0
> #5  0x40949312 in Tcl_GetMemoryInfo () from /usr/lib/libtcl8.4.so.0
> #6  0x40948d18 in TclpFree () from /usr/lib/libtcl8.4.so.0
> #7  0x408ee12f in Tcl_Free () from /usr/lib/libtcl8.4.so.0
> #8  0x40935124 in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
> #9  0x422a4bb3 in Itcl_DeleteClass () from /usr/lib/libitcl3.2.so.1
> #10 0x40934fad in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
> #11 0x40934da8 in Tcl_DeleteNamespace () from /usr/lib/libtcl8.4.so.0
> #12 0x40934eab in TclTeardownNamespace () from /usr/lib/libtcl8.4.so.0
> #13 0x40934da8 in Tcl_DeleteNamespace () from /usr/lib/libtcl8.4.so.0
> #14 0x40937155 in Tcl_NamespaceObjCmd () from /usr/lib/libtcl8.4.so.0
> #15 0x409369a3 in Tcl_NamespaceObjCmd () from /usr/lib/libtcl8.4.so.0
> #16 0x408e96a4 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
> #17 0x409109c8 in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
> #18 0x4090ff0b in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
> #19 0x40943147 in TclObjInterpProc () from /usr/lib/libtcl8.4.so.0
> #20 0x408e96a4 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
> #21 0x409109c8 in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
> #22 0x4090ff0b in TclCompEvalObj () from /usr/lib/libtcl8.4.so.0
> #23 0x408ea8f6 in Tcl_EvalObjEx () from /usr/lib/libtcl8.4.so.0
> #24 0x407b44b0 in Rivet_ParseExecFile () from
> /usr/lib/apache/1.3/mod_rivet.so
>
> and the error log entries are
>
> /var/log/apache/error.log:541:[Mon Jul 17 16:29:49 2006] [crit] Critical
> error in request: /~manghi/webdev/template.rvt?func=login
> /var/log/apache/error.log:542:[Mon Jul 17 16:29:49 2006] [crit] alloc:
> invalid block: bogus bogus %p84733c0 a0 8\n
> /var/log/apache/error.log:543:[Mon Jul 17 16:29:49 2006] [notice] child
> pid 32379 exit signal Aborted (6)
>
> (I'm using apache 1.3 and rivet 0.5.0-3 with the session and
> dio packages taken from the svn repository, with the add-ons
> Arnulf Widemann  made for Mysql)

Odd... looks like something to do with removing the namespace,
although that could be a false lead.

One thing you could do to help track this down is check this out of
subversion and use it to create your Apache module:

https://svn.apache.org/repos/asf/tcl/rivet/branches/1_0

That's the last 'good' version of Rivet before I added Eckhard
Lehmann's Apache 2.0 stuff.  Looks like the recent DIO/other stuff
changes haven't been back ported though...  More work I don't have
time for:-(

-- 
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org