You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Stas Bekman <st...@stason.org> on 2002/03/12 11:46:17 UTC
auto-solib-add and threaded httpd don't play together
Hi Daniel,
Remember you've helped me at the mod_perl dev list with my gdb starting
very slowly?
http://marc.theaimsgroup.com/?l=apache-modperl-dev&m=100679267518055&w=2
Your auto-solib-add tip works great with preforked httpd.
Now, I'm trying to debug mod_perl over threaded httpd (worker mpm) and
the two simply won't play together. If I set auto-solib-add to 0, gdb
won't work with threaded httpd. Have you seen this problem before?
Any ideas how to tell gdb not to resolve so's automatically and still
have threads working? without auto-solib-add=0, I'm back to the painful
1+ minute startups :(
Thanks!
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Stas Bekman <st...@stason.org>.
Daniel Jacobowitz wrote:
> On Mon, Mar 18, 2002 at 03:21:53PM +0800, Stas Bekman wrote:
>
>>Daniel Jacobowitz wrote:
>>
>>>On Sun, Mar 17, 2002 at 03:22:04AM +0800, Stas Bekman wrote:
>>>
>>>
>>>>Thanks Daniel,
>>>>
>>>>I guess first of all we are after being able to use the debugger with
>>>>threads at all, next working on the speed. If I knew how to make gdb
>>>>automatically switch to the thread which I want that would be a first
>>>>big step (i don't know which thread number do I want). What I see
>>>>currently is this: (I'm running on dual-proc smp kernel) I've a bp set
>>>>and I press 'continue' and at random I get my thread which runs the code
>>>>but most of the time not. I suppose this has to do with threads
>>>>scheduling and its interaction with the debugger.
>>>>
>>>>
>>>Well, the same code is being called in the thread you want and another?
>>>How do you know which thread you want?
>>>
>>>GDB supports both thread-specific breakpoints and conditional
>>>breakpoints. From the manual, you can do things like:
>>> (gdb) break main thread 1 if a == 2
>>>
>>>So if you can express the condition that says "this is my thread", you
>>>can do
>>> (gdb) break apr_poll if thread->magic == MY_MAGIC
>>>or whatever.
>>>
>>Interesting. Has anybody an idea what would be the condition in httpd
>>threads? looking through worker.c I cannot see any variables I can have
>>in the scope of the running thread. How do I refer to the thread, while
>>I'm inside of it so I can get get to its local storage?
>>
>
> The same way the thread figures out what its local storage is
> (remember, you can make function calls from GDB...).
of course :) I was asking what tls data can I use for indentification of
the desired thread, strictly a modperl specific question.
I think that the best way to work with threads is to have only one
breakpoint, so once I reach the point I want to start debugging from I
think I should disable all the temp breakpoints. And then the debugger
will never start switching between threads. Currently if I still have
apr_poll bp set, while I'm already inside my working thread, and step
through it with step/next once in a while I get switched to other
threads which call apr_poll. Cannot use tbreak here, since I need to
call them a few times. So I guess I'll try to use disable/delete instead.
Indeed this startup script's version seems to work perfectly with
threads without any workarounds:
file /home/stas/httpd/worker/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
define myrun
tbreak main
break ap_run_pre_config
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf \
-DONE_PROCESS -DNO_DETATCH -DAPACHE2 -DPERL_USEITHREADS
set auto-solib-add 0
continue
end
define gopoll
b apr_poll
continue
continue
end
define mybp
# load Apache/Filter.so
sharedlibrary Filter
b mpxs_Apache__Filter_print
# no longer needed and they just make debugging harder under threads
disable 2
disable 3
continue
end
myrun
gopoll
mybp
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Daniel Jacobowitz <dm...@andrew.cmu.edu>.
On Mon, Mar 18, 2002 at 03:21:53PM +0800, Stas Bekman wrote:
> Daniel Jacobowitz wrote:
> >On Sun, Mar 17, 2002 at 03:22:04AM +0800, Stas Bekman wrote:
> >
> >>Thanks Daniel,
> >>
> >>I guess first of all we are after being able to use the debugger with
> >>threads at all, next working on the speed. If I knew how to make gdb
> >>automatically switch to the thread which I want that would be a first
> >>big step (i don't know which thread number do I want). What I see
> >>currently is this: (I'm running on dual-proc smp kernel) I've a bp set
> >>and I press 'continue' and at random I get my thread which runs the code
> >>but most of the time not. I suppose this has to do with threads
> >>scheduling and its interaction with the debugger.
> >>
> >
> >Well, the same code is being called in the thread you want and another?
> >How do you know which thread you want?
> >
> >GDB supports both thread-specific breakpoints and conditional
> >breakpoints. From the manual, you can do things like:
> > (gdb) break main thread 1 if a == 2
> >
> >So if you can express the condition that says "this is my thread", you
> >can do
> > (gdb) break apr_poll if thread->magic == MY_MAGIC
> >or whatever.
>
> Interesting. Has anybody an idea what would be the condition in httpd
> threads? looking through worker.c I cannot see any variables I can have
> in the scope of the running thread. How do I refer to the thread, while
> I'm inside of it so I can get get to its local storage?
The same way the thread figures out what its local storage is
(remember, you can make function calls from GDB...).
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Stas Bekman <st...@stason.org>.
Daniel Jacobowitz wrote:
> On Sun, Mar 17, 2002 at 03:22:04AM +0800, Stas Bekman wrote:
>
>>Thanks Daniel,
>>
>>I guess first of all we are after being able to use the debugger with
>>threads at all, next working on the speed. If I knew how to make gdb
>>automatically switch to the thread which I want that would be a first
>>big step (i don't know which thread number do I want). What I see
>>currently is this: (I'm running on dual-proc smp kernel) I've a bp set
>>and I press 'continue' and at random I get my thread which runs the code
>>but most of the time not. I suppose this has to do with threads
>>scheduling and its interaction with the debugger.
>>
>
> Well, the same code is being called in the thread you want and another?
> How do you know which thread you want?
>
> GDB supports both thread-specific breakpoints and conditional
> breakpoints. From the manual, you can do things like:
> (gdb) break main thread 1 if a == 2
>
> So if you can express the condition that says "this is my thread", you
> can do
> (gdb) break apr_poll if thread->magic == MY_MAGIC
> or whatever.
Interesting. Has anybody an idea what would be the condition in httpd
threads? looking through worker.c I cannot see any variables I can have
in the scope of the running thread. How do I refer to the thread, while
I'm inside of it so I can get get to its local storage?
>>define bring_me_there
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>> continue
>>end
>>
>
> I recommend looking up the "ignore" command, if you want to skip a
> breakpoint N times.
Thanks Daniel, that works:
# threads specific
define bring_me_there
# ignoring apr_poll (bp 3) 12 times
ignore 3 12
continue
end
I wish there was a syntax allowing to do it by name rather by number.
--
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Daniel Jacobowitz <dm...@andrew.cmu.edu>.
On Sun, Mar 17, 2002 at 03:22:04AM +0800, Stas Bekman wrote:
> Thanks Daniel,
>
> I guess first of all we are after being able to use the debugger with
> threads at all, next working on the speed. If I knew how to make gdb
> automatically switch to the thread which I want that would be a first
> big step (i don't know which thread number do I want). What I see
> currently is this: (I'm running on dual-proc smp kernel) I've a bp set
> and I press 'continue' and at random I get my thread which runs the code
> but most of the time not. I suppose this has to do with threads
> scheduling and its interaction with the debugger.
Well, the same code is being called in the thread you want and another?
How do you know which thread you want?
GDB supports both thread-specific breakpoints and conditional
breakpoints. From the manual, you can do things like:
(gdb) break main thread 1 if a == 2
So if you can express the condition that says "this is my thread", you
can do
(gdb) break apr_poll if thread->magic == MY_MAGIC
or whatever.
> define bring_me_there
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> continue
> end
I recommend looking up the "ignore" command, if you want to skip a
breakpoint N times.
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Stas Bekman <st...@stason.org>.
Daniel Jacobowitz wrote:
> On Tue, Mar 12, 2002 at 06:46:17PM +0800, Stas Bekman wrote:
>
>>Hi Daniel,
>>
>>Remember you've helped me at the mod_perl dev list with my gdb starting
>>very slowly?
>>http://marc.theaimsgroup.com/?l=apache-modperl-dev&m=100679267518055&w=2
>>Your auto-solib-add tip works great with preforked httpd.
>>
>>Now, I'm trying to debug mod_perl over threaded httpd (worker mpm) and
>>the two simply won't play together. If I set auto-solib-add to 0, gdb
>>won't work with threaded httpd. Have you seen this problem before?
>>
>>Any ideas how to tell gdb not to resolve so's automatically and still
>>have threads working? without auto-solib-add=0, I'm back to the painful
>>1+ minute startups :(
>>
>
> Threads: in shared libraries. Basically, auto-solib-add 0 had two
> effects: Don't stop all the time to load libraries, and don't notice
> the thread library getting loaded.
>
> There's not much we can do without improving the speed of thread
> debugging. I'm thinking about it, but it's really quite complicated...
Thanks Daniel,
I guess first of all we are after being able to use the debugger with
threads at all, next working on the speed. If I knew how to make gdb
automatically switch to the thread which I want that would be a first
big step (i don't know which thread number do I want). What I see
currently is this: (I'm running on dual-proc smp kernel) I've a bp set
and I press 'continue' and at random I get my thread which runs the code
but most of the time not. I suppose this has to do with threads
scheduling and its interaction with the debugger.
... time passes, me trying more combinations ...
On the other hand at this moment for some reason gdb always switches to
the right thread after I hit the apr_poll bp 14 times. And then it
happens, i get what I want:
Breakpoint 3, apr_poll (aprset=0x8187eb0, nsds=0xbf5ffaec, timeout=-1)
at poll.c:143
(gdb) cont
... 12 more times...
Breakpoint 3, apr_poll (aprset=0x8187eb0, nsds=0xbf5ffaec, timeout=-1)
at poll.c:143
(gdb) cont
[Switching to Thread 3076 (LWP 18425)]
Breakpoint 4, mpxs_Apache__Scoreboard_image (my_perl=0x81e7960,
Class=0x88fb814, pool=0x844c7e0) at
/home/stas/apache.org/mp-sb/xs/Apache/Scoreboard/Apache__Scoreboard.h:233
(gdb) info threads
* 5 Thread 3076 (LWP 18425) mpxs_Apache__Scoreboard_image
(my_perl=0x81e7960, Class=0x88fb814, pool=0x844c7e0) at
/home/stas/apache.org/mp-sb/xs/Apache/Scoreboard/Apache__Scoreboard.h:233
4 Thread 2051 (LWP 18424) 0x402dfd94 in poll () from /lib/libc.so.6
2 Thread 2049 (LWP 18422) 0x402dfd94 in poll () from /lib/libc.so.6
1 Thread 1024 (LWP 18374) 0x402267a2 in sigsuspend () from
/lib/libc.so.6
(gdb)
I don't know why this happens, because under forked httpd there is no 14
times apr_poll bp hits. I suppose that there is some other thread
running and hits the apr_poll bp. Why 14 times, I dunno.
So I've automated the process and now I have:
.debug
-----------
file /home/stas/httpd/worker/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
define myrun
tbreak main
break ap_run_pre_config
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf -DONE_PROCESS
-DNO_DETATCH -DAPACHE2
set auto-solib-add 0
continue
end
define gopoll
b apr_poll
continue
continue
end
define bring_me_there
continue
continue
continue
continue
continue
continue
continue
continue
continue
continue
continue
continue
continue
end
define mybp
sharedlibrary Scoreboard
b mpxs_Apache__Scoreboard_image
end
myrun
gopoll
mybp
bring_me_there
which I run as:
ddd -command=.debug & ; sleep 2 ; t/TEST -v -run apache/scoreboard &
and voila, I can debug my code :) takes about 10-15 secs to get me
there, which is not bad.
But I'm almost sure that this won't work when I really need it :) I
don't like this 14 times number, which makes no sense to me.
Hey, gozer you have an smp machine, can you reproduce the same? Use some
other bp for the mod_perl code, since scoreboard stuff wasn't committed yet.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Daniel Jacobowitz <dm...@andrew.cmu.edu>.
On Tue, Mar 12, 2002 at 06:46:17PM +0800, Stas Bekman wrote:
> Hi Daniel,
>
> Remember you've helped me at the mod_perl dev list with my gdb starting
> very slowly?
> http://marc.theaimsgroup.com/?l=apache-modperl-dev&m=100679267518055&w=2
> Your auto-solib-add tip works great with preforked httpd.
>
> Now, I'm trying to debug mod_perl over threaded httpd (worker mpm) and
> the two simply won't play together. If I set auto-solib-add to 0, gdb
> won't work with threaded httpd. Have you seen this problem before?
>
> Any ideas how to tell gdb not to resolve so's automatically and still
> have threads working? without auto-solib-add=0, I'm back to the painful
> 1+ minute startups :(
Threads: in shared libraries. Basically, auto-solib-add 0 had two
effects: Don't stop all the time to load libraries, and don't notice
the thread library getting loaded.
There's not much we can do without improving the speed of thread
debugging. I'm thinking about it, but it's really quite complicated...
--
Daniel Jacobowitz Carnegie Mellon University
MontaVista Software Debian GNU/Linux Developer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Stas Bekman <st...@stason.org>.
Doug MacEachern wrote:
> i started seeing the problem when i moved to a new laptop/kernel. the
> original trick did not work for me. i ended up with the below. i use
> 'myrun' to start and run 'apshared' when the pre_config breakpoint is hit.
> haven't tried with a threaded mpm yet though.
Thanks Doug,
Yes, that get you started, with the attached debug script I get to the
apr_poll
point under worker (threaded) mpm. Then I get a bunch of threads
starting with 4
and growing and I have no idea which of the threads is going to run the
test
script's response. If i just try to switch to various threads, sometimes
i get nothing,
something I get segfault, sometimes I'm lucky to guess the right thread,
but then i
get kicked back to the previous thread.
Got no help from httpd-dev list regarding this issue, any hints
regarding threads debug?
file /home/stas/httpd/worker/bin/httpd
handle SIGPIPE nostop
handle SIGPIPE pass
define myrun
tbreak main
break ap_run_pre_config
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf -DONE_PROCESS
-DNO_DETATCH -DAPACHE2
set auto-solib-add 0
continue
end
define gopoll
b apr_poll
continue
continue
end
myrun
gopoll
sharedlibrary Scoreboard
b mpxs_Apache__Scoreboard_image
the relevant section from httpd.conf is the automatic one:
<IfModule worker.c>
StartServers 1
MaxClients 1
MinSpareThreads 1
MaxSpareThreads 1
ThreadsPerChild 1
MaxRequestsPerChild 0
</IfModule>
this is with the stock perl5.7.3 and the latest httpd cvs
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: auto-solib-add and threaded httpd don't play together
Posted by Doug MacEachern <do...@covalent.net>.
i started seeing the problem when i moved to a new laptop/kernel. the
original trick did not work for me. i ended up with the below. i use
'myrun' to start and run 'apshared' when the pre_config breakpoint is hit.
haven't tried with a threaded mpm yet though.
handle SIGPIPE nostop
handle SIGPIPE pass
define apshared
shared apr
shared aprutil
shared mod_ssl.so
shared mod_proxy.*.so
# shared libphp4.so
continue
end
define myrun
tbreak main
break ap_run_pre_config
run -d `pwd`/t -f `pwd`/t/conf/httpd.conf -DONE_PROCESS -DNO_DETATCH -DAPACHE2
set auto-solib-add 0
continue
end
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org