You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by "William A. Rowe, Jr." <wr...@rowe-clan.net> on 2003/08/08 22:38:46 UTC
CUJ Sept issue APR Article
Hey Team, thanks to Mr. Bloom's writing habit we have some very interesting
publicity that will hit the newsstands in the coming days.
http://www.cuj.com/documents/s=8188/cuj0309bloom/
The C Users Journal September issue is dedicated to Open Source. Ryan's
article discusses APR from the simplest baseline, implementing cat and rm
as examples of how trivial development is using APR. He goes on, of course,
to describe APR and touches on the design approach (pools, etc.)
Enjoy, and please remember that rbb isn't subscribed, cc him if you want
to share your comments.
Bill
RE: unix/thread_mutex.c 1.19 and --enable-pool-debug=yes core dump
Posted by Sander Striker <st...@apache.org>.
> From: William A. Rowe, Jr. [mailto:wrowe@rowe-clan.net]
> Sent: Friday, August 22, 2003 8:20 AM
> My thought is that the existing code is resilient enough without any
> cas logic; but I have been waiting for others to take the time to prove
> that to themselves before we back down to simple inc/dec/assignment.
>
> My other question - why is it necessary to explicitly use nested locks
> within the pool code? Do we have a nesting issue that could be fixed
> and return to more optimal 'default' thread mutexes for pools?
This only for the pools debug code. From apr_pool_create_ex_debug:
/* No matter what the creation flags say, always create
* a lock. Without it integrity_check and apr_pool_num_bytes
* blow up (because they traverse pools child lists that
* possibly belong to another thread, in combination with
* the pool having no lock). However, this might actually
* hide problems like creating a child pool of a pool
* belonging to another thread.
*/
Sander
RE: unix/thread_mutex.c 1.19 and --enable-pool-debug=yes core
dump
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
My thought is that the existing code is resilient enough without any
cas logic; but I have been waiting for others to take the time to prove
that to themselves before we back down to simple inc/dec/assignment.
My other question - why is it necessary to explicitly use nested locks
within the pool code? Do we have a nesting issue that could be fixed
and return to more optimal 'default' thread mutexes for pools?
Bill
At 11:13 PM 8/21/2003, Sander Striker wrote:
>> From: Blair Zajac [mailto:blair@orcaware.com]
>> Sent: Thursday, August 21, 2003 11:49 PM
>
>Hi Blair!
>
>> I tracked my core dump down to the following combination.
>>
>> Using apr CVS HEAD with --enable-pool-debug=yes and with thread_mutex
>> 1.19 or greater causes the coredumps in apr/test/testall I reported in
>> my messages with the subject "testall core dump with apr HEAD". Backing
>> thread_mutex.c down to 1.18 has no problems with -enable-pool-debug=yes.
>
>The introduction of atomics in the thread_mutex code. Let's see...
>
>Program received signal SIGSEGV, Segmentation fault.
>[Switching to Thread 1074592640 (LWP 16810)]
>0x4003356e in apr_atomic_cas (mem=0x80c10b8, with=0, cmp=0) at apr_atomic.c:166
>166 apr_thread_mutex_t *lock = hash_mutex[ATOMIC_HASH(mem)];
>(gdb) bt
>#0 0x4003356e in apr_atomic_cas (mem=0x80c10b8, with=0, cmp=0)
> at apr_atomic.c:166
>#1 0x40019944 in _r_debug ()
> from /export/home1/blair/Code/Apache/2.0/h/srclib/apr/.libs/libapr-0.so.0
>#2 0x4002fe0a in apr_thread_mutex_lock (mutex=0x0) at thread_mutex.c:129
>#3 0x4003219a in apr_pool_create_ex_debug (newpool=0xbfffed40,
> parent=0x80c1048, abort_fn=0, allocator=0x0,
> file_line=0x40035a83 "start.c:96") at apr_pools.c:1540
>#4 0x4002e512 in apr_initialize () at start.c:96
>
>Right. The core dump happens when apr_atomic_init() hasn't been called yet.
>
>>>From apr_initialize() [misc/unix/start.c]
>--
> if ((status = apr_pool_initialize()) != APR_SUCCESS)
> return status;
>
> if (apr_pool_create(&pool, NULL) != APR_SUCCESS) {
> return APR_ENOPOOL;
> }
>
> apr_pool_tag(pool, "apr_initialize");
>
> if ((status = apr_atomic_init(pool)) != APR_SUCCESS) {
> return status;
>--
>
>Since pools depend on apr_thread_mutex-es, the atomics change might
>have introduced a circular dependency. And we're getting bit by
>it in the debug code. Stepping through the code will positively
>point out where the thread mutexes are first used before
>apr_atomic_init() is called. If you get around to that before I do
>please post ;) :).
>
>> I'm on RedHat 9 and I've used the RedHat gcc 3.2.2 and my own gcc 3.3.1
>> and they both have the same behavior.
>>
>> I haven't received any response to my previous messages, so I'd
>> appreciate some advice in getting a response from the APR developers.
>> It seems that other messages from APR newbies that ask questions
>> implying that they did no homework on reading APR documentation
>> gets more of a response then somebody who's got a decent amount of
>> feedback to a core dump from an APR supplied program, which isn't
>> obviously then operator error. So the question is, what's missing
>> in my reports to get a response.
>
>Nothing. Sorry for the late reply.
>
>> I do like to contribute, so anything to make this easier for me
>> would be appreciated.
>>
>> I'm hoping I've narrowed this problem down to a small enough
>> changeset where somebody that's familar with the locking code
>> can make quick progress on this bug.
>>
>> BTW, I'm using --enable-pool-debug=yes with APR as part of
>> Subversion.
>
>
>Sander
Re: unix/thread_mutex.c 1.19 and --enable-pool-debug=yes core dump
Posted by Jacob Lewallen <jl...@cs.ucr.edu>.
Blair Zajac wrote:
> I tracked my core dump down to the following combination.
I also have had problems with pool debugging and multithreaded APR
applications. I've attached an e-mail I sent with a few months ago
detailing my problems. It'd be nice to get these things cleaned up, they
seem a tad bit related. I looked over the changes made from 1.18 to 1.19
and there seems to have been some changes to how the mutex ownership was
manipulated, which, again, is something that the pools debugging code
doesn't handle correctly itself at the moment. It appears this patch
I've attached may not handle everything, but it's at least a starting
point and goes to the heart of the problem that I've experienced with
pool debugging. Perhaps someone more familar with thread_mutex.c and
take a look and see if Blair's problem is related?
>
> BTW, I'm using --enable-pool-debug=yes with APR as part of
> Subversion.
>
This is in reference to my previous e-mail regarding Solaris and an
EOF when reading from apr_proc_create'd processes. I narrowed this down
to another problem I had experienced with apr_thread_exit.
There are a few places where we're doing things to pools from threads
who don't "own" the given pool, thereby causing the
apr_pool_check_integrity to fail horrible when debugging. For example,
in apr_thread_exit (we call apr_pool_destroy on the pool we created in
apr_pool_create) Another example is in apr_proc_create, where we call
apr_pool_cleanup_kill to remove some apr_file_t cleanups.
I checked the docs and at one point there seemed to have been the desire
to include an apr_pool_owner_set (just grep, it's only mentioned once in
the source) So here's a patch that implements the apr_pool_owner_set
function and places calls to fix the above mentioned problems. The
reason for the EOF was such that the child process would simply abort
(with no messages because stdout and such had been redirected) Things
worked fine under Linux because of the way thread ID's are assigned, I
would guess. Fire away.
--
jacob lewallen
jlewalle@cs.ucr.edu
RE: unix/thread_mutex.c 1.19 and --enable-pool-debug=yes core dump
Posted by Sander Striker <st...@apache.org>.
> From: Blair Zajac [mailto:blair@orcaware.com]
> Sent: Thursday, August 21, 2003 11:49 PM
Hi Blair!
> I tracked my core dump down to the following combination.
>
> Using apr CVS HEAD with --enable-pool-debug=yes and with thread_mutex
> 1.19 or greater causes the coredumps in apr/test/testall I reported in
> my messages with the subject "testall core dump with apr HEAD". Backing
> thread_mutex.c down to 1.18 has no problems with -enable-pool-debug=yes.
The introduction of atomics in the thread_mutex code. Let's see...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1074592640 (LWP 16810)]
0x4003356e in apr_atomic_cas (mem=0x80c10b8, with=0, cmp=0) at apr_atomic.c:166
166 apr_thread_mutex_t *lock = hash_mutex[ATOMIC_HASH(mem)];
(gdb) bt
#0 0x4003356e in apr_atomic_cas (mem=0x80c10b8, with=0, cmp=0)
at apr_atomic.c:166
#1 0x40019944 in _r_debug ()
from /export/home1/blair/Code/Apache/2.0/h/srclib/apr/.libs/libapr-0.so.0
#2 0x4002fe0a in apr_thread_mutex_lock (mutex=0x0) at thread_mutex.c:129
#3 0x4003219a in apr_pool_create_ex_debug (newpool=0xbfffed40,
parent=0x80c1048, abort_fn=0, allocator=0x0,
file_line=0x40035a83 "start.c:96") at apr_pools.c:1540
#4 0x4002e512 in apr_initialize () at start.c:96
Right. The core dump happens when apr_atomic_init() hasn't been called yet.
>From apr_initialize() [misc/unix/start.c]
--
if ((status = apr_pool_initialize()) != APR_SUCCESS)
return status;
if (apr_pool_create(&pool, NULL) != APR_SUCCESS) {
return APR_ENOPOOL;
}
apr_pool_tag(pool, "apr_initialize");
if ((status = apr_atomic_init(pool)) != APR_SUCCESS) {
return status;
--
Since pools depend on apr_thread_mutex-es, the atomics change might
have introduced a circular dependency. And we're getting bit by
it in the debug code. Stepping through the code will positively
point out where the thread mutexes are first used before
apr_atomic_init() is called. If you get around to that before I do
please post ;) :).
> I'm on RedHat 9 and I've used the RedHat gcc 3.2.2 and my own gcc 3.3.1
> and they both have the same behavior.
>
> I haven't received any response to my previous messages, so I'd
> appreciate some advice in getting a response from the APR developers.
> It seems that other messages from APR newbies that ask questions
> implying that they did no homework on reading APR documentation
> gets more of a response then somebody who's got a decent amount of
> feedback to a core dump from an APR supplied program, which isn't
> obviously then operator error. So the question is, what's missing
> in my reports to get a response.
Nothing. Sorry for the late reply.
> I do like to contribute, so anything to make this easier for me
> would be appreciated.
>
> I'm hoping I've narrowed this problem down to a small enough
> changeset where somebody that's familar with the locking code
> can make quick progress on this bug.
>
> BTW, I'm using --enable-pool-debug=yes with APR as part of
> Subversion.
Sander
unix/thread_mutex.c 1.19 and --enable-pool-debug=yes core dump
Posted by Blair Zajac <bl...@orcaware.com>.
I tracked my core dump down to the following combination.
Using apr CVS HEAD with --enable-pool-debug=yes and with thread_mutex
1.19 or greater causes the coredumps in apr/test/testall I reported in
my messages with the subject "testall core dump with apr HEAD". Backing
thread_mutex.c down to 1.18 has no problems with -enable-pool-debug=yes.
I'm on RedHat 9 and I've used the RedHat gcc 3.2.2 and my own gcc 3.3.1
and they both have the same behavior.
I haven't received any response to my previous messages, so I'd
appreciate some advice in getting a response from the APR developers.
It seems that other messages from APR newbies that ask questions
implying that they did no homework on reading APR documentation
gets more of a response then somebody who's got a decent amount of
feedback to a core dump from an APR supplied program, which isn't
obviously then operator error. So the question is, what's missing
in my reports to get a response.
I do like to contribute, so anything to make this easier for me
would be appreciated.
I'm hoping I've narrowed this problem down to a small enough
changeset where somebody that's familar with the locking code
can make quick progress on this bug.
BTW, I'm using --enable-pool-debug=yes with APR as part of
Subversion.
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
Re: CUJ Sept issue APR Article
Posted by Ben Laurie <be...@algroup.co.uk>.
rbb@rkbloom.net wrote:
> I should also mention that cat and rm are now the basis of my new
> porttools project, which aims to write the standard Unix tools using APR.
> This is meant to replace cygwin for people who want the tools, but as
> native apps. I haven't done much work on it, but I keep hoping to have
> more time.
Neat idea!
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html http://www.thebunker.net/
"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff
Re: CUJ Sept issue APR Article
Posted by rb...@rkbloom.net.
I should also mention that cat and rm are now the basis of my new
porttools project, which aims to write the standard Unix tools using APR.
This is meant to replace cygwin for people who want the tools, but as
native apps. I haven't done much work on it, but I keep hoping to have
more time.
Ryan
On Fri, 8 Aug 2003, William A. Rowe, Jr. wrote:
> Hey Team, thanks to Mr. Bloom's writing habit we have some very interesting
> publicity that will hit the newsstands in the coming days.
>
> http://www.cuj.com/documents/s=8188/cuj0309bloom/
>
> The C Users Journal September issue is dedicated to Open Source. Ryan's
> article discusses APR from the simplest baseline, implementing cat and rm
> as examples of how trivial development is using APR. He goes on, of course,
> to describe APR and touches on the design approach (pools, etc.)
>
> Enjoy, and please remember that rbb isn't subscribed, cc him if you want
> to share your comments.
>
> Bill
>
>