You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Rainer Jung <ra...@kippdata.de> on 2021/04/23 09:26:02 UTC

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

When compiling and running make check for latest head of APU 1.6.x in 
preparation of httpd release testing, I observe very frequent hangs 
during testqueue and testpass.

It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.

The hangs seem not to depend on the exact APR version. I observed them 
with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.

I have not yet tested APU 1.7.x head.

I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they 
do not happen on RHEL 7 and 8.

Here are some stacks:

RHEL 6

#0  0x0000003906c0b44c in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x00007fe39c453cb5 in apr_thread_pool_tasks_cancel () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x00007fe39c453d45 in thread_pool_cleanup () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x00007fe39bff6c6e in apr_pool_destroy () from 
/shared/build/dev/httpd/install/apr/1.6.x/r1876940/rhel6.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x00007fe39c45492c in apr_thread_pool_destroy () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#5  0x000000000040853c in test_threadsafe ()
#6  0x000000000040610c in abts_run_test ()
#7  0x00000000004081e2 in testpass ()
#8  0x0000000000405675 in main ()

SLES 11

#1  0x00002b1e34104315 in apr_thread_pool_tasks_cancel () from 
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x00002b1e34104395 in thread_pool_cleanup () from 
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x00002b1e3455dcff in apr_pool_destroy () from 
/path/to/httpd/install/apr/1.6.x/1.6.5/sles11.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x00002b1e34104fdc in apr_thread_pool_destroy () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#5  0x000000000040b7e0 in test_queue_producer_consumer ()
#6  0x00000000004067ed in abts_run_test ()
#7  0x000000000040b611 in testqueue ()
#8  0x0000000000405d6d in main ()

SLES 12

#0  0x00007f39ed06c05f in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x00007f39ed8e6c2d in apr_thread_cond_wait (cond=<optimized out>, 
mutex=<optimized out>) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/locks/unix/thread_cond.c:68
#2  0x00007f39edd49cd1 in wait_on_busy_threads (owner=0x0, me=0x20c5630) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thread_pool_tasks_cancel (me=me@entry=0x20c5630, 
owner=owner@entry=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:793
#4  0x00007f39edd49fe5 in thread_pool_cleanup (me=0x20c5630) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:397
#5  0x00007f39ed8e7f9e in run_cleanups (cref=0x20dd3d8) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:2705
#6  apr_pool_destroy (pool=0x20dd368) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:978
#7  0x00007f39edd4995c in apr_thread_pool_destroy (me=<optimized out>) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:457
#8  0x000000000040b0e1 in test_queue_producer_consumer (tc=0x21de5d0, 
data=<optimized out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:120
#9  0x00000000004060ee in abts_run_test (ts=<optimized out>, f=0x40afe0 
<test_queue_producer_consumer>, value=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:172
#10 0x000000000040b231 in testqueue (suite=0x20c7160) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:131
#11 0x0000000000405a3d in main (argc=<optimized out>, argv=<optimized 
out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:416

SLES 15

#0  0x00007f41a1a2b89d in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x00007f41a22a576d in apr_thread_cond_wait (cond=<optimized out>, 
mutex=<optimized out>) at 
/path/to/sources/apr/1.6.x/r1876940/locks/unix/thread_cond.c:68
#2  0x00007f41a270d3d9 in wait_on_busy_threads (owner=0x0, me=0x1164870) 
at /path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thread_pool_tasks_cancel (me=me@entry=0x1164870, 
owner=owner@entry=0x0) at 
/path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:793
#4  0x00007f41a270d6a5 in thread_pool_cleanup (me=0x1164870) at 
/path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:397
#5  0x00007f41a22a685e in run_cleanups (cref=0x11705e8) at 
/path/to/sources/apr/1.6.x/r1876940/memory/unix/apr_pools.c:2666
#6  apr_pool_destroy (pool=0x1170578) at 
/shared/build/dev/httpd/sources/apr/1.6.x/r1876940/memory/unix/apr_pools.c:993
#7  0x00007f41a270cfec in apr_thread_pool_destroy (me=<optimized out>) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:457
#8  0x000000000040b099 in test_queue_producer_consumer (tc=0x12d3800, 
data=<optimized out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:120
#9  0x0000000000405fc8 in abts_run_test (ts=<optimized out>, f=0x40af90 
<test_queue_producer_consumer>, value=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:172
#10 0x000000000040b1f1 in testqueue (suite=0x11663b0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:131
#11 0x0000000000405934 in main (argc=<optimized out>, argv=<optimized 
out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:416


Solaris

-----------------  lwp# 1 / thread# 1  --------------------
  fe5c9590 lwp_park (0, 0, 0)
  fe5c35d8 cond_wait_queue (442e8, 3f118, 0, 0, ff3f48e8, 0) + 4c
  fe5c3a1c cond_wait_common (442e8, 3f118, 0, 0, 0, 0) + 294
  fe5c3b20 cond_wait (442e8, 3f118, 0, 1c00, fe5c7b30, ff3f6ad0) + 10
  fe5c3b5c pthread_cond_wait (442e8, 3f118, 6, 10c00, 3f118, a) + 8
  ff36ebf8 apr_thread_pool_tasks_cancel (3f088, 0, ff36e8cc, 0, 1, 
ffbff2cc) + 134
  ff36eeec thread_pool_cleanup (3f088, fefa0c78, 17248, 0, ff3f4380, 0) + 14
  fedcef18 apr_pool_destroy (44278, 0, fe635900, 6, fec92a00, 3f118) + 20
  ff36e8cc apr_thread_pool_destroy (0, 0, 0, 71, 0, 44760) + 8
  00017248 test_threadsafe (41e70, 0, d95ac, ff, fe6303d8, 0) + 8c
  0001450c abts_run_test (41e70, 171bc, 0, 0, fec92a00, 410c0) + 94
  000172a4 testpass (41e50, 3f030, 0, 4, fec92a00, 5400) + 4c
  00022444 main     (1, ffbff5f8, 386d4, 378e8, 378ac, 41e50) + 140
  00013e00 _start   (0, 0, 0, 0, 0, 0) + 5c
-----------------  lwp# 2 / thread# 2  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 3 / thread# 3  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 4 / thread# 4  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 5 / thread# 5  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 6 / thread# 6  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 7 / thread# 7  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 8 / thread# 8  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 9 / thread# 9  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 10 / thread# 10  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 11 / thread# 11  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **

Thanks and regards,

Rainer


Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

Posted by Rainer Jung <ra...@kippdata.de>.
Confirmed again, patch works for me. Thanks!

Am 25.04.2021 um 16:23 schrieb Rainer Jung:
> Thanks a bunch Yann, looks very promising. Testing is ongoing and I will 
> report back later today, but I did not yet observe any hangs with the 
> patch although on the specific platform I am testing now (SLES 12) they 
> happened pretty frequent without the patch.
> 
> Best regards,
> 
> Rainer
> 
> Am 25.04.2021 um 13:36 schrieb Yann Ylavic:
>> On Fri, Apr 23, 2021 at 11:26 AM Rainer Jung <ra...@kippdata.de> 
>> wrote:
>>>
>>> When compiling and running make check for latest head of APU 1.6.x in
>>> preparation of httpd release testing, I observe very frequent hangs
>>> during testqueue and testpass.
>>>
>>> It seems the hang happens only for APU 1.6.x head, but not for APU 
>>> 1.6.1.
>>>
>>> The hangs seem not to depend on the exact APR version. I observed them
>>> with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.
>>>
>>> I have not yet tested APU 1.7.x head.
>>>
>>> I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they
>>> do not happen on RHEL 7 and 8.
>>>
>>> Here are some stacks:
>>
>> Thanks Rainer for the details.
>>
>> Does the attached patch (for 1.6.x) fix the hang?
>>
>> Regards;
>> Yann.

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

Posted by Rainer Jung <ra...@kippdata.de>.
Thanks a bunch Yann, looks very promising. Testing is ongoing and I will 
report back later today, but I did not yet observe any hangs with the 
patch although on the specific platform I am testing now (SLES 12) they 
happened pretty frequent without the patch.

Best regards,

Rainer

Am 25.04.2021 um 13:36 schrieb Yann Ylavic:
> On Fri, Apr 23, 2021 at 11:26 AM Rainer Jung <ra...@kippdata.de> wrote:
>>
>> When compiling and running make check for latest head of APU 1.6.x in
>> preparation of httpd release testing, I observe very frequent hangs
>> during testqueue and testpass.
>>
>> It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.
>>
>> The hangs seem not to depend on the exact APR version. I observed them
>> with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.
>>
>> I have not yet tested APU 1.7.x head.
>>
>> I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they
>> do not happen on RHEL 7 and 8.
>>
>> Here are some stacks:
> 
> Thanks Rainer for the details.
> 
> Does the attached patch (for 1.6.x) fix the hang?
> 
> Regards;
> Yann.

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Apr 23, 2021 at 11:26 AM Rainer Jung <ra...@kippdata.de> wrote:
>
> When compiling and running make check for latest head of APU 1.6.x in
> preparation of httpd release testing, I observe very frequent hangs
> during testqueue and testpass.
>
> It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.
>
> The hangs seem not to depend on the exact APR version. I observed them
> with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.
>
> I have not yet tested APU 1.7.x head.
>
> I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they
> do not happen on RHEL 7 and 8.
>
> Here are some stacks:

Thanks Rainer for the details.

Does the attached patch (for 1.6.x) fix the hang?

Regards;
Yann.

RE: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

Posted by Luke Perkins <lu...@epicdgs.us>.
Please unsubscribe this person from the list. He has cancer and is unable to make any  cognitive decisions to this discussion.

Thank-you,

Caretaker of Luke Perkins
Cell: 719-339-0987

-----Original Message-----
From: Rainer Jung <ra...@kippdata.de> 
Sent: Friday, April 23, 2021 5:26 AM
To: dev@apr.apache.org
Subject: Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

When compiling and running make check for latest head of APU 1.6.x in preparation of httpd release testing, I observe very frequent hangs during testqueue and testpass.

It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.

The hangs seem not to depend on the exact APR version. I observed them with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.

I have not yet tested APU 1.7.x head.

I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they do not happen on RHEL 7 and 8.

Here are some stacks:

RHEL 6

#0  0x0000003906c0b44c in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x00007fe39c453cb5 in apr_thread_pool_tasks_cancel () from
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x00007fe39c453d45 in thread_pool_cleanup () from
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x00007fe39bff6c6e in apr_pool_destroy () from
/shared/build/dev/httpd/install/apr/1.6.x/r1876940/rhel6.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x00007fe39c45492c in apr_thread_pool_destroy () from
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#5  0x000000000040853c in test_threadsafe ()
#6  0x000000000040610c in abts_run_test ()
#7  0x00000000004081e2 in testpass ()
#8  0x0000000000405675 in main ()

SLES 11

#1  0x00002b1e34104315 in apr_thread_pool_tasks_cancel () from
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x00002b1e34104395 in thread_pool_cleanup () from
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x00002b1e3455dcff in apr_pool_destroy () from
/path/to/httpd/install/apr/1.6.x/1.6.5/sles11.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x00002b1e34104fdc in apr_thread_pool_destroy () from
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#5  0x000000000040b7e0 in test_queue_producer_consumer ()
#6  0x00000000004067ed in abts_run_test ()
#7  0x000000000040b611 in testqueue ()
#8  0x0000000000405d6d in main ()

SLES 12

#0  0x00007f39ed06c05f in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x00007f39ed8e6c2d in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>) at
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/locks/unix/thread_cond.c:68
#2  0x00007f39edd49cd1 in wait_on_busy_threads (owner=0x0, me=0x20c5630) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thread_pool_tasks_cancel (me=me@entry=0x20c5630,
owner=owner@entry=0x0) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:793
#4  0x00007f39edd49fe5 in thread_pool_cleanup (me=0x20c5630) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:397
#5  0x00007f39ed8e7f9e in run_cleanups (cref=0x20dd3d8) at
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:2705
#6  apr_pool_destroy (pool=0x20dd368) at
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:978
#7  0x00007f39edd4995c in apr_thread_pool_destroy (me=<optimized out>) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:457
#8  0x000000000040b0e1 in test_queue_producer_consumer (tc=0x21de5d0, data=<optimized out>) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:120
#9  0x00000000004060ee in abts_run_test (ts=<optimized out>, f=0x40afe0 <test_queue_producer_consumer>, value=0x0) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:172
#10 0x000000000040b231 in testqueue (suite=0x20c7160) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:131
#11 0x0000000000405a3d in main (argc=<optimized out>, argv=<optimized 
out>) at
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:416

SLES 15

#0  0x00007f41a1a2b89d in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x00007f41a22a576d in apr_thread_cond_wait (cond=<optimized out>, 
mutex=<optimized out>) at 
/path/to/sources/apr/1.6.x/r1876940/locks/unix/thread_cond.c:68
#2  0x00007f41a270d3d9 in wait_on_busy_threads (owner=0x0, me=0x1164870) 
at /path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thread_pool_tasks_cancel (me=me@entry=0x1164870, 
owner=owner@entry=0x0) at 
/path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:793
#4  0x00007f41a270d6a5 in thread_pool_cleanup (me=0x1164870) at 
/path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:397
#5  0x00007f41a22a685e in run_cleanups (cref=0x11705e8) at 
/path/to/sources/apr/1.6.x/r1876940/memory/unix/apr_pools.c:2666
#6  apr_pool_destroy (pool=0x1170578) at 
/shared/build/dev/httpd/sources/apr/1.6.x/r1876940/memory/unix/apr_pools.c:993
#7  0x00007f41a270cfec in apr_thread_pool_destroy (me=<optimized out>) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:457
#8  0x000000000040b099 in test_queue_producer_consumer (tc=0x12d3800, 
data=<optimized out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:120
#9  0x0000000000405fc8 in abts_run_test (ts=<optimized out>, f=0x40af90 
<test_queue_producer_consumer>, value=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:172
#10 0x000000000040b1f1 in testqueue (suite=0x11663b0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:131
#11 0x0000000000405934 in main (argc=<optimized out>, argv=<optimized 
out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:416


Solaris

-----------------  lwp# 1 / thread# 1  --------------------
  fe5c9590 lwp_park (0, 0, 0)
  fe5c35d8 cond_wait_queue (442e8, 3f118, 0, 0, ff3f48e8, 0) + 4c
  fe5c3a1c cond_wait_common (442e8, 3f118, 0, 0, 0, 0) + 294
  fe5c3b20 cond_wait (442e8, 3f118, 0, 1c00, fe5c7b30, ff3f6ad0) + 10
  fe5c3b5c pthread_cond_wait (442e8, 3f118, 6, 10c00, 3f118, a) + 8
  ff36ebf8 apr_thread_pool_tasks_cancel (3f088, 0, ff36e8cc, 0, 1, 
ffbff2cc) + 134
  ff36eeec thread_pool_cleanup (3f088, fefa0c78, 17248, 0, ff3f4380, 0) + 14
  fedcef18 apr_pool_destroy (44278, 0, fe635900, 6, fec92a00, 3f118) + 20
  ff36e8cc apr_thread_pool_destroy (0, 0, 0, 71, 0, 44760) + 8
  00017248 test_threadsafe (41e70, 0, d95ac, ff, fe6303d8, 0) + 8c
  0001450c abts_run_test (41e70, 171bc, 0, 0, fec92a00, 410c0) + 94
  000172a4 testpass (41e50, 3f030, 0, 4, fec92a00, 5400) + 4c
  00022444 main     (1, ffbff5f8, 386d4, 378e8, 378ac, 41e50) + 140
  00013e00 _start   (0, 0, 0, 0, 0, 0) + 5c
-----------------  lwp# 2 / thread# 2  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 3 / thread# 3  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 4 / thread# 4  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 5 / thread# 5  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 6 / thread# 6  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 7 / thread# 7  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 8 / thread# 8  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 9 / thread# 9  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 10 / thread# 10  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **
-----------------  lwp# 11 / thread# 11  --------------------
  feddafd4 dummy_worker(), exit value = 0x00000000
         ** zombie (exited, not detached, not yet joined) **

Thanks and regards,

Rainer