[users@httpd] hang on read(), threads won't/can't die?

[Bob Tanner <ta...@real-time.com>]

Have the folling:

apache-1.3.33
libapache-mod-ssl-2.8.22
kernel-2.4.26-1-686-smp

Having a problem where https connections just won't die. Over time the process 
table fills and the box crawls or falls to its knees.

Installed debugging version of apache, here is gdb's backtrace showing the
block on read() called from mod_ssl's ssl_io_unregister() function.

(gdb) bt
#0  0x40048a98 in read () from /lib/libpthread.so.0
#1  0x40357a30 in ?? () from /usr/lib/apache/1.3/mod_ssl.so
#2  0x40340a95 in ssl_io_unregister () from /usr/lib/apache/1.3/mod_ssl.so
#3  0x080730ae in ap_hook_call_func (ap=0x0, he=0x3, hf=0x809cdac)
    at ap_hook.c:721
#4  0x08072d00 in ap_hook_call (
    hook=0xfffffe00 <Address 0xfffffe00 out of bounds>) at ap_hook.c:382
                              ^^^^^^^^^^^^^^^^^^^^^^^ problem here too?

#5  0x080536b7 in ap_read (fb=0x809cd64, buf=0x809cdac, nbyte=4096)
    at buff.c:254
#6  0x08053bf3 in read_with_errors (fb=0x809cd64, buf=0x809cdac, nbyte=4096)
    at buff.c:298
#7  0x08053ec5 in ap_bgets (buff=0xbfffc240 "", n=8192, fb=0x809cd64)
    at buff.c:882
#8  0x08063423 in ap_getline (s=0xbfffc240 "", n=8192, in=0x809cd64, fold=0)
    at http_protocol.c:867
#9  0x0806367e in read_request_line (r=0x8095074) at http_protocol.c:991
#10 0x08063d13 in ap_read_request (conn=0x8336f3c) at http_protocol.c:1187
#11 0x08060a80 in child_main (child_num_arg=-512) at http_main.c:4862
#12 0x08060de7 in make_child (s=0xfffffe00, slot=5, now=-512)
    at http_main.c:5052
#13 0x080610c7 in perform_idle_server_maintenance () at http_main.c:5237
#14 0x08061a48 in standalone_main (argc=1, argv=0xbfffe514) at
http_main.c:5500
#15 0x08061ff8 in main (argc=1, argv=0xbfffe514) at http_main.c:5768

Using strace -p <pid> I get:
read(3,

So, I'm pretty sure the block is happening on the read of file description 3.

Using lsof -n -p <pid> I get and looking for FD 3, I see this:

apache.db 21547 www-data    3u  IPv4 16364769             TCP
xx.xxx.xxx.xxx:www->xxx.xxx.xxx.xxx:51923 (ESTABLISHED)

So I know fd 3 is an IPv4 client and the socket is established.

After collecting several days of data, the client IPs seem to be all cache
servers (based on their hostnames).

Anyone know of a reason why the socket can't/won't close?

That's all the info I can think of reporting.

Anyone seen this problem before?


Thanks.

-- 
Bob Tanner <ta...@mn-linux.org>          | Phone : (952)943-8700
http://www.mn-linux.org, Minnesota, Linux | Fax   : (952)943-8500
Key fingerprint = AB15 0BDF BCDE 4369 5B42  1973 7CF1 A709 2CC1 B288

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org