You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2011/03/09 15:34:37 UTC

DO NOT REPLY [Bug 50902] New: on major load on the server, poll() hangs

https://issues.apache.org/bugzilla/show_bug.cgi?id=50902

           Summary: on major load on the server, poll() hangs
           Product: Apache httpd-2
           Version: 2.2.16
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: worker
        AssignedTo: bugs@httpd.apache.org
        ReportedBy: tal.yalon@gmail.com


Hi there,

Thought this bug is related to bug #50247, but as per Eric's suggestion I'm
opening a new bug.

The httpd in question is 2.2.16 running on EC2.

We see that after ~10 minutes of significant load, some of the worker processes
get hang on poll().

By hang I mean that poll() call didn't finish for more than 30 seconds.

This is the stacktrace from one of the processes that are in this state:

#0  0x00007f76fa70c748 in poll () from /lib64/libc.so.6
#1  0x00007f76fabe2822 in apr_wait_for_io_or_timeout () from
/usr/lib64/libapr-1.so.0
#2  0x00007f76fabdd2da in apr_socket_recv () from /usr/lib64/libapr-1.so.0
#3  0x00007f76fc0dfddd in ap_lingering_close ()
#4  0x00007f76fc0eaa7e in ?? ()
#5  0x00007f76fc0ead1a in ?? ()
#6  0x00007f76fc0eadd0 in ?? ()
#7  0x00007f76fc0eb908 in ap_mpm_run ()
#8  0x00007f76fc0c54fb in main ()

Please let me know if there's any more information I can provide - this problem
is obviously a major concern to us.

Thanks,
Tal

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 50902] on major load on the server, poll() hangs

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50902

--- Comment #2 from Ruediger Pluem <rp...@apache.org> 2011-03-09 11:47:33 EST ---
(In reply to comment #1)
> We have to see the parameters passed to poll() (perhaps from strace, perhaps
> from a backtrace on a debug build, perhaps from somebody understanding the
> assembly code with the same build you got the backtraces from).
> 
> The timeout is hard-coded to 2 seconds, so it isn't supposed to block longer
> than that.

If the remote partner (in this case the client) does not close the socket we
can return to the poll call over and over again for the next 30 seconds. We
only stay in a single poll call for at max 2 seconds, but may have up to 15
calls to poll.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 50902] on major load on the server, poll() hangs

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50902

--- Comment #3 from Jeff Trawick <tr...@apache.org> 2011-03-09 12:01:57 EST ---
thanks, Ruediger
we need to distinguish between blocking in a single poll() call for a long time
vs. repeatedly calling poll(); strace would make that clear; gdb could make
that clear as well if used properly

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 50902] on major load on the server, poll() hangs

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50902

tal.yalon@gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tal.yalon@gmail.com

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 50902] on major load on the server, poll() hangs

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50902

Jeff Trawick <tr...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #1 from Jeff Trawick <tr...@apache.org> 2011-03-09 10:06:52 EST ---
We have to see the parameters passed to poll() (perhaps from strace, perhaps
from a backtrace on a debug build, perhaps from somebody understanding the
assembly code with the same build you got the backtraces from).

The timeout is hard-coded to 2 seconds, so it isn't supposed to block longer
than that.
A FIN has already been sent on the connection, so poll() could wake up on
socket activity before "too long."

If server-status is enabled, a thread with this backtrace should show up as "C"
(closing).

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org