You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Martin Knoblauch <kn...@gmail.com> on 2014/06/16 17:44:19 UTC
Problems with excessive connection timeouts
Hi,
my environment is:
OS SuSE SLES11/SP2 64-bit
HTTP frontend: Apache2-2.2.21 (64-bit)
mod_jk: 1.2.37
Problem short: excessive connection timeouts with weird socket ids
Problem long: we frontend 4 Tomcats (7.0.42) using Apache2(2.2.21) and
mod_jk(1.2.37). We are seeing a heavy amount of connection timeouts when
the tomcats are loaded. Surprisingly the logs show socket id "-1", which
should not happen in my opinion, because most actions are guarded by the
IS_VALID_SOCKET macro. Here is one example. The code emmitting the line
should not be reached due to the check in line 1543:
[Mon Jun 16 17:12:05.405 2014] [7233:140275337774848] [info]
ajp_send_request::jk_ajp_common.c (1562): (lpsdm83-1) failed sending
request, socket -1 prepost cping/cpong failure (errno=110)
1543: while (IS_VALID_SOCKET(ae->sd)) {
int err = JK_FALSE;
if (jk_is_socket_connected(ae->sd, l) == JK_FALSE) {
ae->last_errno = errno;
jk_log(l, JK_LOG_DEBUG,
"(%s) failed sending request, "
"socket %d is not connected any more (errno=%d)",
ae->worker->name, ae->sd, ae->last_errno);
ajp_abort_endpoint(ae, JK_FALSE, l);
err = JK_TRUE;
err_conn++;
}
if (ae->worker->prepost_timeout > 0 && !err) {
/* handle cping/cpong if prepost_timeout is set
* If the socket is disconnected no need to handle
* the cping/cpong
*/
if (ajp_handle_cping_cpong(ae,
ae->worker->prepost_timeout, l) == JK_FALSE) {
1562: jk_log(l, JK_LOG_INFO,
"(%s) failed sending request, "
"socket %d prepost cping/cpong failure (errno=%d)",
ae->worker->name, ae->sd, ae->last_errno);
/* XXX: Is there any reason to try other
* connections to the node if one of them fails
* the cping/cpong heartbeat?
* Tomcat can be either too busy or simply dead, so
* there is a chance that all other connections would
* fail as well.
*/
err = JK_TRUE;
err_cping++;
}
}
So, before trying to tell me that I am missing workers.properties and
complete logs, I want to check first whether the "-1" socket ids are to be
expected here or whether there is some kind of bug involved.
Thanks in advance
Martin
--
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Re: Problems with excessive connection timeouts
Posted by Martin Knoblauch <kn...@gmail.com>.
Hi Konstantin,
yeah, found out two hours after posting. Embarassing, isn't it.
Thanks
Martin
On Tue, Jun 17, 2014 at 7:14 PM, Konstantin Kolinko <kn...@gmail.com>
wrote:
> 2014-06-16 19:44 GMT+04:00 Martin Knoblauch <kn...@gmail.com>:
> > Hi,
> >
> > my environment is:
> >
> > OS SuSE SLES11/SP2 64-bit
> > HTTP frontend: Apache2-2.2.21 (64-bit)
> > mod_jk: 1.2.37
> >
> >
> > Problem short: excessive connection timeouts with weird socket ids
> > Problem long: we frontend 4 Tomcats (7.0.42) using Apache2(2.2.21) and
> > mod_jk(1.2.37). We are seeing a heavy amount of connection timeouts when
> > the tomcats are loaded. Surprisingly the logs show socket id "-1", which
> > should not happen in my opinion, because most actions are guarded by the
> > IS_VALID_SOCKET macro. Here is one example. The code emmitting the line
> > should not be reached due to the check in line 1543:
> >
> > [Mon Jun 16 17:12:05.405 2014] [7233:140275337774848] [info]
> > ajp_send_request::jk_ajp_common.c (1562): (lpsdm83-1) failed sending
> > request, socket -1 prepost cping/cpong failure (errno=110)
> >
> >
> > 1543: while (IS_VALID_SOCKET(ae->sd)) {
> > int err = JK_FALSE;
> > if (jk_is_socket_connected(ae->sd, l) == JK_FALSE) {
> > ae->last_errno = errno;
> > jk_log(l, JK_LOG_DEBUG,
> > "(%s) failed sending request, "
> > "socket %d is not connected any more (errno=%d)",
> > ae->worker->name, ae->sd, ae->last_errno);
> > ajp_abort_endpoint(ae, JK_FALSE, l);
> > err = JK_TRUE;
> > err_conn++;
> > }
> > if (ae->worker->prepost_timeout > 0 && !err) {
> > /* handle cping/cpong if prepost_timeout is set
> > * If the socket is disconnected no need to handle
> > * the cping/cpong
> > */
> > if (ajp_handle_cping_cpong(ae,
> > ae->worker->prepost_timeout, l) == JK_FALSE) {
> > 1562: jk_log(l, JK_LOG_INFO,
> > "(%s) failed sending request, "
> > "socket %d prepost cping/cpong failure
> (errno=%d)",
> > ae->worker->name, ae->sd, ae->last_errno);
> > /* XXX: Is there any reason to try other
> > * connections to the node if one of them fails
> > * the cping/cpong heartbeat?
> > * Tomcat can be either too busy or simply dead, so
> > * there is a chance that all other connections would
> > * fail as well.
> > */
> > err = JK_TRUE;
> > err_cping++;
> > }
> > }
> >
> > So, before trying to tell me that I am missing workers.properties and
> > complete logs, I want to check first whether the "-1" socket ids are to
> be
> > expected here or whether there is some kind of bug involved.
> >
>
> The ajp_handle_cping_cpong(...) method that is called a few lines
> earlier (line 1560) processes some failures by calling
> ajp_abort_endpoint(...). That method closes the socket and sets it to
> "-1".
>
> Thus the socket is not "-1" by the start of the loop, but can become
> "-1" in the process.
>
>
> Best regards,
> Konstantin Kolinko
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Re: Problems with excessive connection timeouts
Posted by Konstantin Kolinko <kn...@gmail.com>.
2014-06-16 19:44 GMT+04:00 Martin Knoblauch <kn...@gmail.com>:
> Hi,
>
> my environment is:
>
> OS SuSE SLES11/SP2 64-bit
> HTTP frontend: Apache2-2.2.21 (64-bit)
> mod_jk: 1.2.37
>
>
> Problem short: excessive connection timeouts with weird socket ids
> Problem long: we frontend 4 Tomcats (7.0.42) using Apache2(2.2.21) and
> mod_jk(1.2.37). We are seeing a heavy amount of connection timeouts when
> the tomcats are loaded. Surprisingly the logs show socket id "-1", which
> should not happen in my opinion, because most actions are guarded by the
> IS_VALID_SOCKET macro. Here is one example. The code emmitting the line
> should not be reached due to the check in line 1543:
>
> [Mon Jun 16 17:12:05.405 2014] [7233:140275337774848] [info]
> ajp_send_request::jk_ajp_common.c (1562): (lpsdm83-1) failed sending
> request, socket -1 prepost cping/cpong failure (errno=110)
>
>
> 1543: while (IS_VALID_SOCKET(ae->sd)) {
> int err = JK_FALSE;
> if (jk_is_socket_connected(ae->sd, l) == JK_FALSE) {
> ae->last_errno = errno;
> jk_log(l, JK_LOG_DEBUG,
> "(%s) failed sending request, "
> "socket %d is not connected any more (errno=%d)",
> ae->worker->name, ae->sd, ae->last_errno);
> ajp_abort_endpoint(ae, JK_FALSE, l);
> err = JK_TRUE;
> err_conn++;
> }
> if (ae->worker->prepost_timeout > 0 && !err) {
> /* handle cping/cpong if prepost_timeout is set
> * If the socket is disconnected no need to handle
> * the cping/cpong
> */
> if (ajp_handle_cping_cpong(ae,
> ae->worker->prepost_timeout, l) == JK_FALSE) {
> 1562: jk_log(l, JK_LOG_INFO,
> "(%s) failed sending request, "
> "socket %d prepost cping/cpong failure (errno=%d)",
> ae->worker->name, ae->sd, ae->last_errno);
> /* XXX: Is there any reason to try other
> * connections to the node if one of them fails
> * the cping/cpong heartbeat?
> * Tomcat can be either too busy or simply dead, so
> * there is a chance that all other connections would
> * fail as well.
> */
> err = JK_TRUE;
> err_cping++;
> }
> }
>
> So, before trying to tell me that I am missing workers.properties and
> complete logs, I want to check first whether the "-1" socket ids are to be
> expected here or whether there is some kind of bug involved.
>
The ajp_handle_cping_cpong(...) method that is called a few lines
earlier (line 1560) processes some failures by calling
ajp_abort_endpoint(...). That method closes the socket and sets it to
"-1".
Thus the socket is not "-1" by the start of the loop, but can become
"-1" in the process.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org