You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Phineas Dole <ph...@yahoo.com> on 2012/07/24 22:59:44 UTC

Default configuration; Connection reset by peer

I am currently benchmarking tomcat to see what sort of performance I can expect.

I setup tomcat with the default configuration and then run "ab -c 1000 -n 1000000 http://localhost:8080/examples/servlets/servlet/HelloWorldExample"

Occasionally, ab will terminate prematurely with error such as "apr_socket_recv: Connection reset by peer (104)"

This happens with bio, nio, and apr connectors.

How do I configure tomcat or my OS to handle this workload? I have not experienced this problem with other webservers or Jetty... what is happening?

Versions:
Tomcat 7.0.29
ab 2.3

Re: Default configuration; Connection reset by peer

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phineas,

On 7/24/12 7:19 PM, Phineas Dole wrote:
> Test configuration: <Connector port="8080"
> protocol="org.apache.coyote.http11.Http11NioProtocol" 
> acceptCount="10240" numThreads="1024"

numThreads is not a recognized configuration attribute. Read the docs:
http://tomcat.apache.org/tomcat-7.0-doc/config/http.html

> I have increased somaxconn.

Are you sure?

> The error has become very consistent. I can trigger it by
> restarting tomcat and then running the test. Typically it takes <5s
> for ab to terminate. (Though ab states that several thousand
> requests have completed by that point, which appears true from the
> access logs).

With a trivial servlet, you should be able to handle a *lot* of load.
Are you testing on localhost (i.e. both ab and Tomcat run on the same
machine)?

> The high (10240) acceptCount seems to be the reason for the 
> consistent error. Decreasing it seems to make things more stable.

In general, I would expect that. There's not a really good reason for
a high acceptCount. If your machine can't handle a huge amount of
load, don't make a whole bunch of clients get in a huge line: just
refuse their connections.

> In fact, setting acceptCount="5" and connectionTimeout="-1" seems
> to delay the error by a huge amount, but it still occurs. Also,
> the connectionTimeout resolves "length" and "exception" errors 
> occasionally reported by ab. I presume this is related more to how 
> the client (ab) operates and not tomcat itself.

Perhaps. If it's appropriate for your testing, try using "-k" to
enable HTTP keepalive to see if you get better throughput (and fewer
errors).

> The acceptCount is counter-intuitive to me. Was this the intended 
> solution you had in mind.

No: I think you should change maxThreads. Better yet, use an
<Executor> so you have better control over everything. TC7 always uses
an <Executor> implicitly, but there are fewer configuration options
when doing it through the <Connector>.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAQEn4ACgkQ9CaO5/Lv0PA1RACgpOoW6lPrO4vEDA8/kMnKFtKk
7QoAnA7G7uz2wD2YmoLmuHUKA3sp4W3l
=RCwk
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Default configuration; Connection reset by peer

Posted by Phineas Dole <ph...@yahoo.com>.
Test configuration:
<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
               acceptCount="10240"
               numThreads="1024"
               connectionTimeout="20000"
               redirectPort="8443" />

I have increased somaxconn.

The error has become very consistent. I can trigger it by restarting tomcat and then running the test. Typically it takes <5s for ab to terminate. (Though ab states that several thousand requests have completed by that point, which appears true from the access logs).

The high (10240) acceptCount seems to be the reason for the consistent error. Decreasing it seems to make things more stable. In fact, setting acceptCount="5" and connectionTimeout="-1" seems to delay the error by a huge amount, but it still occurs. Also, the connectionTimeout resolves "length" and "exception" errors occasionally reported by ab. I presume this is related more to how the client (ab) operates and not tomcat itself.

The acceptCount is counter-intuitive to me. Was this the intended solution you had in mind.

Thanks for the insight!


________________________________
 From: Christopher Schultz <ch...@christopherschultz.net>
To: Tomcat Users List <us...@tomcat.apache.org> 
Sent: Tuesday, July 24, 2012 2:34 PM
Subject: Re: Default configuration; Connection reset by peer
 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phineas,

On 7/24/12 4:59 PM, Phineas Dole wrote:
> I am currently benchmarking tomcat to see what sort of performance
> I can expect.
> 
> I setup tomcat with the default configuration and then run "ab -c 
> 1000 -n 1000000 
> http://localhost:8080/examples/servlets/servlet/HelloWorldExample"

Note that the default configuration of Tomcat's Connector has
maxThreads=200 and acceptCount=100 meaning that Tomcat will only
actively process 200 simultaneous connections and the OS is expected to
accept 100 connections and queue them until Tomcat is ready to actively
handle them.

I suspect you will very quickly fill-up the accept queue and start
getting connections dropped, unless your test servlet runs very quickly.

> Occasionally, ab will terminate prematurely with error such as 
> "apr_socket_recv: Connection reset by peer (104)"
> 
> This happens with bio, nio, and apr connectors.
> 
> How do I configure tomcat or my OS to handle this workload? I have 
> not experienced this problem with other webservers or Jetty...
> what is happening?

What was their configuration in terms of request processors, etc.?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAPFNYACgkQ9CaO5/Lv0PC0LACgrUikOLHu3LBmwZap9ZzwG0Ap
B6IAnA+ze3YX2a4elffzB3cHsxLmwKgY
=UKxo
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Default configuration; Connection reset by peer

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phineas,

On 7/24/12 4:59 PM, Phineas Dole wrote:
> I am currently benchmarking tomcat to see what sort of performance
> I can expect.
> 
> I setup tomcat with the default configuration and then run "ab -c 
> 1000 -n 1000000 
> http://localhost:8080/examples/servlets/servlet/HelloWorldExample"

Note that the default configuration of Tomcat's Connector has
maxThreads=200 and acceptCount=100 meaning that Tomcat will only
actively process 200 simultaneous connections and the OS is expected to
accept 100 connections and queue them until Tomcat is ready to actively
handle them.

I suspect you will very quickly fill-up the accept queue and start
getting connections dropped, unless your test servlet runs very quickly.

> Occasionally, ab will terminate prematurely with error such as 
> "apr_socket_recv: Connection reset by peer (104)"
> 
> This happens with bio, nio, and apr connectors.
> 
> How do I configure tomcat or my OS to handle this workload? I have 
> not experienced this problem with other webservers or Jetty...
> what is happening?

What was their configuration in terms of request processors, etc.?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAlAPFNYACgkQ9CaO5/Lv0PC0LACgrUikOLHu3LBmwZap9ZzwG0Ap
B6IAnA+ze3YX2a4elffzB3cHsxLmwKgY
=UKxo
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org