You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Marc Wilmots <de...@gmail.com> on 2010/11/05 00:27:05 UTC
Re: mod_jk makes ajp connector run out of free connections
Rainer, that's very true.
So I am wondering...If 1200 > 200 (Tomcat maxThreads), then what would be
the right way to solve this? Imagine that 2 Tomcats would crash, that would
leave me with 1200 > 400. I can't just raise the maxThreads to 600 for each
Tomcat, as that would probably perform even worse (I'm thinking about
excessive context switching, etc).
The best solution would be to enable DisableReuse as Mark stated, although
I'm afraid that would be too big of a performance hit.
The other option would be to configure the connection_pool_size, according
to Mladen's link. Although, this last one isn't very clear to me either. I
state from the URL Mladen passed:
*"There are 4 JBoss servers each containing 1 JBoss instance(s) giving a
total of 4 mod_jk workers*
*Given the ThreadsPerChild of 60 and the MaxClients (600) / ThreadsPerChild
(60) = Processes (10) and with 2 Apache instance(s) then the
connection_pool_size has been best determined to be 15*
*The total Jboss threads 1600 are > the total Apache threads 1200. The
mod_jk connection_pool_size was determined to be 15 via the math 600 Apache
MaxClients / 4 total Jboss instances 150 threads per JBoss which will be
equal to or less than the maxThreads 400 150 / 10 Apache server processes to
get the connection_pool_size of 15. in Jboss. Take this number"*
I'm wondering, setting connection_pool_size 15, wouldn't this mean that the
maximum number of connections being handled by tomcat at one time is limited
to 15?? Or would there be a pool per Apache server process? Which would mean
150 connections (10 server processes) per Tomcat. If so, then this would
total 600 connections (4 Tomcats).
1200 > 600. Same problem, right?
2010/10/28 Rainer Jung <ra...@kippdata.de>
> On 26.10.2010 20:36, Marc Wilmots wrote:
>
>> 2010/10/26 Mladen Turk<mt...@apache.org>
>>
>> On 10/26/2010 02:47 PM, Marc Wilmots wrote:
>>>
>>> Hi Experts,
>>>>
>>>> I have the following setup:
>>>>
>>>> Apache1
>>>> / \
>>>> Apache2 Apache3
>>>> \ /
>>>> Tomcat1
>>>>
>>>> All Apaches are version 2.2.3 (RedHat)
>>>> Apache2 and Apache3 loadbalance Tomcat1 (6.0.18) with mod_jk (2.2.28).
>>>>
>>>> In idle state the AJP connector of Tomcat1 only has 7 active
>>>> connections.
>>>> After launching a stress test of Tomcat1, it's AJP connector has reached
>>>> maxThreads (200). After the stress test has finished, there are still
>>>> 200
>>>> active connections in the AJP connector.
>>>>
>>>> Because of this, apache2 and apache2 cannot receive any heartbeat
>>>> message
>>>> anymore from the AJP connector and mark Tomcat1 as dead. I can access
>>>> perfectly through port 8080, so Tomcat1 isn't dead at all!
>>>>
>>>>
>>>> You have a usual setup problem where the number of connections
>>> in httpd fronting tomcat is too high for a tomcat.
>>> And you don't have any mechanism for limiting those connections.
>>>
>>> Sorry, I realize I had to mention I have a total of 4 Tomcats, which in
>>>
>> total make up a maxThreads of 800 (200 each).
>> The Apaches have a maxClients of each 600 (1200 total).
>>
>> 1200> 1000, thus your theory is probably right. However, I'm just
>> wondering... I read that when Tomcat reaches the max number of
>> connections,
>> just rejects new connections (I'm omitting the backlog on purpose).
>>
>
> Yes.
>
> But your calculation above is not right. Each Apache will connect to each
> Tomcat. So from the point of view of one Tomcat, it has 200 threads, but
> 1200 posible incoming connections from the Apache Servers. So it is not 1200
> > 800 but 1200 > 200!
>
> Regards,
>
> Rainer
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: mod_jk makes ajp connector run out of free connections
Posted by Mark Thomas <ma...@apache.org>.
On 04/11/2010 20:01, Marc Wilmots wrote:
> Honestly....no, I didn't.
> The mod_jk scared the hell out of me....wouldn't you? :-)
>
> "Using this option will have a strong performance penalty for Apache and
> Tomcat. Use this only as a last resort in case of unfixable network
> problems."
Experience has shown that the impact is usually (YMMV) a lot less
significant than that. You're right - we should update the docs.
Mark
>
> I'll give it a try tomorrow and see what it gives.
>
> By the way, my last calculation about connection_pool_size was wrong. I
> forgot to include the second Apache server. The math would look like this
> then:
>
> Apache1 -> 10 server processes x 15 x 4 (tomcats) = 600
> Apache2 -> 10 server processes x 15 x 4 (tomcats) = 600, which makes 1200 in
> total
>
> Each Tomcat would receive a maximum of 300 connections, which would leave
> 100 free connections per Tomcat. This would solve the problem that Tomcat
> runs out of connections. The thing is...not all of my available Apache
> connections can be used:
>
> Apache has 60 threads per server process (10) -> 600 connections.
> connection_pool_size per server process: 15. This would mean only 15 of 60
> possible connections per server process can actually establish a connection
> to Tomcat.
>
> In order to be able to server all Apache connections simultaneously, I would
> need 4 more Tomcats?
>
>
>
>
>
>
> 2010/11/5 Mark Thomas <ma...@apache.org>
>
>> On 04/11/2010 19:27, Marc Wilmots wrote:
>>> The best solution would be to enable DisableReuse as Mark stated,
>> although
>>> I'm afraid that would be too big of a performance hit.
>>
>> Have you tried it?
>>
>> The cost of the required CPING/CPONG isn't that different to the cost of
>> creating a TCP/IP connection. On a fast local area network I doubt you'd
>> notice it. The time taken to establish a TCP/IP connection is usually
>> (YMMV so test it) << the time taken to process a request.
>>
>> Mark
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk makes ajp connector run out of free connections
Posted by Marc Wilmots <de...@gmail.com>.
Forget about my math question.
Obviously, connection_pool_size of 15, would mean there are 60 (4 tomcats)
in total per server process.
Then the math works out....
Sorry, it's 1:08am over here :-)
2010/11/5 Marc Wilmots <de...@gmail.com>
> Honestly....no, I didn't.
> The mod_jk scared the hell out of me....wouldn't you? :-)
>
> "Using this option will have a strong performance penalty for Apache and
> Tomcat. Use this only as a last resort in case of unfixable network
> problems."
>
> I'll give it a try tomorrow and see what it gives.
>
> By the way, my last calculation about connection_pool_size was wrong. I
> forgot to include the second Apache server. The math would look like this
> then:
>
> Apache1 -> 10 server processes x 15 x 4 (tomcats) = 600
> Apache2 -> 10 server processes x 15 x 4 (tomcats) = 600, which makes 1200
> in total
>
> Each Tomcat would receive a maximum of 300 connections, which would leave
> 100 free connections per Tomcat. This would solve the problem that Tomcat
> runs out of connections. The thing is...not all of my available Apache
> connections can be used:
>
> Apache has 60 threads per server process (10) -> 600 connections.
> connection_pool_size per server process: 15. This would mean only 15 of 60
> possible connections per server process can actually establish a connection
> to Tomcat.
>
> In order to be able to server all Apache connections simultaneously, I
> would need 4 more Tomcats?
>
>
>
>
>
>
> 2010/11/5 Mark Thomas <ma...@apache.org>
>
> On 04/11/2010 19:27, Marc Wilmots wrote:
>> > The best solution would be to enable DisableReuse as Mark stated,
>> although
>> > I'm afraid that would be too big of a performance hit.
>>
>> Have you tried it?
>>
>> The cost of the required CPING/CPONG isn't that different to the cost of
>> creating a TCP/IP connection. On a fast local area network I doubt you'd
>> notice it. The time taken to establish a TCP/IP connection is usually
>> (YMMV so test it) << the time taken to process a request.
>>
>> Mark
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
Re: mod_jk makes ajp connector run out of free connections
Posted by Marc Wilmots <de...@gmail.com>.
Honestly....no, I didn't.
The mod_jk scared the hell out of me....wouldn't you? :-)
"Using this option will have a strong performance penalty for Apache and
Tomcat. Use this only as a last resort in case of unfixable network
problems."
I'll give it a try tomorrow and see what it gives.
By the way, my last calculation about connection_pool_size was wrong. I
forgot to include the second Apache server. The math would look like this
then:
Apache1 -> 10 server processes x 15 x 4 (tomcats) = 600
Apache2 -> 10 server processes x 15 x 4 (tomcats) = 600, which makes 1200 in
total
Each Tomcat would receive a maximum of 300 connections, which would leave
100 free connections per Tomcat. This would solve the problem that Tomcat
runs out of connections. The thing is...not all of my available Apache
connections can be used:
Apache has 60 threads per server process (10) -> 600 connections.
connection_pool_size per server process: 15. This would mean only 15 of 60
possible connections per server process can actually establish a connection
to Tomcat.
In order to be able to server all Apache connections simultaneously, I would
need 4 more Tomcats?
2010/11/5 Mark Thomas <ma...@apache.org>
> On 04/11/2010 19:27, Marc Wilmots wrote:
> > The best solution would be to enable DisableReuse as Mark stated,
> although
> > I'm afraid that would be too big of a performance hit.
>
> Have you tried it?
>
> The cost of the required CPING/CPONG isn't that different to the cost of
> creating a TCP/IP connection. On a fast local area network I doubt you'd
> notice it. The time taken to establish a TCP/IP connection is usually
> (YMMV so test it) << the time taken to process a request.
>
> Mark
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: mod_jk makes ajp connector run out of free connections
Posted by Mark Thomas <ma...@apache.org>.
On 04/11/2010 19:27, Marc Wilmots wrote:
> The best solution would be to enable DisableReuse as Mark stated, although
> I'm afraid that would be too big of a performance hit.
Have you tried it?
The cost of the required CPING/CPONG isn't that different to the cost of
creating a TCP/IP connection. On a fast local area network I doubt you'd
notice it. The time taken to establish a TCP/IP connection is usually
(YMMV so test it) << the time taken to process a request.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk makes ajp connector run out of free connections
Posted by Marc Wilmots <de...@gmail.com>.
I just got my own question answered whether connection_pool_size sets a pool
per server process.
"This defines the number of connections made to the AJP backend that are
maintained as a connection pool. It will limit the number of those
connection that each web server child process can made."
This still leaves me with the question; 1200 (apache connections) is still
bigger than 600 (15 x 10 server processes x 4 tomcat nodes)
2010/11/5 Marc Wilmots <de...@gmail.com>
> Rainer, that's very true.
> So I am wondering...If 1200 > 200 (Tomcat maxThreads), then what would be
> the right way to solve this? Imagine that 2 Tomcats would crash, that would
> leave me with 1200 > 400. I can't just raise the maxThreads to 600 for each
> Tomcat, as that would probably perform even worse (I'm thinking about
> excessive context switching, etc).
>
> The best solution would be to enable DisableReuse as Mark stated, although
> I'm afraid that would be too big of a performance hit.
> The other option would be to configure the connection_pool_size, according
> to Mladen's link. Although, this last one isn't very clear to me either. I
> state from the URL Mladen passed:
>
> *"There are 4 JBoss servers each containing 1 JBoss instance(s) giving a
> total of 4 mod_jk workers*
>
> *Given the ThreadsPerChild of 60 and the MaxClients (600) /
> ThreadsPerChild (60) = Processes (10) and with 2 Apache instance(s) then
> the connection_pool_size has been best determined to be 15*
>
> *The total Jboss threads 1600 are > the total Apache threads 1200. The
> mod_jk connection_pool_size was determined to be 15 via the math 600Apache MaxClients /
> 4 total Jboss instances 150 threads per JBoss which will be equal to or
> less than the maxThreads 400 150 / 10 Apache server processes to get the
> connection_pool_size of 15. in Jboss. Take this number"*
>
> I'm wondering, setting connection_pool_size 15, wouldn't this mean that the
> maximum number of connections being handled by tomcat at one time is limited
> to 15?? Or would there be a pool per Apache server process? Which would mean
> 150 connections (10 server processes) per Tomcat. If so, then this would
> total 600 connections (4 Tomcats).
>
> 1200 > 600. Same problem, right?
>
> 2010/10/28 Rainer Jung <ra...@kippdata.de>
>
> On 26.10.2010 20:36, Marc Wilmots wrote:
>>
>>> 2010/10/26 Mladen Turk<mt...@apache.org>
>>>
>>> On 10/26/2010 02:47 PM, Marc Wilmots wrote:
>>>>
>>>> Hi Experts,
>>>>>
>>>>> I have the following setup:
>>>>>
>>>>> Apache1
>>>>> / \
>>>>> Apache2 Apache3
>>>>> \ /
>>>>> Tomcat1
>>>>>
>>>>> All Apaches are version 2.2.3 (RedHat)
>>>>> Apache2 and Apache3 loadbalance Tomcat1 (6.0.18) with mod_jk (2.2.28).
>>>>>
>>>>> In idle state the AJP connector of Tomcat1 only has 7 active
>>>>> connections.
>>>>> After launching a stress test of Tomcat1, it's AJP connector has
>>>>> reached
>>>>> maxThreads (200). After the stress test has finished, there are still
>>>>> 200
>>>>> active connections in the AJP connector.
>>>>>
>>>>> Because of this, apache2 and apache2 cannot receive any heartbeat
>>>>> message
>>>>> anymore from the AJP connector and mark Tomcat1 as dead. I can access
>>>>> perfectly through port 8080, so Tomcat1 isn't dead at all!
>>>>>
>>>>>
>>>>> You have a usual setup problem where the number of connections
>>>> in httpd fronting tomcat is too high for a tomcat.
>>>> And you don't have any mechanism for limiting those connections.
>>>>
>>>> Sorry, I realize I had to mention I have a total of 4 Tomcats, which in
>>>>
>>> total make up a maxThreads of 800 (200 each).
>>> The Apaches have a maxClients of each 600 (1200 total).
>>>
>>> 1200> 1000, thus your theory is probably right. However, I'm just
>>> wondering... I read that when Tomcat reaches the max number of
>>> connections,
>>> just rejects new connections (I'm omitting the backlog on purpose).
>>>
>>
>> Yes.
>>
>> But your calculation above is not right. Each Apache will connect to each
>> Tomcat. So from the point of view of one Tomcat, it has 200 threads, but
>> 1200 posible incoming connections from the Apache Servers. So it is not 1200
>> > 800 but 1200 > 200!
>>
>> Regards,
>>
>> Rainer
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>