You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Rick Noel <RN...@westwoodone.com.INVALID> on 2024/03/28 19:38:49 UTC
RE: [EXT]Re: [EXT]Re: performance tunning of Tomcat 10
Chris,
my bottleneck (app slowness to respond is the at the database connection max), I am not even going to try and adjust minThreads and maxThreads.
When I tried to increase maxConnection to our database from and existing connection max of 20, to a new connectionMax of 50, I got errors, see attached snapshot.
I was able to raise the maxConnection to database up to 30 with no errors, so I am just going to deploy at that level until we upgrade our Postgess database
We are using a really old Postgres and plan in the near to future to upgrade that database.
BTW,
I did find and use the Apache Benchmark tool and was able to see how our app responses with a test requests of 10000 and concurrent requests of 40 and those results look good to me with worst response time of only 1000 miliseconds.
Rick Noel
Systems Programmer | Westwood One
RNoel@westwoodone.com
-----Original Message-----
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Thursday, March 28, 2024 3:09 PM
To: users@tomcat.apache.org
Subject: [EXT]Re: [EXT]Re: performance tunning of Tomcat 10
Rick,
On 3/27/24 09:22, Rick Noel wrote:
> -----Original Message-----
> From: Christopher Schultz <ch...@christopherschultz.net>
> Sent: Wednesday, March 27, 2024 8:24 AM
> To: Tomcat Users List <us...@tomcat.apache.org>; Rick Noel
> <RN...@westwoodone.com.INVALID>
> Cc: Voodoo nmulcahy gmail <nm...@gmail.com>; David Jung
> <Da...@cumulus.com>
> Subject: [EXT]Re: performance tunning of Tomcat 10
>
> Rick,
>
> On 3/27/24 07:53, Rick Noel wrote:
>> I was wondering if the apache foundation has any tools we can use to
>> fine tune Tomcat 10. Tools to deteming how to set the best heap size
>> for Tomcat startup and the best connection attributes of
>> minSpareThreads and MaxThreads.
> What is your goal?
> Our application is a sip phone call handling application.(A voice
> response xml application) The goal is to not have call bottleneck at peak call volume.
> Sometimes too many folks call at one time hit our app and calls are not handle correctly.
>
>> I know my application at times will reach 100 concurrent connections
> > and some times goes has high as 500 connections.
>
> Okay.
> Well I do not have actual traffic info down to the sec, But from the
> application logs I know that that points in the day more than 300
> calls can come in
Okay, you need to get better information. Check out this presentation on Tomcat monitoring:
https://tomcat.apache.org/presentations.html#latest-monitoring-with-jmx
There is a lot in there, but I do talk about checking on the request processor to see how many requests are in-flight at once. You can do the same with your database pool.
You'll want to grab samples of those things at regular intervals to see how they correlate. I suspect you know when your peak times are, so schedule the samples to be taken during that timeframe. You could even sample once per second if you wanted to -- the calls are pretty inexpensive.
>> Should I boost minSpareThreads and maxThread values of what I plan to
>> use below? > Or why would we not just set very high minSpareThreads
>> and maxThread values like minSpareThreads =300 and maxThread=1000
>>
>> This is a snippet of my server.xml
>>
>> <Executor name="tomcatPhoneAppThreadPool" namePrefix="catalina-executor-"
>> minSpareThreads="50"
>> maxThreads="300" />
>>
>> <Connector port="8585"
>> executor="tomcatPhoneAppThreadPool"
>> compression="on"
>> compressionMinSize="2048"
>> compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml"
>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>> redirectPort="8443">
>> <UpgradeProtocol
>> className="org.apache.coyote.http2.Http2Protocol" />
>>
>> </Connector>
>>
>> Also, am I good with setting protocol="org.apache.coyote.http11.Http11NioProtocol"
>> And then setting <UpgradeProtocol
>> className="org.apache.coyote.http2.Http2Protocol" />
>>
>> That will tell Tomcat to do HTTP2 Correct?
>
> That's the only way to enable h2. Well... you could use Http11Nio2Protocol, too. NIO is the default protocol so you don't even need to add that specifically if you don't want to.
>
> Back to threads.
>
> Each thread (unless you go virtual, but that's not really production-ready IMHO at this point unless you have very strict circumstances where it will work great for you) takes up a bunch of memory, so you can't just set maxThreads=1M. Threads take "time" to start, but it's not really that much. If starting and stopping threads is what is making your application slow, than you have a very high-performance application and environment indeed.
>
> Your question as stated is unanswerable.
>
> You say you are sometimes hitting 500 connections. The default maximum number of connections is 10000 and you are only using 500. That means you aren't being flooded, which is a Good Thing. (BTW: How are you measuring "how many connections" you have? Make sure you are measuring the right thing...
>
> I am estimating the actual connections just based on application call logs.
>
> Is your *current* maxThreads set to 500? If so, then your thread pool maximum is set to your high-water mark which seems like it should be fine. If you set your maxThreads to 1000 you won't get any benefit because only 500 requests are ever being sent at once, right?
>
> What else does your application do?
> Our application is a sip phone call handling application.(A voice
> response xml application)
>
> For example, if you have a thread-pool max-threads of 1000 and your application uses an RDBMS for every request but your db connection pool size is more like 10, then many threads waiting on a small number of connections gets you absolutely no benefit. You'd have to make changes elsewhere in your application in order to make use of those extra threads.
>
> Similarly, if you have a big thread pool and a big db connection pool, but your database performs slowly, then having all that power on the application server doesn't really help you.
>
> Yes our app is using a database (we use Postres) And yes I assume
> that is our bottleneck Our tomcat context.xml defines that database
> maxConnction to be only maxTotal="20"
So if you are allowing up to 300 or 500 threads and most of them need a database connection, then you are really limiting yourself to more like
20 simultaneous requests served.
You will likely need to increase that db connection pool size.
> We have other apps and that hit this same database, so we define low
> maxTotal on each server so each server can have only limit
> connection access to the database
Well...
> I am thinkingit would be best to try and NOT optimize minSpareThreads
> and maxThreads but instead to determine what is the optimal Database
> connection maxTotal to set
I agree. And if your database can't handle it, then you need to invest more $$$ in your database, or re-think how to store data.
One way to re-think things is to consider replication/clustering of your existing database. I'm not super familiar with PostgreSQL but I suspect you can use multi-primary deployments and read-only replicas as long as your application understands how to interact properly with that kind of deployment.
Another option would be to use a different kind of database like Cassandra or whatever which is optimized for writes (if you are write-heavy). You might also be able to segment your data differently, so information about current in-progress calls goes to a high-performance but small database (possibly in-memory) and longer-term storage can be used for things that need to out-last the phone call such as statistics and stuff like that.
> So anyone trying to answer "how big should be thread pool be" really needs to understand the nature of your application and the other things happening in your environment.
>
> Sometimes the answer is "just add more threads/CPUs/memory" and sometimes the answer is "re-think your application architecture".
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you know the sender and you are sure the content is safe. Please report the message using the Report Message feature in your email client if you believe the email is suspicious.
Re: [EXT]Re: [EXT]Re: performance tunning of Tomcat 10
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Rick,
On 3/28/24 15:38, Rick Noel wrote:
> my bottleneck (app slowness to respond is the at the database
> connection max), I am not even going to try and adjust minThreads and
> maxThreads. When I tried to increase maxConnection to our database
> from and existing connection max of 20, to a new connectionMax of 50,
> I got errors, see attached snapshot.
This list strips most attachments including images. Please post text.
> I was able to raise the maxConnection to database up to 30 with no
> errors, so I am just going to deploy at that level until we upgrade our
> Postgess database
> We are using a really old Postgres and plan in the near to future to
> upgrade that database.
It's not the software version that matters, it's the (possibly virtual)
hardware.
> BTW,
> I did find and use the Apache Benchmark tool and was able to see how
> our app responses with a test requests of 10000 and concurrent
> requests of 40 and those results look good to me with worst response
> time of only 1000 miliseconds.
That's great for 40 simultaneous requests, but what happens when you
crank it up to 300? Or 500? Or whatever?
You may want to use JMeter which is a better load-testing tool than 'ab'
once you get beyond just throwing X requests at a site.
-chris
> -----Original Message-----
> From: Christopher Schultz <ch...@christopherschultz.net>
> Sent: Thursday, March 28, 2024 3:09 PM
> To: users@tomcat.apache.org
> Subject: [EXT]Re: [EXT]Re: performance tunning of Tomcat 10
>
> Rick,
>
> On 3/27/24 09:22, Rick Noel wrote:
>> -----Original Message-----
>> From: Christopher Schultz <ch...@christopherschultz.net>
>> Sent: Wednesday, March 27, 2024 8:24 AM
>> To: Tomcat Users List <us...@tomcat.apache.org>; Rick Noel
>> <RN...@westwoodone.com.INVALID>
>> Cc: Voodoo nmulcahy gmail <nm...@gmail.com>; David Jung
>> <Da...@cumulus.com>
>> Subject: [EXT]Re: performance tunning of Tomcat 10
>>
>> Rick,
>>
>> On 3/27/24 07:53, Rick Noel wrote:
>>> I was wondering if the apache foundation has any tools we can use to
>>> fine tune Tomcat 10. Tools to deteming how to set the best heap size
>>> for Tomcat startup and the best connection attributes of
>>> minSpareThreads and MaxThreads.
>> What is your goal?
>> Our application is a sip phone call handling application.(A voice
>> response xml application) The goal is to not have call bottleneck at peak call volume.
>> Sometimes too many folks call at one time hit our app and calls are not handle correctly.
>>
>>> I know my application at times will reach 100 concurrent connections
>> > and some times goes has high as 500 connections.
>>
>> Okay.
>> Well I do not have actual traffic info down to the sec, But from the
>> application logs I know that that points in the day more than 300
>> calls can come in
>
> Okay, you need to get better information. Check out this presentation on Tomcat monitoring:
>
> https://tomcat.apache.org/presentations.html#latest-monitoring-with-jmx
>
> There is a lot in there, but I do talk about checking on the request processor to see how many requests are in-flight at once. You can do the same with your database pool.
>
> You'll want to grab samples of those things at regular intervals to see how they correlate. I suspect you know when your peak times are, so schedule the samples to be taken during that timeframe. You could even sample once per second if you wanted to -- the calls are pretty inexpensive.
>
>>> Should I boost minSpareThreads and maxThread values of what I plan to
>>> use below? > Or why would we not just set very high minSpareThreads
>>> and maxThread values like minSpareThreads =300 and maxThread=1000
>>>
>>> This is a snippet of my server.xml
>>>
>>> <Executor name="tomcatPhoneAppThreadPool" namePrefix="catalina-executor-"
>>> minSpareThreads="50"
>>> maxThreads="300" />
>>>
>>> <Connector port="8585"
>>> executor="tomcatPhoneAppThreadPool"
>>> compression="on"
>>> compressionMinSize="2048"
>>> compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml"
>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>> redirectPort="8443">
>>> <UpgradeProtocol
>>> className="org.apache.coyote.http2.Http2Protocol" />
>>>
>>> </Connector>
>>>
>>> Also, am I good with setting protocol="org.apache.coyote.http11.Http11NioProtocol"
>>> And then setting <UpgradeProtocol
>>> className="org.apache.coyote.http2.Http2Protocol" />
>>>
>>> That will tell Tomcat to do HTTP2 Correct?
>>
>> That's the only way to enable h2. Well... you could use Http11Nio2Protocol, too. NIO is the default protocol so you don't even need to add that specifically if you don't want to.
>>
>> Back to threads.
>>
>> Each thread (unless you go virtual, but that's not really production-ready IMHO at this point unless you have very strict circumstances where it will work great for you) takes up a bunch of memory, so you can't just set maxThreads=1M. Threads take "time" to start, but it's not really that much. If starting and stopping threads is what is making your application slow, than you have a very high-performance application and environment indeed.
>>
>> Your question as stated is unanswerable.
>>
>> You say you are sometimes hitting 500 connections. The default maximum number of connections is 10000 and you are only using 500. That means you aren't being flooded, which is a Good Thing. (BTW: How are you measuring "how many connections" you have? Make sure you are measuring the right thing...
>>
>> I am estimating the actual connections just based on application call logs.
>>
>> Is your *current* maxThreads set to 500? If so, then your thread pool maximum is set to your high-water mark which seems like it should be fine. If you set your maxThreads to 1000 you won't get any benefit because only 500 requests are ever being sent at once, right?
>>
>> What else does your application do?
>> Our application is a sip phone call handling application.(A voice
>> response xml application)
>>
>> For example, if you have a thread-pool max-threads of 1000 and your application uses an RDBMS for every request but your db connection pool size is more like 10, then many threads waiting on a small number of connections gets you absolutely no benefit. You'd have to make changes elsewhere in your application in order to make use of those extra threads.
>>
>> Similarly, if you have a big thread pool and a big db connection pool, but your database performs slowly, then having all that power on the application server doesn't really help you.
>>
>> Yes our app is using a database (we use Postres) And yes I assume
>> that is our bottleneck Our tomcat context.xml defines that database
>> maxConnction to be only maxTotal="20"
>
> So if you are allowing up to 300 or 500 threads and most of them need a database connection, then you are really limiting yourself to more like
> 20 simultaneous requests served.
>
> You will likely need to increase that db connection pool size.
>
>> We have other apps and that hit this same database, so we define low
>> maxTotal on each server so each server can have only limit
>> connection access to the database
>
> Well...
>
>> I am thinkingit would be best to try and NOT optimize minSpareThreads
>> and maxThreads but instead to determine what is the optimal Database
>> connection maxTotal to set
>
> I agree. And if your database can't handle it, then you need to invest more $$$ in your database, or re-think how to store data.
>
> One way to re-think things is to consider replication/clustering of your existing database. I'm not super familiar with PostgreSQL but I suspect you can use multi-primary deployments and read-only replicas as long as your application understands how to interact properly with that kind of deployment.
>
> Another option would be to use a different kind of database like Cassandra or whatever which is optimized for writes (if you are write-heavy). You might also be able to segment your data differently, so information about current in-progress calls goes to a high-performance but small database (possibly in-memory) and longer-term storage can be used for things that need to out-last the phone call such as statistics and stuff like that.
>
>> So anyone trying to answer "how big should be thread pool be" really needs to understand the nature of your application and the other things happening in your environment.
>>
>> Sometimes the answer is "just add more threads/CPUs/memory" and sometimes the answer is "re-think your application architecture".
>
> -chris
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you know the sender and you are sure the content is safe. Please report the message using the Report Message feature in your email client if you believe the email is suspicious.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org