You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Piyush Kumar Nayak <pn...@adobe.com.INVALID> on 2017/06/06 06:30:53 UTC
Tomcat 8/NIO performance discrepancies
We have been testing different connectors of tomcat for performance.
We have tested the following server versions:
- Tomcat 7.077
- Tomcat 8.0.44
- Tomcat 8.0.32
- Tomcat 8.5.14
with the following HTTP connectors:
- BIO
- BIO with Executor
- NIO
- NIO with Executor
Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
I would appreciate any help with the following:
- Why is the executor not boosting the performance for NIO, as it is for BIO?
- BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?
I'd be happy to share any other details from my testing for any clarification.
Regards,
Piyush.
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Mark,
I apologize for not bringing this up before, but is that BIO vs NIO or BIO with executor vs NIO with executor?
Thanks again, for looking into this.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Friday, June 09, 2017 12:30 AM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
On 08/06/17 14:42, Mark Thomas wrote:
> I'll look at BIO vs NIO next.
At the sort of request rates and response sizes we are talking about, we are a long way from real-world applications and the differences you see between BIO and NIO are largely down to the differences between doing a blocking read for the next request (BIO) and polling (NIO).
What you will notice is that as the number of concurrent clients rises, BIO performance will drop considerably as soon as the disableKeepAlivePercentage is exceeded and will continue to fall as client concurrency rises. NIO, on the other hand, will retain broadly the same performance even when client concurrency >> threads.
I have spent some time looking at this in detail with a profiler but the timing differences we are talking about are so small that often the profiler overhead ends up dominating.
I think the general lesson here is that benchmarks like this can be interesting but the further removed they are from real-world examples, the less useful they are likely to be.
As you move towards more real-world examples, what you tend to find is that I/O and/or application logic become far more dominant.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 08/06/17 14:42, Mark Thomas wrote:
> I'll look at BIO vs NIO next.
At the sort of request rates and response sizes we are talking about, we
are a long way from real-world applications and the differences you see
between BIO and NIO are largely down to the differences between doing a
blocking read for the next request (BIO) and polling (NIO).
What you will notice is that as the number of concurrent clients rises,
BIO performance will drop considerably as soon as the
disableKeepAlivePercentage is exceeded and will continue to fall as
client concurrency rises. NIO, on the other hand, will retain broadly
the same performance even when client concurrency >> threads.
I have spent some time looking at this in detail with a profiler but the
timing differences we are talking about are so small that often the
profiler overhead ends up dominating.
I think the general lesson here is that benchmarks like this can be
interesting but the further removed they are from real-world examples,
the less useful they are likely to be.
As you move towards more real-world examples, what you tend to find is
that I/O and/or application logic become far more dominant.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 6/12/17 3:43 PM, Piyush Kumar Nayak wrote:
> Sure, I'll check it out and get back. But the last time I tried,
> JMeter gave me better numbers than ab. Don't want the client to be
> the bottleneck.
+1
ab evidently doesn't "do" high-concurrency very well. Something about
a single global-lock. Mladen and Jean-Frederic have more information
about it than I do.
- -chris
> -----Original Message----- From: Mark Thomas
> [mailto:markt@apache.org] Sent: Tuesday, June 13, 2017 1:05 AM To:
> Tomcat Users List <us...@tomcat.apache.org> Subject: Re: Tomcat
> 8/NIO performance discrepancies
>
> On 12/06/17 20:27, Piyush Kumar Nayak wrote:
>> Thanks, Mark. I have been testing with 100 concurrent users.
>> While the executor had the maxThreads set to 150, the connector
>> without the executor was left to the default value (which should
>> be 200). I did not have the disableKeepAlivePercentage attribute
>> set, which means that it should take to the default value of 75.
>>
>> So, the number of threads in use before the keepalive is disabled
>> in case of non-executor connector should be 150. At 100 users,
>> the no. of clients is at 50% of the maxThread count. Unless I'm
>> missing something here, I don't see why keep-alive would be
>> disabled. I'd appreciate if you can give me some pointers on how
>> you can tell if requests are processed at the server end with
>> keep-alive.
>
> I suggest you use ab rather than JMeter. I've seen all sorts of odd
> behaviour with JMeter when processing requests very quickly that I
> haven't seen with ab. Also, ab tells you how many of your requests
> used keep-alive.
>
> Mark
>
>
>>
>> I'll try setting disableKeepAlivePercentage to 100, and see if
>> that improves the performance.
>>
>> -----Original Message----- From: Mark Thomas
>> [mailto:markt@apache.org] Sent: Thursday, June 08, 2017 7:13 PM
>> To: Tomcat Users List <us...@tomcat.apache.org> Subject: Re:
>> Tomcat 8/NIO performance discrepancies
>>
>> On 07/06/17 22:28, Mark Thomas wrote:
>>
>> <snip/>
>>
>>> I'm still seeing significant differences in BIO with and
>>> without an executor.
>>>
>>> I'm still looking for the root cause. I have noticed that
>>> without an executor I see far fewer keep-alive requests than
>>> I'd expect. That could explain the difference, or some of it at
>>> least.
>>
>> Further testing indicates that most, if not all, of the
>> performance difference is down to the fewer number of requests
>> processed using HTTP keep-alive. This is because BIO without an
>> executor is subject to disableKeepAlivePercentage whereas BIO
>> with an executor is not.
>>
>> If the number of concurrent clients is kept below 75% of
>> maxThreads, performance remains very similar.
>>
>> The problems you are likely to experience in the real world
>> without disableKeepAlivePercentage (i.e. if you use BIO with an
>> executor) are described in this thread:
>>
>> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomca
t
>>
>>
.markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122
>> aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6
3
>>
>>
6325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&
>> reserved=0
>>
>> The better tuning options are probably: - increase maxThreads to
>> max expected client concurrency + headroom or if your memory/CPU
>> won't permit that, as high as you can - increase
>> disableKeepAlivePercentage
>>
>> I'll look at BIO vs NIO next.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZPvXxAAoJEBzwKT+lPKRYCb8QAL555dkQlFCPEXsjqtP7IGIA
7eQjX7gE1Lxk3ycCmTcndyixFWGoGdLwp41OPadVYF9TSe003fiHbptiSsYgCCEM
HEOMisKNkVZg22Wyenm0eJLPaALMOE7xU95rmPpacVxgslPWmMLSsNk9TC/c2xQ1
1o7iiS9/hEF6LsoT5oOLxe/ZGp8jxTHXBgahvqCierzN8T089G7GomVS1Q/FDD2t
OLOu/JRcTUKzxlISONbchtF1fSc9HenLmrZxqVw52iZGLGrbsIAg8D/x7kPhAxbk
HLHG9GwkHh7iOavW25UWFVVlmC58XioDJJVZWTGzEhfxAU/zoyUZbZ2CtqSDTdUG
rcsBjk33A1LYc3yioq8i55fq7d84cLdG2Xf//kUaBq+yP3tuNpWYk/WEOL/kuQFp
IY1z+QykqF1Agfiubc8GyDWhyBvi2+g0uX09796STLfQdk1eC6+aPIpFOtrTh2ou
a8H73w73nJ2R04GnqBm9T+RFJSvjR9HGZ8mGoFYypMeK0qJXOn7t1CVK6/bX7d9n
J4Op5AyncpSlnqRCofMk4zSYGUasKR6BXtZ+2pu+5McnjndHLnUdGAfwy9ReD5fi
5McU18S7EUlvRaXt5pgO9RSDRilm3J+Ar0rGFlSBNmanFTvlgCdKYPTAWmkTcbGW
AOvM4/3itRoZi7DBtEdg
=0H+N
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Sure, I'll check it out and get back.
But the last time I tried, JMeter gave me better numbers than ab. Don't want the client to be the bottleneck.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Tuesday, June 13, 2017 1:05 AM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
On 12/06/17 20:27, Piyush Kumar Nayak wrote:
> Thanks, Mark.
> I have been testing with 100 concurrent users. While the executor had the maxThreads set to 150, the connector without the executor was left to the default value (which should be 200). I did not have the disableKeepAlivePercentage attribute set, which means that it should take to the default value of 75.
>
> So, the number of threads in use before the keepalive is disabled in case of non-executor connector should be 150. At 100 users, the no. of clients is at 50% of the maxThread count. Unless I'm missing something here, I don't see why keep-alive would be disabled.
> I'd appreciate if you can give me some pointers on how you can tell if requests are processed at the server end with keep-alive.
I suggest you use ab rather than JMeter. I've seen all sorts of odd behaviour with JMeter when processing requests very quickly that I haven't seen with ab. Also, ab tells you how many of your requests used keep-alive.
Mark
>
> I'll try setting disableKeepAlivePercentage to 100, and see if that improves the performance.
>
> -----Original Message-----
> From: Mark Thomas [mailto:markt@apache.org]
> Sent: Thursday, June 08, 2017 7:13 PM
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
> On 07/06/17 22:28, Mark Thomas wrote:
>
> <snip/>
>
>> I'm still seeing significant differences in BIO with and without an
>> executor.
>>
>> I'm still looking for the root cause. I have noticed that without an
>> executor I see far fewer keep-alive requests than I'd expect. That
>> could explain the difference, or some of it at least.
>
> Further testing indicates that most, if not all, of the performance difference is down to the fewer number of requests processed using HTTP keep-alive. This is because BIO without an executor is subject to disableKeepAlivePercentage whereas BIO with an executor is not.
>
> If the number of concurrent clients is kept below 75% of maxThreads, performance remains very similar.
>
> The problems you are likely to experience in the real world without disableKeepAlivePercentage (i.e. if you use BIO with an executor) are described in this thread:
>
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomcat
> .markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122
> aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63
> 6325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&
> reserved=0
>
> The better tuning options are probably:
> - increase maxThreads to max expected client concurrency + headroom
> or if your memory/CPU won't permit that, as high as you can
> - increase disableKeepAlivePercentage
>
> I'll look at BIO vs NIO next.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 12/06/17 20:27, Piyush Kumar Nayak wrote:
> Thanks, Mark.
> I have been testing with 100 concurrent users. While the executor had the maxThreads set to 150, the connector without the executor was left to the default value (which should be 200). I did not have the disableKeepAlivePercentage attribute set, which means that it should take to the default value of 75.
>
> So, the number of threads in use before the keepalive is disabled in case of non-executor connector should be 150. At 100 users, the no. of clients is at 50% of the maxThread count. Unless I'm missing something here, I don't see why keep-alive would be disabled.
> I'd appreciate if you can give me some pointers on how you can tell if requests are processed at the server end with keep-alive.
I suggest you use ab rather than JMeter. I've seen all sorts of odd
behaviour with JMeter when processing requests very quickly that I
haven't seen with ab. Also, ab tells you how many of your requests used
keep-alive.
Mark
>
> I'll try setting disableKeepAlivePercentage to 100, and see if that improves the performance.
>
> -----Original Message-----
> From: Mark Thomas [mailto:markt@apache.org]
> Sent: Thursday, June 08, 2017 7:13 PM
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
> On 07/06/17 22:28, Mark Thomas wrote:
>
> <snip/>
>
>> I'm still seeing significant differences in BIO with and without an
>> executor.
>>
>> I'm still looking for the root cause. I have noticed that without an
>> executor I see far fewer keep-alive requests than I'd expect. That
>> could explain the difference, or some of it at least.
>
> Further testing indicates that most, if not all, of the performance difference is down to the fewer number of requests processed using HTTP keep-alive. This is because BIO without an executor is subject to disableKeepAlivePercentage whereas BIO with an executor is not.
>
> If the number of concurrent clients is kept below 75% of maxThreads, performance remains very similar.
>
> The problems you are likely to experience in the real world without disableKeepAlivePercentage (i.e. if you use BIO with an executor) are described in this thread:
>
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomcat.markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&reserved=0
>
> The better tuning options are probably:
> - increase maxThreads to max expected client concurrency + headroom
> or if your memory/CPU won't permit that, as high as you can
> - increase disableKeepAlivePercentage
>
> I'll look at BIO vs NIO next.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Thanks, Mark.
I have been testing with 100 concurrent users. While the executor had the maxThreads set to 150, the connector without the executor was left to the default value (which should be 200). I did not have the disableKeepAlivePercentage attribute set, which means that it should take to the default value of 75.
So, the number of threads in use before the keepalive is disabled in case of non-executor connector should be 150. At 100 users, the no. of clients is at 50% of the maxThread count. Unless I'm missing something here, I don't see why keep-alive would be disabled.
I'd appreciate if you can give me some pointers on how you can tell if requests are processed at the server end with keep-alive.
I'll try setting disableKeepAlivePercentage to 100, and see if that improves the performance.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Thursday, June 08, 2017 7:13 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
On 07/06/17 22:28, Mark Thomas wrote:
<snip/>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That
> could explain the difference, or some of it at least.
Further testing indicates that most, if not all, of the performance difference is down to the fewer number of requests processed using HTTP keep-alive. This is because BIO without an executor is subject to disableKeepAlivePercentage whereas BIO with an executor is not.
If the number of concurrent clients is kept below 75% of maxThreads, performance remains very similar.
The problems you are likely to experience in the real world without disableKeepAlivePercentage (i.e. if you use BIO with an executor) are described in this thread:
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftomcat.markmail.org%2Fthread%2Fmcfdk2273v2fsh5k&data=02%7C01%7C%7C6816158122aa4914b20208d4ae744a1e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636325261860429837&sdata=8t1j6lxgc3aTCDq45mXbH7LqedtIuRgPsrgqUKXyCrc%3D&reserved=0
The better tuning options are probably:
- increase maxThreads to max expected client concurrency + headroom
or if your memory/CPU won't permit that, as high as you can
- increase disableKeepAlivePercentage
I'll look at BIO vs NIO next.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 07/06/17 22:28, Mark Thomas wrote:
<snip/>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
Further testing indicates that most, if not all, of the performance
difference is down to the fewer number of requests processed using HTTP
keep-alive. This is because BIO without an executor is subject to
disableKeepAlivePercentage whereas BIO with an executor is not.
If the number of concurrent clients is kept below 75% of maxThreads,
performance remains very similar.
The problems you are likely to experience in the real world without
disableKeepAlivePercentage (i.e. if you use BIO with an executor) are
described in this thread:
http://tomcat.markmail.org/thread/mcfdk2273v2fsh5k
The better tuning options are probably:
- increase maxThreads to max expected client concurrency + headroom
or if your memory/CPU won't permit that, as high as you can
- increase disableKeepAlivePercentage
I'll look at BIO vs NIO next.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by km...@gmail.com.
Envoyé de mon smartphone BlackBerry 10.
Message d'origine
De: kmaxwilliams43@gmail.com
Envoyé: vendredi 9 juin 2017 00:12
À: Astrat Goe; Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies
Gvhuf
Envoyé de mon smartphone BlackBerry 10.
Message d'origine
De: Astrat Goe
Envoyé: mercredi 7 juin 2017 21:29
À: Tomcat Users List
Répondre à: Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies
Ggfggthgd
Vid
> On Jun 8, 2017, at 4:28 AM, Mark Thomas <ma...@apache.org> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:markt@apache.org]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <us...@tomcat.apache.org>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>> maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by km...@gmail.com.
Gvhuf
Envoyé de mon smartphone BlackBerry 10.
Message d'origine
De: Astrat Goe
Envoyé: mercredi 7 juin 2017 21:29
À: Tomcat Users List
Répondre à: Tomcat Users List
Objet: Re: Tomcat 8/NIO performance discrepancies
> On Jun 8, 2017, at 4:28 AM, Mark Thomas <ma...@apache.org> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:markt@apache.org]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <us...@tomcat.apache.org>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>> maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Astrat Goe <as...@gmail.com>.
> On Jun 8, 2017, at 4:28 AM, Mark Thomas <ma...@apache.org> wrote:
>
> On 07/06/17 20:25, Piyush Kumar Nayak wrote:
>> Mark,
>>
>> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
>
> I'm aware you were using two machines but that doesn't appear to be a
> significant factor in the observed differences.
>
>> Can you also please provide some pointers on the following:
>> - How did you disable session on your JSP?
>
> Look in the index.jsp for the ROOT web application that ships with
> Tomcat for an example. Full details are in the JSP specification.
>
>> - Did you observe anything that suggests that the client is not closing the connections?
>
> Yes. This looks like a JMeter issue. It isn't the first time JMeter has
> given strange results under load. I've switched to ab which I have found
> to be generally much more reliable for these sorts of tests.
>
> I'm still seeing significant differences in BIO with and without an
> executor.
>
> I'm still looking for the root cause. I have noticed that without an
> executor I see far fewer keep-alive requests than I'd expect. That could
> explain the difference, or some of it at least.
>
> Mark
>
>
>> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>>
>> Thanks
>> Piyush.
>>
>> -----Original Message-----
>> From: Mark Thomas [mailto:markt@apache.org]
>> Sent: Wednesday, June 07, 2017 8:27 PM
>> To: Tomcat Users List <us...@tomcat.apache.org>
>> Subject: Re: Tomcat 8/NIO performance discrepancies
>>
>> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>>> Thanks, Mark.
>>>
>>> Here are some additional details.
>>
>> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>>
>>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>>
>>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>>
>> I've disabled session in the JSP for my test and used a simpler JSP.
>> That should amplify any difference in connector performance.
>>
>> I'm also testing on localhost.
>>
>>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>>
>>>
>>> The BIO with Executor configuration we are using is:
>>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>> maxThreads="150" minSpareThreads="4"/>
>>>
>>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>>> connectionTimeout="20000"
>>> redirectPort="8443" />
>>
>> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>>
>> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>>
>> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>>
>> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 07/06/17 20:25, Piyush Kumar Nayak wrote:
> Mark,
>
> It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
I'm aware you were using two machines but that doesn't appear to be a
significant factor in the observed differences.
> Can you also please provide some pointers on the following:
> - How did you disable session on your JSP?
Look in the index.jsp for the ROOT web application that ships with
Tomcat for an example. Full details are in the JSP specification.
> - Did you observe anything that suggests that the client is not closing the connections?
Yes. This looks like a JMeter issue. It isn't the first time JMeter has
given strange results under load. I've switched to ab which I have found
to be generally much more reliable for these sorts of tests.
I'm still seeing significant differences in BIO with and without an
executor.
I'm still looking for the root cause. I have noticed that without an
executor I see far fewer keep-alive requests than I'd expect. That could
explain the difference, or some of it at least.
Mark
> I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
>
> Thanks
> Piyush.
>
> -----Original Message-----
> From: Mark Thomas [mailto:markt@apache.org]
> Sent: Wednesday, June 07, 2017 8:27 PM
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
> On 06/06/17 09:10, Piyush Kumar Nayak wrote:
>> Thanks, Mark.
>>
>> Here are some additional details.
>
> Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
>
>> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>>
>> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
>
> I've disabled session in the JSP for my test and used a simpler JSP.
> That should amplify any difference in connector performance.
>
> I'm also testing on localhost.
>
>> I have disabled access log in tomcat. All the other server.xml settings are the default.
>>
>>
>> The BIO with Executor configuration we are using is:
>> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>> maxThreads="150" minSpareThreads="4"/>
>>
>> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
>> connectionTimeout="20000"
>> redirectPort="8443" />
>
> FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
>
> I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
>
> If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
>
> My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Mark,
It may not be important, but just to be clear, I'm not testing on localhost. My JMeter and Tomcat are on different machines, with 1 Gbps NIC interfaces between them.
Can you also please provide some pointers on the following:
- How did you disable session on your JSP?
- Did you observe anything that suggests that the client is not closing the connections?
I did not specify the implementation for the HTTP sampler that I configured with JMeter, and it is not set in my jmeter.properties file. So, going by JMeter's documentation, Apache HttpComponents HttpClient 4.x. should be in use.
Thanks
Piyush.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Wednesday, June 07, 2017 8:27 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
On 06/06/17 09:10, Piyush Kumar Nayak wrote:
> Thanks, Mark.
>
> Here are some additional details.
Thanks. These help a lot to make sure we are testing (at least roughly) the same thing.
> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
I've disabled session in the JSP for my test and used a simpler JSP.
That should amplify any difference in connector performance.
I'm also testing on localhost.
> I have disabled access log in tomcat. All the other server.xml settings are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
> maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
> connectionTimeout="20000"
> redirectPort="8443" />
FYI for anyone following along. That is a non-executor config. The connector needs to specify the executor it wants to used.
I do see odd behaviour with BIO + executor. It locks up very easily. It appears that the client isn't closing the connections. I don't see this problem with BIO but that may be related to how BIO without an executor handles large numbers of connections.
If I lower the client thread count, the odd behaviour stops and I do see notably higher throughput with BIO + executor. That is unexpected.
My initial thoughts are maybe a contention issue related to the thread pool. I'm continuing to investigate. I plan to look at performance first and then the locking up.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 7 June 2017 17:33:02 BST, Christopher Schultz <ch...@christopherschultz.net> wrote:
>Mark,
>
>On 6/7/17 10:57 AM, Mark Thomas wrote:
>> I do see odd behaviour with BIO + executor. It locks up very
>> easily. It appears that the client isn't closing the connections. I
>> don't see this problem with BIO but that may be related to how BIO
>> without an executor handles large numbers of connections.
>
>Was one of those BIOs supposed to be an "NIO"?
No. The second BIO should have been BIO without an executor.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mark,
On 6/7/17 10:57 AM, Mark Thomas wrote:
> I do see odd behaviour with BIO + executor. It locks up very
> easily. It appears that the client isn't closing the connections. I
> don't see this problem with BIO but that may be related to how BIO
> without an executor handles large numbers of connections.
Was one of those BIOs supposed to be an "NIO"?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZOCq+AAoJEBzwKT+lPKRYguwQAIgksfcD9loUKPQoR5LZsjgg
0VNrWVSmn5o4enqjIA5RLOt55XzflK7H4drBN2IB/QfJ/5Z40d9HYAHh9x5xdFst
kyY+3kFveS0GG8lxcutsl2sr2zetV9D7/vYsjfBcvgTB+5SPcwlL9oQjk0SKSO1I
lA6pQCTdEvNfXl+gYuY9SJnDQz6SXxsLXGjUNu7CN5xesjoTGyDdX43DwhGRmrRC
rn00X2sYOlELr2jQ6K7jEPKNM0O2gkIBqlbrYOMY4hbas8K9UI+m4apsk+jPoFeG
iSGUIl7bgLIymawFyWcjhAjkjm6qrzEorBkKbO+HqSr85SsvtyfCax39RmtzqTR/
VQZka/0NFSO57pScAeWz1hZxC92Iyxj6o3LLtWtm2vWFzyw8t9QhSMvtHxok5Moj
hENhDaz+bOv5s+yWjrvWJB7zHOD8mmjrZka8tmvTdcqoESFggXG1yYPwb3NrPHoc
p7/Sy97g21t8lCbPV2LUfa8mpEf0L6I98SpwxJ3dGuXS3r9FqdsTPPEIOXjD/fsv
/0GkupemYMk9AycIbCapK1xklj3qUDKuCsePtl3A/InZlPxiKMhQ357xz/sH1AT7
ZPCoItvp+gsEGb01moIX1TwaxQ8ubiXtHYgXAAAHhVeMUIrEmuW8b7nC4DXyMJUe
cmSZQy23IkBUaqVDcY/d
=gPB6
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 06/06/17 09:10, Piyush Kumar Nayak wrote:
> Thanks, Mark.
>
> Here are some additional details.
Thanks. These help a lot to make sure we are testing (at least roughly)
the same thing.
> I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
I've disabled session in the JSP for my test and used a simpler JSP.
That should amplify any difference in connector performance.
I'm also testing on localhost.
> I have disabled access log in tomcat. All the other server.xml settings are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
> maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
> connectionTimeout="20000"
> redirectPort="8443" />
FYI for anyone following along. That is a non-executor config. The
connector needs to specify the executor it wants to used.
I do see odd behaviour with BIO + executor. It locks up very easily. It
appears that the client isn't closing the connections. I don't see this
problem with BIO but that may be related to how BIO without an executor
handles large numbers of connections.
If I lower the client thread count, the odd behaviour stops and I do see
notably higher throughput with BIO + executor. That is unexpected.
My initial thoughts are maybe a contention issue related to the thread
pool. I'm continuing to investigate. I plan to look at performance first
and then the locking up.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Just realized that the comparison matrix formatting was lost earlier, so sharing it again.
---------------------------------------------------------------------------------------------------------------------------------------------
helloWorld.JSP BIO NIO BIO w/ Exec NIO w/ Exec
Throughput ART Throughput ART Throughput ART Throughput ART
---------------------------------------------------------------------------------------------------------------------------------------------
Tomcat 7.077 38600 2 42664 2 57104 1 43660 2
Tomcat 8.0.44 38585 2 44752 2 63000 1 44341 2
Tomcat 8.0.32 -- 41211 - --
Tomcat 8.5.14 N/A 45600 1.67 N/A 46000 1.67
---------------------------------------------------------------------------------------------------------------------------------------------
Throughput: requests per sec.
ART: msecs
Also, forgot to add earlier, the jMeter client has keep-alive header enabled, and the server is left to use the default keepalive values.
-----Original Message-----
From: Piyush Kumar Nayak [mailto:pnayak@adobe.com.INVALID]
Sent: Tuesday, June 06, 2017 7:59 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: RE: Tomcat 8/NIO performance discrepancies
Violeta,
The data you see in an average of 3 samples of 180 second duration each.
Missed out on mentioning it before, but the thoughput is in no. of requests per second, and ART is in milliseconds.
Thanks
Piyush.
-----Original Message-----
From: Violeta Georgieva [mailto:violetagg@apache.org]
Sent: Tuesday, June 06, 2017 7:09 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
Hi,
2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <pn...@adobe.com.invalid>>:
>
> Thanks, Mark.
>
> Here are some additional details.
>
> I am using Apache JMeter to inject load. I am using a simple
> hello-world
JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5
secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
Do you specify "Loop Count" for the number of iterations or you specify time duration?
Thanks,
Violeta
>
> I have disabled access log in tomcat. All the other server.xml
> settings
are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
> maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
> connectionTimeout="20000"
> redirectPort="8443" />
>
>
>
> It's the same for NIO. Just the protocol changes.
>
>
>
> The client(jMeter) and server (Tomcat) are on 2 separate physical
machines with the same configuration:
>
> RHEL 7.1 x64
>
> 16 core CPU;
>
> 32 GB RAM;
>
> 1Gbps NIC
>
>
>
> ------------------------------------------------------------- JSP test
page --------------------------------------------
>
> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"
>
> pageEncoding="ISO-8859-1"%>
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Floose.dtd&data=02%7C01%7C%7Ce541b882239d42e880f508d4ace1739f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636323531683671518&sdata=AwpErTK9ClLCwDZ23QnOqj195rlITTyAeG4JX0BFv0o%3D&reserved=0">
>
> <html>
>
> <head>
>
> <meta http-equiv="Content-Type" content="text/html;
> charset=ISO-8859-1">
>
> <title>Hello World - JSP tomcat test</title>
>
> </head>
>
> <body>
>
> <%= "Hello World! - JSP on Tomcat 8.15" %>
>
> </body>
>
> </html>
>
> ------------------------------------------------------------- JSP
> test
page -------------------------------------------------------------
>
>
>
> Test result:
> helloWorld.JSP
>
>
>
> BIO
>
> NIO
>
> BIO w/ Exec
>
> NIO w/ Exec
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Tomcat 7.077
>
> 38600
>
> 2
>
> 42664
>
> 2
>
> 57104
>
> 1
>
> 43660
>
> 2
>
> Tomcat 8.0.44
>
> 38585
>
> 2
>
> 44752
>
> 2
>
> 63000
>
> 1
>
> 44341
>
> 2
>
> Tomcat 8.0.32
>
>
>
>
>
>
>
>
>
> 41211
>
>
>
>
>
>
>
> Tomcat 8.5.14
>
> N/A
>
> 45600
>
> 1.67
>
> N/A
>
> 46000
>
> 1.67
>
>
>
>
>
>
> Regards,
>
> Piyush.
>
>
>
> -----Original Message-----
>
> From: Mark Thomas [mailto:markt@apache.org]
>
> Sent: Tuesday, June 06, 2017 12:50 PM
>
> To: Tomcat Users List <us...@tomcat.apache.org>>
>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
>
>
> On 06/06/17 07:30, Piyush Kumar Nayak wrote:
>
> > We have been testing different connectors of tomcat for performance.
>
> > We have tested the following server versions:
>
> >
>
> > - Tomcat 7.077
>
> >
>
> > - Tomcat 8.0.44
>
> >
>
> > - Tomcat 8.0.32
>
> >
>
> > - Tomcat 8.5.14
>
> >
>
> > with the following HTTP connectors:
>
> >
>
> > - BIO
>
> >
>
> > - BIO with Executor
>
> >
>
> > - NIO
>
> >
>
> > - NIO with Executor
>
> >
>
> > Our observation is that NIO is up to 15% faster than BIO. But BIO
> > with
Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
>
> >
>
> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform
> > as
well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
>
> >
>
> > I would appreciate any help with the following:
>
> >
>
> > - Why is the executor not boosting the performance for NIO, as
it is for BIO?
>
>
>
> Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.
>
>
>
> >
>
> > - BIO is not available as an option with Tomcat 8.5. Why has
it been removed? Is there any way to get it back ?
>
>
>
> Because it can't support the non-blocking requirements of the
> WebSocket
API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.
>
>
>
> > I'd be happy to share any other details from my testing for any
clarification.
>
>
>
> You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.
>
>
>
> Mark
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<ma...@tomcat.apache.org>
>
> For additional commands, e-mail: users-help@tomcat.apache.org<ma...@tomcat.apache.org>
>
>
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Violeta,
The data you see in an average of 3 samples of 180 second duration each.
Missed out on mentioning it before, but the thoughput is in no. of requests per second, and ART is in milliseconds.
Thanks
Piyush.
-----Original Message-----
From: Violeta Georgieva [mailto:violetagg@apache.org]
Sent: Tuesday, June 06, 2017 7:09 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
Hi,
2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <pn...@adobe.com.invalid>>:
>
> Thanks, Mark.
>
> Here are some additional details.
>
> I am using Apache JMeter to inject load. I am using a simple
> hello-world
JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5
secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
Do you specify "Loop Count" for the number of iterations or you specify time duration?
Thanks,
Violeta
>
> I have disabled access log in tomcat. All the other server.xml
> settings
are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
> maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
> connectionTimeout="20000"
> redirectPort="8443" />
>
>
>
> It's the same for NIO. Just the protocol changes.
>
>
>
> The client(jMeter) and server (Tomcat) are on 2 separate physical
machines with the same configuration:
>
> RHEL 7.1 x64
>
> 16 core CPU;
>
> 32 GB RAM;
>
> 1Gbps NIC
>
>
>
> ------------------------------------------------------------- JSP test
page --------------------------------------------
>
> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"
>
> pageEncoding="ISO-8859-1"%>
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2FTR%2Fhtml4%2Floose.dtd&data=02%7C01%7C%7Ce541b882239d42e880f508d4ace1739f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636323531683671518&sdata=AwpErTK9ClLCwDZ23QnOqj195rlITTyAeG4JX0BFv0o%3D&reserved=0">
>
> <html>
>
> <head>
>
> <meta http-equiv="Content-Type" content="text/html;
> charset=ISO-8859-1">
>
> <title>Hello World - JSP tomcat test</title>
>
> </head>
>
> <body>
>
> <%= "Hello World! - JSP on Tomcat 8.15" %>
>
> </body>
>
> </html>
>
> ------------------------------------------------------------- JSP
> test
page -------------------------------------------------------------
>
>
>
> Test result:
> helloWorld.JSP
>
>
>
> BIO
>
> NIO
>
> BIO w/ Exec
>
> NIO w/ Exec
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Tomcat 7.077
>
> 38600
>
> 2
>
> 42664
>
> 2
>
> 57104
>
> 1
>
> 43660
>
> 2
>
> Tomcat 8.0.44
>
> 38585
>
> 2
>
> 44752
>
> 2
>
> 63000
>
> 1
>
> 44341
>
> 2
>
> Tomcat 8.0.32
>
>
>
>
>
>
>
>
>
> 41211
>
>
>
>
>
>
>
> Tomcat 8.5.14
>
> N/A
>
> 45600
>
> 1.67
>
> N/A
>
> 46000
>
> 1.67
>
>
>
>
>
>
> Regards,
>
> Piyush.
>
>
>
> -----Original Message-----
>
> From: Mark Thomas [mailto:markt@apache.org]
>
> Sent: Tuesday, June 06, 2017 12:50 PM
>
> To: Tomcat Users List <us...@tomcat.apache.org>>
>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
>
>
> On 06/06/17 07:30, Piyush Kumar Nayak wrote:
>
> > We have been testing different connectors of tomcat for performance.
>
> > We have tested the following server versions:
>
> >
>
> > - Tomcat 7.077
>
> >
>
> > - Tomcat 8.0.44
>
> >
>
> > - Tomcat 8.0.32
>
> >
>
> > - Tomcat 8.5.14
>
> >
>
> > with the following HTTP connectors:
>
> >
>
> > - BIO
>
> >
>
> > - BIO with Executor
>
> >
>
> > - NIO
>
> >
>
> > - NIO with Executor
>
> >
>
> > Our observation is that NIO is up to 15% faster than BIO. But BIO
> > with
Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
>
> >
>
> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform
> > as
well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
>
> >
>
> > I would appreciate any help with the following:
>
> >
>
> > - Why is the executor not boosting the performance for NIO, as
it is for BIO?
>
>
>
> Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.
>
>
>
> >
>
> > - BIO is not available as an option with Tomcat 8.5. Why has
it been removed? Is there any way to get it back ?
>
>
>
> Because it can't support the non-blocking requirements of the
> WebSocket
API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.
>
>
>
> > I'd be happy to share any other details from my testing for any
clarification.
>
>
>
> You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.
>
>
>
> Mark
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org<ma...@tomcat.apache.org>
>
> For additional commands, e-mail: users-help@tomcat.apache.org<ma...@tomcat.apache.org>
>
>
Re: Tomcat 8/NIO performance discrepancies
Posted by Violeta Georgieva <vi...@apache.org>.
Hi,
2017-06-06 11:10 GMT+03:00 Piyush Kumar Nayak <pn...@adobe.com.invalid>:
>
> Thanks, Mark.
>
> Here are some additional details.
>
> I am using Apache JMeter to inject load. I am using a simple hello-world
JSP.
>
> The test JMX simulates 100 concurrent user threads with a ramp-up of 5
secs and uses an HTTP Cookie Manager, that reuses cookies for each user
thread.
Do you specify "Loop Count" for the number of iterations or you specify
time duration?
Thanks,
Violeta
>
> I have disabled access log in tomcat. All the other server.xml settings
are the default.
>
>
> The BIO with Executor configuration we are using is:
> <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
> maxThreads="150" minSpareThreads="4"/>
>
> <Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
> connectionTimeout="20000"
> redirectPort="8443" />
>
>
>
> It's the same for NIO. Just the protocol changes.
>
>
>
> The client(jMeter) and server (Tomcat) are on 2 separate physical
machines with the same configuration:
>
> RHEL 7.1 x64
>
> 16 core CPU;
>
> 32 GB RAM;
>
> 1Gbps NIC
>
>
>
> ------------------------------------------------------------- JSP test
page --------------------------------------------
>
> <%@ page language="java" contentType="text/html; charset=ISO-8859-1"
>
> pageEncoding="ISO-8859-1"%>
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "
http://www.w3.org/TR/html4/loose.dtd">
>
> <html>
>
> <head>
>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
>
> <title>Hello World - JSP tomcat test</title>
>
> </head>
>
> <body>
>
> <%= "Hello World! - JSP on Tomcat 8.15" %>
>
> </body>
>
> </html>
>
> ------------------------------------------------------------- JSP test
page -------------------------------------------------------------
>
>
>
> Test result:
> helloWorld.JSP
>
>
>
> BIO
>
> NIO
>
> BIO w/ Exec
>
> NIO w/ Exec
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Throughput
>
> ART
>
> Tomcat 7.077
>
> 38600
>
> 2
>
> 42664
>
> 2
>
> 57104
>
> 1
>
> 43660
>
> 2
>
> Tomcat 8.0.44
>
> 38585
>
> 2
>
> 44752
>
> 2
>
> 63000
>
> 1
>
> 44341
>
> 2
>
> Tomcat 8.0.32
>
>
>
>
>
>
>
>
>
> 41211
>
>
>
>
>
>
>
> Tomcat 8.5.14
>
> N/A
>
> 45600
>
> 1.67
>
> N/A
>
> 46000
>
> 1.67
>
>
>
>
>
>
> Regards,
>
> Piyush.
>
>
>
> -----Original Message-----
>
> From: Mark Thomas [mailto:markt@apache.org]
>
> Sent: Tuesday, June 06, 2017 12:50 PM
>
> To: Tomcat Users List <us...@tomcat.apache.org>
>
> Subject: Re: Tomcat 8/NIO performance discrepancies
>
>
>
> On 06/06/17 07:30, Piyush Kumar Nayak wrote:
>
> > We have been testing different connectors of tomcat for performance.
>
> > We have tested the following server versions:
>
> >
>
> > - Tomcat 7.077
>
> >
>
> > - Tomcat 8.0.44
>
> >
>
> > - Tomcat 8.0.32
>
> >
>
> > - Tomcat 8.5.14
>
> >
>
> > with the following HTTP connectors:
>
> >
>
> > - BIO
>
> >
>
> > - BIO with Executor
>
> >
>
> > - NIO
>
> >
>
> > - NIO with Executor
>
> >
>
> > Our observation is that NIO is up to 15% faster than BIO. But BIO with
Executor is up to 65% faster than BIO. Using a shared executor thread pool
with NIO does not improve the performance. We have tried some variations of
connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but
nothing brings us close to the performance that we can get with BIO with
Executor.
>
> >
>
> > We have upgraded to Tomcat 8.5, and can't seem to get it to perform as
well as Tomcat 7 using BIO with Executor, which is what we were using
earlier.
>
> >
>
> > I would appreciate any help with the following:
>
> >
>
> > - Why is the executor not boosting the performance for NIO, as
it is for BIO?
>
>
>
> Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.
>
>
>
> >
>
> > - BIO is not available as an option with Tomcat 8.5. Why has
it been removed? Is there any way to get it back ?
>
>
>
> Because it can't support the non-blocking requirements of the WebSocket
API or the Servlet API and while you can fake non-blocking support, the
result is liable to enter a deadlock.
>
>
>
> > I'd be happy to share any other details from my testing for any
clarification.
>
>
>
> You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.
>
>
>
> Mark
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Tomcat 8/NIO performance discrepancies
Posted by Piyush Kumar Nayak <pn...@adobe.com.INVALID>.
Thanks, Mark.
Here are some additional details.
I am using Apache JMeter to inject load. I am using a simple hello-world JSP.
The test JMX simulates 100 concurrent user threads with a ramp-up of 5 secs and uses an HTTP Cookie Manager, that reuses cookies for each user thread.
I have disabled access log in tomcat. All the other server.xml settings are the default.
The BIO with Executor configuration we are using is:
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
<Connector port="8080" protocol="org.apache.coyote.http11.Http11Protocol"
connectionTimeout="20000"
redirectPort="8443" />
It's the same for NIO. Just the protocol changes.
The client(jMeter) and server (Tomcat) are on 2 separate physical machines with the same configuration:
RHEL 7.1 x64
16 core CPU;
32 GB RAM;
1Gbps NIC
------------------------------------------------------------- JSP test page --------------------------------------------
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Hello World - JSP tomcat test</title>
</head>
<body>
<%= "Hello World! - JSP on Tomcat 8.15" %>
</body>
</html>
------------------------------------------------------------- JSP test page -------------------------------------------------------------
Test result:
helloWorld.JSP
BIO
NIO
BIO w/ Exec
NIO w/ Exec
Throughput
ART
Throughput
ART
Throughput
ART
Throughput
ART
Tomcat 7.077
38600
2
42664
2
57104
1
43660
2
Tomcat 8.0.44
38585
2
44752
2
63000
1
44341
2
Tomcat 8.0.32
41211
Tomcat 8.5.14
N/A
45600
1.67
N/A
46000
1.67
Regards,
Piyush.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Tuesday, June 06, 2017 12:50 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat 8/NIO performance discrepancies
On 06/06/17 07:30, Piyush Kumar Nayak wrote:
> We have been testing different connectors of tomcat for performance.
> We have tested the following server versions:
>
> - Tomcat 7.077
>
> - Tomcat 8.0.44
>
> - Tomcat 8.0.32
>
> - Tomcat 8.5.14
>
> with the following HTTP connectors:
>
> - BIO
>
> - BIO with Executor
>
> - NIO
>
> - NIO with Executor
>
> Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
>
> We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
>
> I would appreciate any help with the following:
>
> - Why is the executor not boosting the performance for NIO, as it is for BIO?
Without knowing the details of your tests - no idea. The numbers are a long way from what I'd expect.
>
> - BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?
Because it can't support the non-blocking requirements of the WebSocket API or the Servlet API and while you can fake non-blocking support, the result is liable to enter a deadlock.
> I'd be happy to share any other details from my testing for any clarification.
You'll need to explain your testing methodology - in detail - before anyone can provide any useful input.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8/NIO performance discrepancies
Posted by Mark Thomas <ma...@apache.org>.
On 06/06/17 07:30, Piyush Kumar Nayak wrote:
> We have been testing different connectors of tomcat for performance.
> We have tested the following server versions:
>
> - Tomcat 7.077
>
> - Tomcat 8.0.44
>
> - Tomcat 8.0.32
>
> - Tomcat 8.5.14
>
> with the following HTTP connectors:
>
> - BIO
>
> - BIO with Executor
>
> - NIO
>
> - NIO with Executor
>
> Our observation is that NIO is up to 15% faster than BIO. But BIO with Executor is up to 65% faster than BIO. Using a shared executor thread pool with NIO does not improve the performance. We have tried some variations of connector attribute with Tomcat 8 (maxThreads, acceptorThreadCount), but nothing brings us close to the performance that we can get with BIO with Executor.
>
> We have upgraded to Tomcat 8.5, and can't seem to get it to perform as well as Tomcat 7 using BIO with Executor, which is what we were using earlier.
>
> I would appreciate any help with the following:
>
> - Why is the executor not boosting the performance for NIO, as it is for BIO?
Without knowing the details of your tests - no idea. The numbers are a
long way from what I'd expect.
>
> - BIO is not available as an option with Tomcat 8.5. Why has it been removed? Is there any way to get it back ?
Because it can't support the non-blocking requirements of the WebSocket
API or the Servlet API and while you can fake non-blocking support, the
result is liable to enter a deadlock.
> I'd be happy to share any other details from my testing for any clarification.
You'll need to explain your testing methodology - in detail - before
anyone can provide any useful input.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org