You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jess Holle <je...@ptc.com> on 2011/05/02 23:49:39 UTC
Async servlets and Tomcat 7 vs. mod_jk
What are the limitations/requirements of using asynchronous servlets in
Tomcat 7?
We use Apache and mod_jk to balance load over multiple Tomcats. I note
that there is no NIO AJP connector -- only BIO and APR. I have /no
/interest in the native APR connectors -- as it's simply /*far* /too
painful to produce 7 different native builds of this (some on platforms
with unbelievably horrific linkers).
What does this mean for us? Does this mean asynchronous servlets won't
work for us? Or that they'll "work", but we won't actually reduce the
thread usage in Tomcat at all?
In Apache, I note that there are noises about broader/better support for
the mod_event MPM worker. Does mod_jk work with mod_event there to
reduce the threads required in Apache?
--
Jess Holle
Re: Async servlets and Tomcat 7 vs. mod_jk
Posted by Jess Holle <je...@ptc.com>.
On 5/3/2011 2:55 PM, Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Mark,
>
> On 5/3/2011 4:18 AM, Mark Thomas wrote:
>>> In Apache, I note that there are noises about broader/better support for
>>> the mod_event MPM worker. Does mod_jk work with mod_event there to
>>> reduce the threads required in Apache?
>> Sorry, don't know. I have relatively little to do with the native mod_jk
>> code.
> I'm also not sure about mod_jk support for mod_event, but at ApacheCon
> one of the presenters said that httpd 2.4 will have the event MPM as the
> default.
>
> Given that, looking to get mod_jk working well with the event MPM sounds
> like a good thing for us to do.
If mod_event is going to be the default, then ensuring mod_jk works well
with it sounds like a rather critical thing to do.
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Async servlets and Tomcat 7 vs. mod_jk
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Mark,
On 5/3/2011 4:18 AM, Mark Thomas wrote:
>> In Apache, I note that there are noises about broader/better support for
>> the mod_event MPM worker. Does mod_jk work with mod_event there to
>> reduce the threads required in Apache?
>
> Sorry, don't know. I have relatively little to do with the native mod_jk
> code.
I'm also not sure about mod_jk support for mod_event, but at ApacheCon
one of the presenters said that httpd 2.4 will have the event MPM as the
default.
Given that, looking to get mod_jk working well with the event MPM sounds
like a good thing for us to do.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk3AXaoACgkQ9CaO5/Lv0PA/QACgi+hzO65le7nTGCga+mnhTx5G
19sAn3uG3G0Fd/bk4qsGRWAReA0aQEbM
=wBe7
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Async servlets and Tomcat 7 vs. mod_jk
Posted by Jess Holle <je...@ptc.com>.
On 5/3/2011 6:14 AM, Jess Holle wrote:
> On 5/3/2011 3:18 AM, Mark Thomas wrote:
>> As I write this it occurs to me that an AJP-NIO connector would be a big
>> help to you here. I don't know how much work that would be to write but
>> with the refactoring already completed for Tomcat 7 it might be as
>> little as 1000 lines of code. If you would be interested in such a
>> connector create an enhancement request in Bugzilla.
> That would almost certainly be the biggest help to me, yes. While I
> have nothing against Apache HTTPD or APR in theory, in practice native
> builds (and bugs) across numerous platforms are just too much to deal
> with.
Bug 51145 <https://issues.apache.org/bugzilla/show_bug.cgi?id=51145> filed.
--
Jess Holle
Re: Async servlets and Tomcat 7 vs. mod_jk
Posted by Jess Holle <je...@ptc.com>.
On 5/3/2011 3:18 AM, Mark Thomas wrote:
> As I write this it occurs to me that an AJP-NIO connector would be a big
> help to you here. I don't know how much work that would be to write but
> with the refactoring already completed for Tomcat 7 it might be as
> little as 1000 lines of code. If you would be interested in such a
> connector create an enhancement request in Bugzilla.
That would almost certainly be the biggest help to me, yes. While I
have nothing against Apache HTTPD or APR in theory, in practice native
builds (and bugs) across numerous platforms are just too much to deal with.
>> In Apache, I note that there are noises about broader/better support for
>> the mod_event MPM worker. Does mod_jk work with mod_event there to
>> reduce the threads required in Apache?
> Sorry, don't know. I have relatively little to do with the native mod_jk
> code.
Thanks. I was hoping someone on this list could answer (Mladen?), but
realize there are relatively few involved in mod_jk code.
--
Jess Holle
Re: Async servlets and Tomcat 7 vs. mod_jk
Posted by Mark Thomas <ma...@apache.org>.
On 02/05/2011 22:49, Jess Holle wrote:
> What are the limitations/requirements of using asynchronous servlets in
> Tomcat 7?
>
> We use Apache and mod_jk to balance load over multiple Tomcats. I note
> that there is no NIO AJP connector -- only BIO and APR. I have /no
> /interest in the native APR connectors -- as it's simply /*far* /too
> painful to produce 7 different native builds of this (some on platforms
> with unbelievably horrific linkers).
>
> What does this mean for us? Does this mean asynchronous servlets won't
> work for us? Or that they'll "work", but we won't actually reduce the
> thread usage in Tomcat at all?
They'll work, and you'll see decreased thread usage will all connectors
including HTTP-BIO and AJP-BIO. However, it isn't quite that simple.
With Servlet 3 async requests in progress, Tomcat will have more
concurrent connections than it is using threads to process them since
some of those connections will be associated with async requests that
are currently 'paused'. Since maxThreads still controls the number of
maximum threads then this means BIO now supports more concurrent
connections than there are threads available. This causes a different
problem.
When there are more connections than threads, Tomcat needs to know which
connection service next when a thread becomes available. With NIO and
APR this is easy since they support non-blocking IO. Add all the
connections to a poller and then poller will signal when one of the
connections has some data to read. This connection can then be passed to
a thread for processing. With BIO pollers simply aren't available. The
current BIO implementation adds the connections to a queue and they are
processed in turn as threads become available. This can create some
unepxected behaviours. Consider the following:
maxThreads = 200
connectionTimeout = 5000
connections = 1000, all in http keep-alive
In the worst case scenario if those 1000 connections enter keep-alive
around same time, a new connection is going to wait more than 25 seconds
to be processed. Let me explain why:
1001 connections in keep-alive, 200 of which are assigned to threads
5s later, those 200 threads timeout the connection and are passed the
next 200 connections to process
now 801 connections in keep-alive, 200 of which are assigned to threads
another 5 seconds, those 200 threads timeout
now 601 connections
another 5s...
now 401 connections
another 5s...
now 201 connections
another 5s
now 1 connection left in the queue (the new one) this now gets processed.
So despite there being nothing to process in those 1000 keep-alive
connections, the new connection where there was data took 25s+ to process.
Part of this is due to a TODO in the BIO connector (that is fixed for
the next release). Timeouts where calculated from when the thread
started processing the connection, not from when the connection entered
the kepp-alive state (i.e. when it was added to the processing queue).
With this bug fixed, the new connection in the scenario above would wait
just over 5 seconds.
I am currently working on simulating polling with the BIO connector. The
good news is that it works but the price is significantly increased CPU
usage. In my tests CPU usage increased from ~38% to 51% when simulated
polling is used.
I haven't completed the patch and once committed the other committers
may not like it so keep an eye on the dev list for the eventual solution
but I am currently working towards something along the lines of:
maxThreads defaults to 200 (as it does now)
maxConnections defaults to maxThreads (currently defaults to 10000)
maxConnections <=maxThreads do nothing
maxConnections > maxThreads enable pollTime (new feature, defaults to 100ms)
The recommendation will be not to set maxConnections > maxThreads
without a lot of performance testing and to use NIO or APR instead if at
all possible.
pollTime is how long a thread waits for data on a connection before
putting it back in the connection queue. By using this, Tomcat
effectively scans through the current connections looking for data to
process. Gong back to my example above this would mean the new
connection waiting ~0.5s before being processed. Not ideal, but better
than the 5s it was.
As I write this it occurs to me that an AJP-NIO connector would be a big
help to you here. I don't know how much work that would be to write but
with the refactoring already completed for Tomcat 7 it might be as
little as 1000 lines of code. If you would be interested in such a
connector create an enhancement request in Bugzilla.
> In Apache, I note that there are noises about broader/better support for
> the mod_event MPM worker. Does mod_jk work with mod_event there to
> reduce the threads required in Apache?
Sorry, don't know. I have relatively little to do with the native mod_jk
code.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org