You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Remy Maucherat <re...@apache.org> on 2003/12/12 11:52:38 UTC
5.next + 4.1.x future
Hi,
I see no reason to create a new branch, which is not what happened with
previous releases, where branches were created sometimes even before the
stable release (4.1.x got forked for 4.0 final, and 5.0.x was forked for
4.1.7). Reasons for no branching:
- Tomcat is rather modular
- Behavior in nearly all cases seems well established, so no major
breakage is to be expected
- Increased maintenance requirements due to diverting resource (we did
have a problem with that, looking at all the BZ items)
- Branching hasn't really been synonymous with stability or quality in
the past, given patch porting decisions have been questionable sometimes
(that's likely my fault, as the RM)
We now have a stable 5.0.16, which so far seems like a quality release.
Since 5.0.x is only an evolution of 4.1.x, I don't see much needs for
more than one additional 4.1.x releases (assuming it's a good one
without regressions :D ). If people want more, a new RM will need to pop
up :) (BTW, I can pass the hot potato right now for 4.x if someone is
interested :D )
I'm interested in the following enhancements to 5.0.x in the future (=
in january or later):
- refactoring of the save to XML feature (that's been requested; I don't
know if I'll use that to be able to use the admin webapp under JBoss,
though); likely the default impl will remain the current one, but we'll
be able to take out all that code from StandardServer :)
- full support for a JMX-ized server.xml, which is used in the embedded
distribution (I think a few components can't be created using that; I
could be wrong though, Costin did a great job)
- I'll add the minimal server.xml to embedded and some startup scripts
and the native wrappers, and I'll "optimize" it some more, so that it
can also be used as a minimal Tomcat installation (without any CL
hierarchy, which sometimes confuses beginners)
- tweaks and fixes
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Costin Manolache <cm...@yahoo.com>.
Remy Maucherat wrote:
> I'm interested in the following enhancements to 5.0.x in the future (=
> in january or later):
> - refactoring of the save to XML feature (that's been requested; I don't
> know if I'll use that to be able to use the admin webapp under JBoss,
> though); likely the default impl will remain the current one, but we'll
> be able to take out all that code from StandardServer :)
IMO the current one is pretty bad. There is a beginning in
commons-modeler, and IMO it is the right solution for the problem ( as
it tracks changes to the config - instead of trying to generate a new
file, and preserves comments and structure ). I don't know when things
will slow down a bit at work - but in january I'll try my best to free
some time.
Costin
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Costin Manolache <cm...@yahoo.com>.
Remy Maucherat wrote:
> I'm interested in the following enhancements to 5.0.x in the future (=
> in january or later):
> - refactoring of the save to XML feature (that's been requested; I don't
> know if I'll use that to be able to use the admin webapp under JBoss,
> though); likely the default impl will remain the current one, but we'll
> be able to take out all that code from StandardServer :)
IMO the current one is pretty bad. There is a beginning in
commons-modeler, and IMO it is the right solution for the problem ( as
it tracks changes to the config - instead of trying to generate a new
file, and preserves comments and structure ). I don't know when things
will slow down a bit at work - but in january I'll try my best to free
some time.
Costin
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Away next week
Posted by Remy Maucherat <re...@apache.org>.
Hi,
I'll be away next week, until friday. I may have some internet access,
but it won't be much.
Please handle incoming BZ items for me so that they don't pile up too
much :)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Costin Manolache <cm...@yahoo.com>.
Remy Maucherat wrote:
> Jeanfrancois Arcand wrote:
>
>> +1
>>
>> I would also like to explore, as a separate module, a connector that
>> uses NIO to see what kind of performance we can have. I have no
>> intention of replacing the current connector, and probably I will
>> waste my time trying to have the same performance as the current
>> connector. But I would like to explore it anyway .... ( latter next
>> year, after the security stuff )....
>
>
> My opinion is that NIO is going to be really useless.
>
> - HTTP processing time is insignificant (BTW, most of the current
> algorithm can be used in non blocking mode, I think)
> - You have to go in blocking mode for the whole Catalina pipeline (ok:
> fast) and for running the user application
> - It will make object reuse more complex (you'll have to have a pool
> holding all the HTTP processors, essentially)
NIO may be good in mod_jk mode - not the select but the mmap.
I agree that select can't help too much given the servlet IO model,
it's the mmap ( and the char-byte conversion ) that are really valuable
in our case.
BTW - another intersting optimization is for JSP static content, which
can be also mmapped to avoid sending it over ( and maybe even get it
sent directly from the disk ).
Costin
>
> So the benefit is for holding HTTP/1.1 persistent connections, given that:
> - It will make the connector code more complex ( :-( )
> - Modern OSes (Linux 2.6, Solaris with Sun's VMs, any Linux with JRockit
> and its thread mapping similar to Solaris) will probably not really care
> about saving a few threads
>
> But, hey, if you have time, it's an interesting experiment (and I'll try
> to help out a bit if I have time) :)
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by David Rees <dr...@greenhydrant.com>.
Costin Manolache wrote:
> Jan-Henrik Haukeland wrote:
>> Remy Maucherat <re...@apache.org> writes:
>>> My opinion is that NIO is going to be really useless.
>>
>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>> perform well it may be useless. But, really, before NIO it was
>> hopeless to try and write a scalable and fast tcp server application
>> in Java. Tomcat's current connection handling with blocing all over
>> the place and "thundering herd" problem doesn't scale or work very
>> well under heavy load.
>
> Only thing you can save is the static servlet - but for that you are
> better off using a real server or a cache.
But as Remy mentioned:
> Static content is handled with a regular (well, almost) servlet, and
> all those requests have to go through the filter pipeline (ie, user
> components). This is the exact same problem when having Apache serve
> static files.
So unless you break the spec, you still will need to spawn a thread to
process static content through the filter pipeline.
If that happens to be OK for your application, then you are right,
better to use a real server or cache in front of your application (like
Squid www.squid-cache.org).
-Dave
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Remy Maucherat <re...@apache.org>.
Costin Manolache wrote:
> Jan-Henrik Haukeland wrote:
>
> Don't forget that servlets ( which is the main job of tomcat ) use
> blocking input/output streams.
>
> NIO select ( which is what most people see first in NIO ) is not going
> to help in this. Select is extremely powerfull - but it requires a
> certain kind of event-based architecture which doesn't match the
> blocking model of servlets ( which has the big benefit of making things
> simple for users ).
>
> Think about it - in any case you do need one thread for each active
> request, for the servlet. Most of the time is spent in the service()
> method - you can do select() as much as you want in http/jk processing,
> but you must have the thread for service() ( which can't be avoided
> since servlets are allowed to block the thread ).
>
> Only thing you can save is the static servlet - but for that you are
> better off using a real server or a cache.
>
> It's sad people see select() first - and don't pay attention to
> char-byte convertors in NIO ( a huge ammount of optimization - tomcat
> already has something that emulates the same model, so we won't benefit
> much ), and the mmapping feature - which is huge for IPC and java-native
> communication.
Well, we're using the C2B from NIO already. The old stuff is just
wrappers around them, AFAIK. Of course, it would be more efficient to
bypass the wrappers completely :)
For mem maps, I don't know. The direct buffers are rather annoying to
manipulate, it seems.
It's good that we have a very efficient HTTP stack already, and we don't
have to rush this stuff :-D
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Costin Manolache <cm...@yahoo.com>.
Jan-Henrik Haukeland wrote:
> Remy Maucherat <re...@apache.org> writes:
>
>
>>My opinion is that NIO is going to be really useless.
>
>
> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
> perform well it may be useless. But, really, before NIO it was
> hopeless to try and write a scalable and fast tcp server application
> in Java. Tomcat's current connection handling with blocing all over
> the place and "thundering herd" problem doesn't scale or work very
> well under heavy load.
Don't forget that servlets ( which is the main job of tomcat ) use
blocking input/output streams.
NIO select ( which is what most people see first in NIO ) is not going
to help in this. Select is extremely powerfull - but it requires a
certain kind of event-based architecture which doesn't match the
blocking model of servlets ( which has the big benefit of making things
simple for users ).
Think about it - in any case you do need one thread for each active
request, for the servlet. Most of the time is spent in the service()
method - you can do select() as much as you want in http/jk processing,
but you must have the thread for service() ( which can't be avoided
since servlets are allowed to block the thread ).
Only thing you can save is the static servlet - but for that you are
better off using a real server or a cache.
It's sad people see select() first - and don't pay attention to
char-byte convertors in NIO ( a huge ammount of optimization - tomcat
already has something that emulates the same model, so we won't benefit
much ), and the mmapping feature - which is huge for IPC and java-native
communication.
Costin
>
>
>>- It will make object reuse more complex (you'll have to have a pool
>>holding all the HTTP processors, essentially)
>
>
> You will have a pool holding incomming connection, and if there is an
> i/o event on the connection such as data present you will hand the
> connection over to a small pool of HTTP processors (5-10). Here's a
> quasi example for the main server loop. (All sockets must be
> non-blocking, including the serversocket, i.e. it must *not* block on
> accept(). Every i/o operation must of course also be non-blocking).
>
> This loop allows the server to handle many thousand concurrent
> connections without using many threads.
>
> for(;;) {
> /* Poll the connection pool, including the server socket and
> * return the number of sockets ready for processing */
> numReady= Poll.poll();
> if(Poll.hasConnection()) {
> /* We have new connections on the server socket; accept
> * every incomming connection and put them into the
> * connection pool */
> doAccept();
> /* Continue checking for more incomming connection;
> * new connections are more important than old */
> continue;
> }
> if(numReady == 0) {
> /* Close and remove connections that has timed out or
> * are done */
> Poll.checkTimeout();
> /* Continue checking for more incomming connection */
> continue;
> }
> /* Get and process only those connections that is
> * ready. I.e. has data avilable. (More optimizing is to only
> * hand over incomming connection that has read and buffered a
> * complete HTTP request */
> while((connection= Poll.next())) {
> /* Push the connection into our HttpProcessor thread
> * worker/queue for processing.
> - Keep-alive connections are not closed but keept in
> the connection pool and will be re-processed.
> - Connections that are finished are closed by the
> HttpProcessor and will be removed from the pool
> in the test above */
> if(! processorPush(connection)) {
> /* If no more processor threads are available
> * we process the request ourself. Alternatively
> * increase the pool of processor threads */
> processConnection(connection, NO_KEEPALIVE);
> }
> }
> }
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
RE: 5.next + 4.1.x future
Posted by Filip Hanik <de...@hanik.com>.
The tomcat clustering uses NIO for the session replication,
here is another fact, NIO sucks pretty bad in almost all the VMs.
In some of them, it just doesn't work as advertised at all.
I agree with Remy, it will not change the scalability that tomcat currently
supports,
because of the nature of the HTTP protocol. What you should be doing for a
heavy load
is to turn off keep alive for one thing, so that you don't have threads
waiting for nothing.
Filip
-----Original Message-----
From: Remy Maucherat [mailto:remm@apache.org]
Sent: Friday, December 12, 2003 1:38 PM
To: Tomcat Developers List
Subject: Re: 5.next + 4.1.x future
Jan-Henrik Haukeland wrote:
> Remy Maucherat <re...@apache.org> writes:
>
>>My opinion is that NIO is going to be really useless.
>
> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
> perform well it may be useless. But, really, before NIO it was
> hopeless to try and write a scalable and fast tcp server application
> in Java. Tomcat's current connection handling with blocing all over
> the place and "thundering herd" problem doesn't scale or work very
> well under heavy load.
You apparently have a very strong opinion on this, and that's fine. You
also obviously don't know what you are talking about. The purpose of
Tomcat is to make the web tier of an application server (Tomcat is
actually a mini application server), not some kind of non blocking I/O
toolkit to be used to build fixed function servers. Non blocking I/O has
great applications, and is a very useful technology, but it does not
apply to the application server world.
I think you should find a servlet container which has NIO, compare with
Tomcat 5.0.16, and come back to report your findings about how much
scalability or speed NIO brings (note: doing the non blocking socket
handling in a native layer doesn't really count, since it's not a fair
comparison with Java's NIO; you might as well use Apache).
Bring facts, not useless rants.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Remy Maucherat <re...@apache.org>.
David Rees wrote:
> On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote:
>
>>I'd be happy to send my data to the group if people are interested.
>>Aside from memory, I was surprised to find that the effect on CPU was
>>negligible (not much of a benefit from no context-switching between
>>threads) -- CPU was virtually the same in both cases. So, the scaling
>>benefits on Windows basically come from not having to allocate more
>>memory to new threads. I'm unfortunately not as familiar with the
>>Tomcat code as I'd like to be, but I assume it makes intelligent use of
>>thread pooling, which may even the memory benefits of NIO negligible.
>>At the same time, though, NIO may remove some of the constraints
>>introduced by thread pooling, possibly allowing Tomcat to handle heavier
>>loads without blowing up. An optimized NIO server would if anything
>>out-perform a blocking server, but maybe by not that much.
>
> On current Linux systems, once you start getting 500+ processes/threads
> active on a typical machine, you will find that context switching starts
> taking up a significant amount of system time, especially if you decide to
> run any system moniting tools (like ps, or top). This is better with the
> upcoming 2.6 kernels, but still doesn't scale to thousands of active
> threads very well.
For purely switching, the benches I saw saw a linear scaling for the
amount on processes in Linnux 2.6. I didn't try it, though.
There would be a significant amount of threads in wait mode. You can't
have thousands of threads actually processing stuff anyway, you'll have
problems with your database first, or your CPU in the case of XML.
I really wonder sometimes why some people think that NIO would make J2EE
scale automagically. It could be a decent optimization for resources use
(memory and some threads), but that's it, and it remains to be seen if
it wouldn't cause a throughtput drop (in that case, it would be useless,
given the amount of memory efficient VMs use to provide a similar
throughtput increase).
> However, given that you need a thread anyway to server any dynamic
> content, I don't see NIO helping that much for your typical web
> application. I could see NIO helping scale the serving of static content
> which would be useful where people are using Tomcat standalone. Maybe
> someone can prove me wrong. ;-)
Static content is handled with a regular (well, almost) servlet, and all
those requests have to go through the filter pipeline (ie, user
components). This is the exact same problem when having Apache serve
static files.
Nice try ;-)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Endre Stølsvik <En...@Stolsvik.com>.
[ a little late here, but hey.. ]
On Fri, 12 Dec 2003, David Rees wrote:
| On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote:
| >
| > I'd be happy to send my data to the group if people are interested.
| > Aside from memory, I was surprised to find that the effect on CPU was
| > negligible (not much of a benefit from no context-switching between
| > threads) -- CPU was virtually the same in both cases. So, the scaling
| > benefits on Windows basically come from not having to allocate more
| > memory to new threads. I'm unfortunately not as familiar with the
| > Tomcat code as I'd like to be, but I assume it makes intelligent use of
| > thread pooling, which may even the memory benefits of NIO negligible.
| > At the same time, though, NIO may remove some of the constraints
| > introduced by thread pooling, possibly allowing Tomcat to handle heavier
| > loads without blowing up. An optimized NIO server would if anything
| > out-perform a blocking server, but maybe by not that much.
|
| On current Linux systems, once you start getting 500+ processes/threads
| active on a typical machine, you will find that context switching starts
| taking up a significant amount of system time, especially if you decide to
| run any system moniting tools (like ps, or top). This is better with the
| upcoming 2.6 kernels, but still doesn't scale to thousands of active
| threads very well.
|
| However, given that you need a thread anyway to server any dynamic
| content, I don't see NIO helping that much for your typical web
| application. I could see NIO helping scale the serving of static content
| which would be useful where people are using Tomcat standalone. Maybe
| someone can prove me wrong. ;-)
I've also been wondering about NIO for some time, and whether a Servlet
Container ever could benefit from it.
What about this idea: if you have two types of threads, one set of Servlet
threads, and then one (or more) threads that do all the IO, and a set of
-memory buffers- inbetween, what would you get?
See, the thing is about caching - and letting the Servlet distance itself
from the client. The normal case is that the output stream is more or less
"directly attached" to the client. But if you do this other thing, having
a memory buffer inbetween, the servlet thread could do its stash, and
exit, immediately. Hopefully it could do several such operations within
-one- context switch, instead of having to IO block "on every byte
output". (Btw: i do realize that the Servlet spec allows for a buffer on
the output stream: this is something slightly different: a forced
total-buffering of the servlet's output)
So, one could tag servlets with "NIO-able" true/false. NIO-able are
servlets that typically don't produce more than 10k of data (or something
like that), and that doesn't need to "hang onto the client", typically a
servlet outputting and flushing (thus, "sending") one byte a second while
it does the heavy database query.
Thus, if a servlet needs this direct link to the client, or produces
very large "result sets", it can't be "NIOed", in the first case because
it -needs- this direct link between the server thread and the client, and
in the second case because the memory-buffer requirement would be to
heavy.
If one have loads of memory, this idea could potentially speed up
processing a bit by not spening an excessive amount of CPU time in context
switches. One have to realize that the clients' line-speed (and slowstart
on the TCP stack) often will lead to a servlet having to spend multiple
context switches before being able to exit due to the blocking of the IO -
this could be totally eliminated in many cases using this aproach of
dividing a dynamic page generation into two totally distinct steps: the
generation of the page, and the sending of that resulting page over the
network.
The connector architecture in earlier tomcats used to not be able to queue
requests; if no worker thread was available, it immediately returned a
"server full" response. I then advocated an approach of having a set of
worker threads (a limited set), and a line of incoming requests (that then
might have a higher number of "slots" than the number of available
workers), and then a dispatcher that stands inbetween. I believe that
something along the same lines have been implemented now. This suggestion
here is something along the same lines - the whole idea is to reduce the
real CPU time spent in context switches - a context switch isn't just the
registers and memory maps, it is also flushing of the CPU's Lx caches!
--
Mvh,
Endre Stølsvik M[+47 93054050] F[+47 51625182]
Developer @ CoreTrek AS - http://www.coretrek.com/
CoreTrek corporate portal / EIP - http://www.corelets.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by David Rees <dr...@greenhydrant.com>.
On Fri, December 12, 2003 at 2:12 pm, Adam Fisk wrote:
>
> I'd be happy to send my data to the group if people are interested.
> Aside from memory, I was surprised to find that the effect on CPU was
> negligible (not much of a benefit from no context-switching between
> threads) -- CPU was virtually the same in both cases. So, the scaling
> benefits on Windows basically come from not having to allocate more
> memory to new threads. I'm unfortunately not as familiar with the
> Tomcat code as I'd like to be, but I assume it makes intelligent use of
> thread pooling, which may even the memory benefits of NIO negligible.
> At the same time, though, NIO may remove some of the constraints
> introduced by thread pooling, possibly allowing Tomcat to handle heavier
> loads without blowing up. An optimized NIO server would if anything
> out-perform a blocking server, but maybe by not that much.
On current Linux systems, once you start getting 500+ processes/threads
active on a typical machine, you will find that context switching starts
taking up a significant amount of system time, especially if you decide to
run any system moniting tools (like ps, or top). This is better with the
upcoming 2.6 kernels, but still doesn't scale to thousands of active
threads very well.
However, given that you need a thread anyway to server any dynamic
content, I don't see NIO helping that much for your typical web
application. I could see NIO helping scale the serving of static content
which would be useful where people are using Tomcat standalone. Maybe
someone can prove me wrong. ;-)
-Dave
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Adam Fisk <af...@limepeer.com>.
I should also mention that I ran these tests on an Athlon 2200 with
512MB RAM.
-Adam
Adam Fisk wrote:
> Here's my excel spreadsheet. All of these numbers are taken from the
> Windows Task Manager. This is somewhat misleading in that the memory
> numbers don't properly reflect garbage collection, but it's informative
> nevertheless.
>
> The "Sends (ms)" column reflects the timeout between client sends. I
> originally intended to run these tests for many timeouts, but this
> became prohibitively annoying over the WAN. These tests were done with
> all clients running on a single machine connecting to the servers over
> the WAN.
>
> Thanks.
>
> -Adam
>
>
> Jeanfrancois Arcand wrote:
>
>>
>>
>> Adam Fisk wrote:
>>
>>> Just to chime in on the NIO issue, I agree that it's not immediately
>>> obvious what the performance benefits are. Perhaps more importantly,
>>> though, the code changes to switch Tomcat (or any other good-size
>>> app) to NIO are tremendous -- basically a rewrite of the hard parts.
>>>
>>> That said, I've done simple tests of NIO vs. blocking IO on Windows
>>> using simple blocking and non-blocking servers with variable numbers
>>> of client connections (from 100 to 10,000). At least on Windows, the
>>> performance benefits come down to memory allocated to threads. My
>>> blocking server used one send thread and one receive thread per
>>> connection (no thread pooling). Given the memory allocations per
>>> thread, though, the blocking server with 1600 connections used 137MB
>>> whereas the NIO server used 11MB, almost exactly the same as the
>>> memory use with 50 connections.
>>>
>>> I'd be happy to send my data to the group if people are interested.
>>
>>
>>
>> Yes, I will be very interested to see those numbers.
>>
>> Thanks,
>>
>> -- Jeanfrancois
>>
>>
>>> Aside from memory, I was surprised to find that the effect on CPU was
>>> negligible (not much of a benefit from no context-switching between
>>> threads) -- CPU was virtually the same in both cases. So, the
>>> scaling benefits on Windows basically come from not having to
>>> allocate more memory to new threads. I'm unfortunately not as
>>> familiar with the Tomcat code as I'd like to be, but I assume it
>>> makes intelligent use of thread pooling, which may even the memory
>>> benefits of NIO negligible. At the same time, though, NIO may remove
>>> some of the constraints introduced by thread pooling, possibly
>>> allowing Tomcat to handle heavier loads without blowing up. An
>>> optimized NIO server would if anything out-perform a blocking server,
>>> but maybe by not that much.
>>>
>>> -Adam
>>>
>>>
>>> Remy Maucherat wrote:
>>>
>>>> Jan-Henrik Haukeland wrote:
>>>>
>>>>> Remy Maucherat <re...@apache.org> writes:
>>>>>
>>>>>> My opinion is that NIO is going to be really useless.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>>>>> perform well it may be useless. But, really, before NIO it was
>>>>> hopeless to try and write a scalable and fast tcp server application
>>>>> in Java. Tomcat's current connection handling with blocing all over
>>>>> the place and "thundering herd" problem doesn't scale or work very
>>>>> well under heavy load.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> You apparently have a very strong opinion on this, and that's fine.
>>>> You also obviously don't know what you are talking about. The
>>>> purpose of Tomcat is to make the web tier of an application server
>>>> (Tomcat is actually a mini application server), not some kind of non
>>>> blocking I/O toolkit to be used to build fixed function servers. Non
>>>> blocking I/O has great applications, and is a very useful
>>>> technology, but it does not apply to the application server world.
>>>>
>>>> I think you should find a servlet container which has NIO, compare
>>>> with Tomcat 5.0.16, and come back to report your findings about how
>>>> much scalability or speed NIO brings (note: doing the non blocking
>>>> socket handling in a native layer doesn't really count, since it's
>>>> not a fair comparison with Java's NIO; you might as well use Apache).
>>>>
>>>> Bring facts, not useless rants.
>>>>
>>>> Rémy
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>>>
>>>>
>>>> .
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>
>>
>> .
>>
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Adam Fisk <af...@limepeer.com>.
Here's my excel spreadsheet. All of these numbers are taken from the
Windows Task Manager. This is somewhat misleading in that the memory
numbers don't properly reflect garbage collection, but it's informative
nevertheless.
The "Sends (ms)" column reflects the timeout between client sends. I
originally intended to run these tests for many timeouts, but this
became prohibitively annoying over the WAN. These tests were done with
all clients running on a single machine connecting to the servers over
the WAN.
Thanks.
-Adam
Jeanfrancois Arcand wrote:
>
>
> Adam Fisk wrote:
>
>> Just to chime in on the NIO issue, I agree that it's not immediately
>> obvious what the performance benefits are. Perhaps more importantly,
>> though, the code changes to switch Tomcat (or any other good-size app)
>> to NIO are tremendous -- basically a rewrite of the hard parts.
>>
>> That said, I've done simple tests of NIO vs. blocking IO on Windows
>> using simple blocking and non-blocking servers with variable numbers
>> of client connections (from 100 to 10,000). At least on Windows, the
>> performance benefits come down to memory allocated to threads. My
>> blocking server used one send thread and one receive thread per
>> connection (no thread pooling). Given the memory allocations per
>> thread, though, the blocking server with 1600 connections used 137MB
>> whereas the NIO server used 11MB, almost exactly the same as the
>> memory use with 50 connections.
>>
>> I'd be happy to send my data to the group if people are interested.
>
>
> Yes, I will be very interested to see those numbers.
>
> Thanks,
>
> -- Jeanfrancois
>
>
>> Aside from memory, I was surprised to find that the effect on CPU was
>> negligible (not much of a benefit from no context-switching between
>> threads) -- CPU was virtually the same in both cases. So, the scaling
>> benefits on Windows basically come from not having to allocate more
>> memory to new threads. I'm unfortunately not as familiar with the
>> Tomcat code as I'd like to be, but I assume it makes intelligent use
>> of thread pooling, which may even the memory benefits of NIO
>> negligible. At the same time, though, NIO may remove some of the
>> constraints introduced by thread pooling, possibly allowing Tomcat to
>> handle heavier loads without blowing up. An optimized NIO server
>> would if anything out-perform a blocking server, but maybe by not that
>> much.
>>
>> -Adam
>>
>>
>> Remy Maucherat wrote:
>>
>>> Jan-Henrik Haukeland wrote:
>>>
>>>> Remy Maucherat <re...@apache.org> writes:
>>>>
>>>>> My opinion is that NIO is going to be really useless.
>>>>
>>>>
>>>>
>>>>
>>>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>>>> perform well it may be useless. But, really, before NIO it was
>>>> hopeless to try and write a scalable and fast tcp server application
>>>> in Java. Tomcat's current connection handling with blocing all over
>>>> the place and "thundering herd" problem doesn't scale or work very
>>>> well under heavy load.
>>>
>>>
>>>
>>>
>>> You apparently have a very strong opinion on this, and that's fine.
>>> You also obviously don't know what you are talking about. The purpose
>>> of Tomcat is to make the web tier of an application server (Tomcat is
>>> actually a mini application server), not some kind of non blocking
>>> I/O toolkit to be used to build fixed function servers. Non blocking
>>> I/O has great applications, and is a very useful technology, but it
>>> does not apply to the application server world.
>>>
>>> I think you should find a servlet container which has NIO, compare
>>> with Tomcat 5.0.16, and come back to report your findings about how
>>> much scalability or speed NIO brings (note: doing the non blocking
>>> socket handling in a native layer doesn't really count, since it's
>>> not a fair comparison with Java's NIO; you might as well use Apache).
>>>
>>> Bring facts, not useless rants.
>>>
>>> Rémy
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>>
>>>
>>> .
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>
>
> .
>
Re: 5.next + 4.1.x future
Posted by Jeanfrancois Arcand <jf...@apache.org>.
Adam Fisk wrote:
> Just to chime in on the NIO issue, I agree that it's not immediately
> obvious what the performance benefits are. Perhaps more importantly,
> though, the code changes to switch Tomcat (or any other good-size app)
> to NIO are tremendous -- basically a rewrite of the hard parts.
>
> That said, I've done simple tests of NIO vs. blocking IO on Windows
> using simple blocking and non-blocking servers with variable numbers
> of client connections (from 100 to 10,000). At least on Windows, the
> performance benefits come down to memory allocated to threads. My
> blocking server used one send thread and one receive thread per
> connection (no thread pooling). Given the memory allocations per
> thread, though, the blocking server with 1600 connections used 137MB
> whereas the NIO server used 11MB, almost exactly the same as the
> memory use with 50 connections.
>
> I'd be happy to send my data to the group if people are interested.
Yes, I will be very interested to see those numbers.
Thanks,
-- Jeanfrancois
> Aside from memory, I was surprised to find that the effect on CPU was
> negligible (not much of a benefit from no context-switching between
> threads) -- CPU was virtually the same in both cases. So, the scaling
> benefits on Windows basically come from not having to allocate more
> memory to new threads. I'm unfortunately not as familiar with the
> Tomcat code as I'd like to be, but I assume it makes intelligent use
> of thread pooling, which may even the memory benefits of NIO
> negligible. At the same time, though, NIO may remove some of the
> constraints introduced by thread pooling, possibly allowing Tomcat to
> handle heavier loads without blowing up. An optimized NIO server
> would if anything out-perform a blocking server, but maybe by not that
> much.
>
> -Adam
>
>
> Remy Maucherat wrote:
>
>> Jan-Henrik Haukeland wrote:
>>
>>> Remy Maucherat <re...@apache.org> writes:
>>>
>>>> My opinion is that NIO is going to be really useless.
>>>
>>>
>>>
>>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>>> perform well it may be useless. But, really, before NIO it was
>>> hopeless to try and write a scalable and fast tcp server application
>>> in Java. Tomcat's current connection handling with blocing all over
>>> the place and "thundering herd" problem doesn't scale or work very
>>> well under heavy load.
>>
>>
>>
>> You apparently have a very strong opinion on this, and that's fine.
>> You also obviously don't know what you are talking about. The purpose
>> of Tomcat is to make the web tier of an application server (Tomcat is
>> actually a mini application server), not some kind of non blocking
>> I/O toolkit to be used to build fixed function servers. Non blocking
>> I/O has great applications, and is a very useful technology, but it
>> does not apply to the application server world.
>>
>> I think you should find a servlet container which has NIO, compare
>> with Tomcat 5.0.16, and come back to report your findings about how
>> much scalability or speed NIO brings (note: doing the non blocking
>> socket handling in a native layer doesn't really count, since it's
>> not a fair comparison with Java's NIO; you might as well use Apache).
>>
>> Bring facts, not useless rants.
>>
>> Rémy
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>>
>>
>> .
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Adam Fisk <af...@limepeer.com>.
Just to chime in on the NIO issue, I agree that it's not immediately
obvious what the performance benefits are. Perhaps more importantly,
though, the code changes to switch Tomcat (or any other good-size app)
to NIO are tremendous -- basically a rewrite of the hard parts.
That said, I've done simple tests of NIO vs. blocking IO on Windows
using simple blocking and non-blocking servers with variable numbers of
client connections (from 100 to 10,000). At least on Windows, the
performance benefits come down to memory allocated to threads. My
blocking server used one send thread and one receive thread per
connection (no thread pooling). Given the memory allocations per
thread, though, the blocking server with 1600 connections used 137MB
whereas the NIO server used 11MB, almost exactly the same as the memory
use with 50 connections.
I'd be happy to send my data to the group if people are interested.
Aside from memory, I was surprised to find that the effect on CPU was
negligible (not much of a benefit from no context-switching between
threads) -- CPU was virtually the same in both cases. So, the scaling
benefits on Windows basically come from not having to allocate more
memory to new threads. I'm unfortunately not as familiar with the
Tomcat code as I'd like to be, but I assume it makes intelligent use of
thread pooling, which may even the memory benefits of NIO negligible.
At the same time, though, NIO may remove some of the constraints
introduced by thread pooling, possibly allowing Tomcat to handle heavier
loads without blowing up. An optimized NIO server would if anything
out-perform a blocking server, but maybe by not that much.
-Adam
Remy Maucherat wrote:
> Jan-Henrik Haukeland wrote:
>
>> Remy Maucherat <re...@apache.org> writes:
>>
>>> My opinion is that NIO is going to be really useless.
>>
>>
>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>> perform well it may be useless. But, really, before NIO it was
>> hopeless to try and write a scalable and fast tcp server application
>> in Java. Tomcat's current connection handling with blocing all over
>> the place and "thundering herd" problem doesn't scale or work very
>> well under heavy load.
>
>
> You apparently have a very strong opinion on this, and that's fine. You
> also obviously don't know what you are talking about. The purpose of
> Tomcat is to make the web tier of an application server (Tomcat is
> actually a mini application server), not some kind of non blocking I/O
> toolkit to be used to build fixed function servers. Non blocking I/O has
> great applications, and is a very useful technology, but it does not
> apply to the application server world.
>
> I think you should find a servlet container which has NIO, compare with
> Tomcat 5.0.16, and come back to report your findings about how much
> scalability or speed NIO brings (note: doing the non blocking socket
> handling in a native layer doesn't really count, since it's not a fair
> comparison with Java's NIO; you might as well use Apache).
>
> Bring facts, not useless rants.
>
> Rémy
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>
>
> .
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Jess Holle <je...@ptc.com>.
Remy Maucherat wrote:
> Jan-Henrik Haukeland wrote:
>
>> Remy Maucherat <re...@apache.org> writes:
>>
>>> My opinion is that NIO is going to be really useless.
>>
>> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
>> perform well it may be useless. But, really, before NIO it was
>> hopeless to try and write a scalable and fast tcp server application
>> in Java. Tomcat's current connection handling with blocing all over
>> the place and "thundering herd" problem doesn't scale or work very
>> well under heavy load.
>
> You apparently have a very strong opinion on this, and that's fine.
> You also obviously don't know what you are talking about. The purpose
> of Tomcat is to make the web tier of an application server (Tomcat is
> actually a mini application server), not some kind of non blocking I/O
> toolkit to be used to build fixed function servers. Non blocking I/O
> has great applications, and is a very useful technology, but it does
> not apply to the application server world.
>
> I think you should find a servlet container which has NIO, compare
> with Tomcat 5.0.16, and come back to report your findings about how
> much scalability or speed NIO brings (note: doing the non blocking
> socket handling in a native layer doesn't really count, since it's not
> a fair comparison with Java's NIO; you might as well use Apache).
>
> Bring facts, not useless rants.
I've lost the reference but did anyone else see the non-blocking NIO
"plug-in" that was supposed to fit right into Java things like Tomcat
and solve all their performance/scalability issues?
I saw this touted somewhere recently. It sounded *way* too good to be
true, but was interesting. I'm thinking it was called something like
"JEngine" or something equally vague...
What little I could make out from the info at the time seemed to suggest
that it might be nice (assuming it worked) if you wanted to allow Apache
to keep loads of connections open to a non-blocking IO layer in Tomcat
without burdening it as much as this would normally imply. Of course
the next layer of Tomcat needs to wait on the thread pool anyway, so I'm
not sure what the real benefit over a reasonably sized mod_jk[2]
connection pool really is...
--
Jess Holle
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Remy Maucherat <re...@apache.org>.
Jan-Henrik Haukeland wrote:
> Remy Maucherat <re...@apache.org> writes:
>
>>My opinion is that NIO is going to be really useless.
>
> Eh, hello!? Oh, okay if it's not important that Tomcat scale and
> perform well it may be useless. But, really, before NIO it was
> hopeless to try and write a scalable and fast tcp server application
> in Java. Tomcat's current connection handling with blocing all over
> the place and "thundering herd" problem doesn't scale or work very
> well under heavy load.
You apparently have a very strong opinion on this, and that's fine. You
also obviously don't know what you are talking about. The purpose of
Tomcat is to make the web tier of an application server (Tomcat is
actually a mini application server), not some kind of non blocking I/O
toolkit to be used to build fixed function servers. Non blocking I/O has
great applications, and is a very useful technology, but it does not
apply to the application server world.
I think you should find a servlet container which has NIO, compare with
Tomcat 5.0.16, and come back to report your findings about how much
scalability or speed NIO brings (note: doing the non blocking socket
handling in a native layer doesn't really count, since it's not a fair
comparison with Java's NIO; you might as well use Apache).
Bring facts, not useless rants.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Jan-Henrik Haukeland <ha...@tildeslash.com>.
Remy Maucherat <re...@apache.org> writes:
> My opinion is that NIO is going to be really useless.
Eh, hello!? Oh, okay if it's not important that Tomcat scale and
perform well it may be useless. But, really, before NIO it was
hopeless to try and write a scalable and fast tcp server application
in Java. Tomcat's current connection handling with blocing all over
the place and "thundering herd" problem doesn't scale or work very
well under heavy load.
> - It will make object reuse more complex (you'll have to have a pool
> holding all the HTTP processors, essentially)
You will have a pool holding incomming connection, and if there is an
i/o event on the connection such as data present you will hand the
connection over to a small pool of HTTP processors (5-10). Here's a
quasi example for the main server loop. (All sockets must be
non-blocking, including the serversocket, i.e. it must *not* block on
accept(). Every i/o operation must of course also be non-blocking).
This loop allows the server to handle many thousand concurrent
connections without using many threads.
for(;;) {
/* Poll the connection pool, including the server socket and
* return the number of sockets ready for processing */
numReady= Poll.poll();
if(Poll.hasConnection()) {
/* We have new connections on the server socket; accept
* every incomming connection and put them into the
* connection pool */
doAccept();
/* Continue checking for more incomming connection;
* new connections are more important than old */
continue;
}
if(numReady == 0) {
/* Close and remove connections that has timed out or
* are done */
Poll.checkTimeout();
/* Continue checking for more incomming connection */
continue;
}
/* Get and process only those connections that is
* ready. I.e. has data avilable. (More optimizing is to only
* hand over incomming connection that has read and buffered a
* complete HTTP request */
while((connection= Poll.next())) {
/* Push the connection into our HttpProcessor thread
* worker/queue for processing.
- Keep-alive connections are not closed but keept in
the connection pool and will be re-processed.
- Connections that are finished are closed by the
HttpProcessor and will be removed from the pool
in the test above */
if(! processorPush(connection)) {
/* If no more processor threads are available
* we process the request ourself. Alternatively
* increase the pool of processor threads */
processConnection(connection, NO_KEEPALIVE);
}
}
}
--
Jan-Henrik Haukeland
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Remy Maucherat <re...@apache.org>.
Jeanfrancois Arcand wrote:
> +1
>
> I would also like to explore, as a separate module, a connector that
> uses NIO to see what kind of performance we can have. I have no
> intention of replacing the current connector, and probably I will waste
> my time trying to have the same performance as the current connector.
> But I would like to explore it anyway .... ( latter next year, after the
> security stuff )....
My opinion is that NIO is going to be really useless.
- HTTP processing time is insignificant (BTW, most of the current
algorithm can be used in non blocking mode, I think)
- You have to go in blocking mode for the whole Catalina pipeline (ok:
fast) and for running the user application
- It will make object reuse more complex (you'll have to have a pool
holding all the HTTP processors, essentially)
So the benefit is for holding HTTP/1.1 persistent connections, given that:
- It will make the connector code more complex ( :-( )
- Modern OSes (Linux 2.6, Solaris with Sun's VMs, any Linux with JRockit
and its thread mapping similar to Solaris) will probably not really care
about saving a few threads
But, hey, if you have time, it's an interesting experiment (and I'll try
to help out a bit if I have time) :)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: 5.next + 4.1.x future
Posted by Jeanfrancois Arcand <jf...@apache.org>.
Remy Maucherat wrote:
> Hi,
>
> I see no reason to create a new branch, which is not what happened
> with previous releases, where branches were created sometimes even
> before the stable release (4.1.x got forked for 4.0 final, and 5.0.x
> was forked for 4.1.7). Reasons for no branching:
> - Tomcat is rather modular
> - Behavior in nearly all cases seems well established, so no major
> breakage is to be expected
> - Increased maintenance requirements due to diverting resource (we did
> have a problem with that, looking at all the BZ items)
> - Branching hasn't really been synonymous with stability or quality in
> the past, given patch porting decisions have been questionable
> sometimes (that's likely my fault, as the RM)
>
> We now have a stable 5.0.16, which so far seems like a quality
> release. Since 5.0.x is only an evolution of 4.1.x, I don't see much
> needs for more than one additional 4.1.x releases (assuming it's a
> good one without regressions :D ). If people want more, a new RM will
> need to pop up :) (BTW, I can pass the hot potato right now for 4.x if
> someone is interested :D )
>
> I'm interested in the following enhancements to 5.0.x in the future (=
> in january or later):
> - refactoring of the save to XML feature (that's been requested; I
> don't know if I'll use that to be able to use the admin webapp under
> JBoss, though); likely the default impl will remain the current one,
> but we'll be able to take out all that code from StandardServer :)
> - full support for a JMX-ized server.xml, which is used in the
> embedded distribution (I think a few components can't be created using
> that; I could be wrong though, Costin did a great job)
> - I'll add the minimal server.xml to embedded and some startup scripts
> and the native wrappers, and I'll "optimize" it some more, so that it
> can also be used as a minimal Tomcat installation (without any CL
> hierarchy, which sometimes confuses beginners)
> - tweaks and fixes
+1
I would also like to explore, as a separate module, a connector that
uses NIO to see what kind of performance we can have. I have no
intention of replacing the current connector, and probably I will waste
my time trying to have the same performance as the current connector.
But I would like to explore it anyway .... ( latter next year, after the
security stuff )....
-- Jeanfrancois
>
> Rémy
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org