You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mauricio Nuñez <mn...@maxmedia.cl> on 2002/01/08 00:27:15 UTC
nbio connector
Hi everybody!
I'm using Tomcat 3.3 and Apache on Linux, and i want to develop a connector
based on nbio (http://www.cs.berkeley.edu/~mdw/proj/java-nbio).
My focus is get a multiplex connector based on few threads waiting
connections, reducing the overhead on my server.
Firstly, i will port the ajp13 interceptor, based on only 2 threads (i guess)
The next step will be to migrate to java.nio, when Sun release a stable
jdk1.4.
Any comment?
Can i submit my code here?
I need to get a stable and scalable Tomcat on Linux. This work try to get
that.
Bye
Mauricio Nuñez
mnunez@maxmedia.cl
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
Yes, after Mauricio brought to my attention the existence of a non-JDK NBIO package few days ago. At this point in time my framework was almost complete. On the first glance, the Sandstorm services platform part of the SEDA quite resembles my effort but I'll have to also take a closer look.
Attila.
----- Original Message -----
From: "Daniel Rall" <dl...@finemaltcoding.com>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Cc: <mn...@maxmedia.cl>
Sent: 2002. január 10. 0:26
Subject: Re: nbio connector
>
> Have you looked at Matt Welsh's SEDA
> <http://www.cs.berkeley.edu/~mdw/proj/sandstorm/>?
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
Re: nbio connector
Posted by Daniel Rall <dl...@finemaltcoding.com>.
"Attila Szegedi" <sz...@freemail.hu> writes:
> Well I didn't want to bring this to the sunlight so soon but since
> you brought up the issue: I'm developing a generic non-blocking
> server framework for JDK 1.4.
>
> It handles all subtleties of the non-blocking server's life, such as
> non-blocking pipes for servlet output buffering, thread scaling
> (since a single selector handles at most 63 channels, you still need
> multiple threads), channel registration from a different thread than
> the thread that listens to the selector via a mechanism similar to
> Swing's invokeAndWait(), etc, etc. Using this framework, all you
> need to do is implement a single ProtocolHandler interface and focus
> your attention on the protocol implementation and not on the details
> of non-blocking I/O.
Have you looked at Matt Welsh's SEDA
<http://www.cs.berkeley.edu/~mdw/proj/sandstorm/>?
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
----- Original Message -----
From: "Remy Maucherat" <re...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: 2002. január 8. 16:35
Subject: Re: nbio connector
> > , and could
> > use a helping hand here and there. In fact, I already have much of
> > the code for building the HttpServletRequest object, but to keep
> > it really elegant I'll need some help from the Tomcat community.
> > Most notably, much of the code duplications could go away if Catalina
> > code in org.apache.catalina.connectors.http was more reusable
> > (some classes are package-private and some public classes take
> > package-private classes in a constructor argument etc.).
> > I'll return to these issues on this list after I've properly set
> > up the project on SourceForge.
>
> Don't use that code, as it is not very efficient / memory intensive, etc.
> Something more useful would be to write a nbio selector for the HTTP/1.1
> connector I'm writing in j-t-c.
>
I assume you're talking about coyote when refering to "connector I'm writing in j-t-c".
Could you be a bit more specific on that? I finally got the chance to jump on the NBIO HTTP connector implementation, and would like to know what is the current state of the coyote connector (seems like it's heavily in the primary "under construction" phase), and what is its long term goal? Will binary distributions of Tomcat include it once it's complete? Is it meant to become the default HTTP connector implementation shipped with Tomcat?
I'm now studying it for bits that could get reused in a NBIO implementation (that means mostly the response code; request processing algorithms can't be generalized accross blocking and non-blocking approach, though (this asymmetry has its deep reasons and is quite amusing)).
Attila.
>
> Remy
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
Re: nbio connector
Posted by Mauricio Nuñez <mn...@maxmedia.cl>.
El Miércoles 09 Enero 2002 06:52, Attila Szegedi escribió:
> ----- Original Message -----
> From: "Remy Maucherat" <re...@apache.org>
> To: "Tomcat Developers List" <to...@jakarta.apache.org>
> Sent: 2002. január 8. 17:35
> Subject: Re: nbio connector
>
> > I'm a bit sceptical about the usefulness of the thing, then, since
> > reading and parsing requests headers is by far the fastest part of the
> > current connector (all this is coming from many OptimizeIt runs I did).
> > The new
>
> Unless the input comes in slowly due to network conditions, or a deliberate
> DoS attack attempt. A client that opens many connections and then supplies
> input one byte per minute will bog down many threads and can cause a DoS
> with a blocking connector.
>
I agree. connections via modem it's a source of latency.
> > > 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely
> > > on the not-yet-final 1.4 JDK.
> >
> > Find something else. We already have JDK 1.4 flags in the build process,
> > so it's a very bad justification I'm afraid.
>
> I wasn't aware of the fact. This argument does not hold then, really.
>
The Nbio project by Matt Welsh is focuses on JDK1.3. He is part of the
java.nio Team, and i will wait some degree of "coherency" ;-) between both.
>
> Attila.
>
> > Remy
Mauricio
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by "Christopher K. St. John" <ck...@distributopia.com>.
Attila Szegedi wrote:
>
> (as it stands now in JDK 1.4, a single thread can manage up to 63
> channels so you still need multiple threads, only less than with
> blocking approach),
>
The 63 channel limit is a Windows-only bug in the Beta,
designing code around it probably isn't a good idea.
--
Christopher St. John cks@distributopia.com
DistribuTopia http://www.distributopia.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
----- Original Message -----
From: "Remy Maucherat" <re...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: 2002. január 8. 17:35
Subject: Re: nbio connector
>
> I'm a bit sceptical about the usefulness of the thing, then, since reading
> and parsing requests headers is by far the fastest part of the current
> connector (all this is coming from many OptimizeIt runs I did). The new
Unless the input comes in slowly due to network conditions, or a deliberate DoS attack attempt. A client that opens many connections and then supplies input one byte per minute will bog down many threads and can cause a DoS with a blocking connector.
>
> > 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely
> > on the not-yet-final 1.4 JDK.
>
> Find something else. We already have JDK 1.4 flags in the build process, so
> it's a very bad justification I'm afraid.
>
I wasn't aware of the fact. This argument does not hold then, really.
> > 2. My framework as it stands now is a generic framework
> > for implementing arbitrary TCP/IP services in a non-blocking fashion.
>
> It is an extension of nio from JDK 1.4 then ?
>
No. Having non-blocking sockets API is not yet a server framework. You still need to attend to many implementation details: output buffering (and the associated memory management), setting up threads that act as selector loops for accepting, reading and writing, creating and tearing down those threads as load rises/lowers (as it stands now in JDK 1.4, a single thread can manage up to 63 channels so you still need multiple threads, only less than with blocking approach), explicit key cancelling, etc. With the framework, the protocol implementor only has to implement a ProtocolHandlerFactory and ProtocolHandler interface, and need not be concerned with low-level connection management. The whole thing is not a rocket science, and is quite trivial AFTER I've spent several nights on trying out how nio works and resolving surfacing issues -- that's what I want the spare the fellow developers of.
> That's not true that way. The ASF wants the copyright on all code in its
> repositories. SO you're free to take our code, but we can't take yours
> without you donating it.
I see, that's really a distinction since in order for you to incorporate the code I have to transfer you the copyright. However, that's the consequence of the ASF policy and not of my licensing model (which is Apache-stlye itself). The "I'll have no objections to migrating it into Tomcat source tree." sentence still holds even if it means donating the code later on.
Attila.
>
> Remy
>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
----- Original Message -----
From: <co...@covalent.net>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: 2002. január 8. 20:35
Subject: Re: nbio connector
>
> I'm not very sure about nbio - most of the time there's a lot
> of complexity ( and a different programming model, etc ) - and
> the benefits seems pretty small. We already spend most of the
> time in user code ( i.e. servlets), the container overhead
> is getting smaller and smaller - so improvements on this
> area are very unlikely to matter. But it's cool - so
> I'm +1 on it as long as it doesn't mess with the existing
> APIs ( i.e. it is implemented as a module, etc ).
>
Another reason why I'm inclined to take on it as a separate project - the whole thing is highly experimental now; oh and the JDK is still in beta. Keeping it isolated for some time until it matures a bit lessens the chance it causes any negative interference with Tomcat in its early days.
Attila.
>
> Costin
>
>
>
>
>
>
>
>
>
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
Re: nbio connector
Posted by "Christopher K. St. John" <ck...@distributopia.com>.
costinm@covalent.net wrote:
>
> I'm not very sure about nbio - most of the time there's a lot
> of complexity ( and a different programming model, etc )
>
Yes.
> - and the benefits seems pretty small.
>
It depends. Using NIO, you can serve static content as fast
or faster than Apache 1.3 in pure Java, which means one less
reason to use a separate front-end web server. (Of course,
performance isn't the only reason to run Apache as a front
end to Tomcat, but it seems to be perceived as the biggest
reason)
--
Christopher St. John cks@distributopia.com
DistribuTopia http://www.distributopia.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by co...@covalent.net.
My opinion on the nbio issue:
I completely agree with Remy, the IO is not the issue at least
for the current code and the near future. The char-byte
conversions are far more expensive.
JDK1.4 provides a very nice solution for the char conversions,
however my tests show it to be comparable in speed ( i.e I
couldn't notice any difference ) with the current 'hack'
in tomcat.util.buf ( which uses normal Writer with some
recycling and other tricks ).
While I would love the clean 1.4 solution, I see no particular
need or benefit - so I'll not spend any time on it, but
if someone contributes that - I would be +1 to add it
to tomcat ( optional, if 1.4 is detected )
Regarding 'sendfile' - as part of j-t-c we'll be using a significant
chunk of APR ( the apache portable runtime ) - and it would be
easy to add support for sendfile as well. This will work on
any platform ( where Apache can run - that's probably >=
than java at least for servers ). I would rather use the APR
code that is available on all VMs and provides the same
feature.
I'm not very sure about nbio - most of the time there's a lot
of complexity ( and a different programming model, etc ) - and
the benefits seems pretty small. We already spend most of the
time in user code ( i.e. servlets), the container overhead
is getting smaller and smaller - so improvements on this
area are very unlikely to matter. But it's cool - so
I'm +1 on it as long as it doesn't mess with the existing
APIs ( i.e. it is implemented as a module, etc ).
Costin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Tom Drake <rt...@pobox.com>.
IMHO, the main benefit of non-blocking io has to do with improved
robustness. If a router or adapter goes off the air while you're in the
middle of a 'blocking' read, you are likely to never to see the light of
day.
This problem is exacerbated for client-side development because it
is 'difficult' to set SO_TIMEOUT on a URLConnection. Actually,
it's impossible, unless you provide your own URLStreamHandler.
And this requires that you start reading sun's java.net source code.
Just my two cents.
Tom Drake
----- Original Message -----
From: "Remy Maucherat" <re...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: Tuesday, January 08, 2002 8:35 AM
Subject: Re: nbio connector
| > I don't think I conflict with this requirement. In fact,
| > to execute a servlet's service() method still requires
| > a separate thread. In my design, the connector first reads
| > the request line and the headers in a non-blocking fashion,
| > naturally multiplexing up to 63 requests on a single thread.
| > I have a working HTTP processing state machine for that.
| > (You simply HAVE to write state machines in nb world, since
| > you can't encapsulate the algorithm in a single-thread linear
| > flow of methods because of multiplexing). Then it buffers the
| > whole request body using multiplexed non-blocking code. This
| > buffering is done using a special non-blocking pipe (more on
| > this below). After that, it constructs the request and response
| > objects, and fires off container.invoke(req, resp) on a separate
| > thread. The response object also has a non-blocking pipe that
| > to the servlet masquerades as a ServletOutputStream.
| > All this is done with the goal of completely avoiding any thread
| > running container.invoke(req, resp) to EVER block on a socket read or
| write.
|
| I'm a bit sceptical about the usefulness of the thing, then, since reading
| and parsing requests headers is by far the fastest part of the current
| connector (all this is coming from many OptimizeIt runs I did). The new
| connector should even be a bit faster for reading. OTOH, the output is
| extremely slow (not because it's blocking IO, but rather because of really
| bad char/byte conversion), and nbio won't help that. The GC problems of
the
| current connector is also a very significant issue which would take away
all
| the benefits nbio would bring.
|
| It's still very useful work as the final optimization for very high load
| scenarios.
|
| > With my design, you still need one thread/request but only for the
| > time required to process container.invoke()
|
| In the real world, the servlets and JSPs are the thing which take by far
the
| most time to complete, so I'm not sure you wouldn't end up spending a lot
of
| time in the threaded part.
|
| > 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely
| > on the not-yet-final 1.4 JDK.
|
| Find something else. We already have JDK 1.4 flags in the build process,
so
| it's a very bad justification I'm afraid.
|
| > 2. My framework as it stands now is a generic framework
| > for implementing arbitrary TCP/IP services in a non-blocking fashion.
|
| It is an extension of nio from JDK 1.4 then ?
|
| > 3. It would be tedious for me to send in CVS patches over long run.
|
| If your contributions are valuable, you would get commit access very
| quickly, but it's up to you.
|
| > If the Tomcat HTTP connector comes to life as part
| > of the nbserver project and works fine, I'll have no
| > objections to migrating it into Tomcat source tree.
| > In fact, since I'm publishing the code under Apache-style license,
| > I *CAN NOT* have any objections if it ever gets transferred into Tomcat
| source tree -
| > either with or without my assistance. :-)
|
| That's not true that way. The ASF wants the copyright on all code in its
| repositories. SO you're free to take our code, but we can't take yours
| without you donating it.
|
| Remy
|
|
| --
| To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
| For additional commands, e-mail:
<ma...@jakarta.apache.org>
|
|
|
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
----- Original Message -----
From: "Remy Maucherat" <re...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: 2002. január 8. 19:19
Subject: Re: nbio connector
>
> I agree on both the points above.
> Especially the first one actually, since you can avoid wasting a thread on
> waiting, and use all of them to do active work if needed.
> For the second point, Catalina uses a servlet to do the static files
> serving. I suppose the connector could do some optimization and serve some
> resources itself, but that's a bit complex, and not as clean as the current
> solution.
Or, the default servlet could check if the response is instanceof non-blocking response and supply it with the appropriate file channel for response body; the response would take care of instructing the connector to transfer the file using the direct filesystem cache-to-socket transferTo() method in its own non-blocking code. This way, the majority of the logic would be inside a connector; the servlet would only have few lines of additional code inside an
if(resp instanceof ...) block
>
> I like that difference between a fun optimization and a useful optimization
> :)
> At the moment, a lot of useful optimization work still has yet to be done.
>
I believe someone on this list once said something like "open source development: have fun or get paid for it". I'm working on NIO solely because it's a fun new technology in Java with a huge potential, and if it happens to turn out useful it's a clear win. :-)
Attila.
>
> Remy
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
Re: nbio connector
Posted by Remy Maucherat <re...@apache.org>.
> Remy Maucherat wrote:
> >
> > > With my design, you still need one thread/request but only for the
> > > time required to process container.invoke()
> >
> > In the real world, the servlets and JSPs are the thing which take by far
the
> > most time to complete, so I'm not sure you wouldn't end up spending a
lot of
> > time in the threaded part.
> >
>
> There was a long discussion on the advanced-servlets list
> a while back. That discussion concluded that the NIO looked
> most promising for:
>
> - Cleanly waiting for persistent connections (which take
> up a connector thread but don't really need to) You
> can kinda sorta do this already with the existing java.io
> classes, but NIO is better. This is a relatively easy
> win.
>
> - Serving static content. It's not just the non-blocking
> io, serving from memory mapped files is a beguilingly
> easy (but maybe not always the best) way to improve file
> serving performance.
>
> Using NIO to reduce the time service() needed to tie up
> a thread is fun, but seemed unlikely to help very much in
> practice. And of course the optimized char conversion
> stuff, but that's obviously useful and therefore boring.
I agree on both the points above.
Especially the first one actually, since you can avoid wasting a thread on
waiting, and use all of them to do active work if needed.
For the second point, Catalina uses a servlet to do the static files
serving. I suppose the connector could do some optimization and serve some
resources itself, but that's a bit complex, and not as clean as the current
solution.
I like that difference between a fun optimization and a useful optimization
:)
At the moment, a lot of useful optimization work still has yet to be done.
> I'm pretty sure newesh versions of Jetty include a first
> pass at integrating NIO, so anybody that's interested
> might want to take a look.
>
> There's some fairly bogus (but still kinda interesting)
> benchmark results using NIO-like routines to serve static
> files at distributopia.com. (Sun says no NIO-bmarks until
> it's out of beta, so I used Matt Welsh's NBIO code)
I didn't know. They're annoying ...
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by "Christopher K. St. John" <ck...@distributopia.com>.
Remy Maucherat wrote:
>
> > With my design, you still need one thread/request but only for the
> > time required to process container.invoke()
>
> In the real world, the servlets and JSPs are the thing which take by far the
> most time to complete, so I'm not sure you wouldn't end up spending a lot of
> time in the threaded part.
>
There was a long discussion on the advanced-servlets list
a while back. That discussion concluded that the NIO looked
most promising for:
- Cleanly waiting for persistent connections (which take
up a connector thread but don't really need to) You
can kinda sorta do this already with the existing java.io
classes, but NIO is better. This is a relatively easy
win.
- Serving static content. It's not just the non-blocking
io, serving from memory mapped files is a beguilingly
easy (but maybe not always the best) way to improve file
serving performance.
Using NIO to reduce the time service() needed to tie up
a thread is fun, but seemed unlikely to help very much in
practice. And of course the optimized char conversion
stuff, but that's obviously useful and therefore boring.
I'm pretty sure newesh versions of Jetty include a first
pass at integrating NIO, so anybody that's interested
might want to take a look.
There's some fairly bogus (but still kinda interesting)
benchmark results using NIO-like routines to serve static
files at distributopia.com. (Sun says no NIO-bmarks until
it's out of beta, so I used Matt Welsh's NBIO code)
--
Christopher St. John cks@distributopia.com
DistribuTopia http://www.distributopia.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Remy Maucherat <re...@apache.org>.
> I don't think I conflict with this requirement. In fact,
> to execute a servlet's service() method still requires
> a separate thread. In my design, the connector first reads
> the request line and the headers in a non-blocking fashion,
> naturally multiplexing up to 63 requests on a single thread.
> I have a working HTTP processing state machine for that.
> (You simply HAVE to write state machines in nb world, since
> you can't encapsulate the algorithm in a single-thread linear
> flow of methods because of multiplexing). Then it buffers the
> whole request body using multiplexed non-blocking code. This
> buffering is done using a special non-blocking pipe (more on
> this below). After that, it constructs the request and response
> objects, and fires off container.invoke(req, resp) on a separate
> thread. The response object also has a non-blocking pipe that
> to the servlet masquerades as a ServletOutputStream.
> All this is done with the goal of completely avoiding any thread
> running container.invoke(req, resp) to EVER block on a socket read or
write.
I'm a bit sceptical about the usefulness of the thing, then, since reading
and parsing requests headers is by far the fastest part of the current
connector (all this is coming from many OptimizeIt runs I did). The new
connector should even be a bit faster for reading. OTOH, the output is
extremely slow (not because it's blocking IO, but rather because of really
bad char/byte conversion), and nbio won't help that. The GC problems of the
current connector is also a very significant issue which would take away all
the benefits nbio would bring.
It's still very useful work as the final optimization for very high load
scenarios.
> With my design, you still need one thread/request but only for the
> time required to process container.invoke()
In the real world, the servlets and JSPs are the thing which take by far the
most time to complete, so I'm not sure you wouldn't end up spending a lot of
time in the threaded part.
> 1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely
> on the not-yet-final 1.4 JDK.
Find something else. We already have JDK 1.4 flags in the build process, so
it's a very bad justification I'm afraid.
> 2. My framework as it stands now is a generic framework
> for implementing arbitrary TCP/IP services in a non-blocking fashion.
It is an extension of nio from JDK 1.4 then ?
> 3. It would be tedious for me to send in CVS patches over long run.
If your contributions are valuable, you would get commit access very
quickly, but it's up to you.
> If the Tomcat HTTP connector comes to life as part
> of the nbserver project and works fine, I'll have no
> objections to migrating it into Tomcat source tree.
> In fact, since I'm publishing the code under Apache-style license,
> I *CAN NOT* have any objections if it ever gets transferred into Tomcat
source tree -
> either with or without my assistance. :-)
That's not true that way. The ASF wants the copyright on all code in its
repositories. SO you're free to take our code, but we can't take yours
without you donating it.
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
----- Original Message -----
From: "Remy Maucherat" <re...@apache.org>
To: "Tomcat Developers List" <to...@jakarta.apache.org>
Sent: 2002. január 8. 16:35
Subject: Re: nbio connector
> > My primary goal for bringing the code in the public is that I want
> > to build a HTTP/1.1 connector for Tomcat 4.0 based on it
>
> I don't think you can have a nbio connector with a compliant servlet
> container, because the J2EE spec mandates at the moment: 1 thread <-> 1
> request.
I don't think I conflict with this requirement. In fact, to execute a servlet's service() method still requires a separate thread. In my design, the connector first reads the request line and the headers in a non-blocking fashion, naturally multiplexing up to 63 requests on a single thread. I have a working HTTP processing state machine for that. (You simply HAVE to write state machines in nb world, since you can't encapsulate the algorithm in a single-thread linear flow of methods because of multiplexing). Then it buffers the whole request body using multiplexed non-blocking code. This buffering is done using a special non-blocking pipe (more on this below). After that, it constructs the request and response objects, and fires off container.invoke(req, resp) on a separate thread. The response object also has a non-blocking pipe that to the servlet masquerades as a ServletOutputStream.
All this is done with the goal of completely avoiding any thread running container.invoke(req, resp) to EVER block on a socket read or write.
A special non-blocking pipe is more-or-less an ordinary pipe: one input stream and one output stream, however it does not have a fixed-size buffer (because that would block when the buffer gets full), instead it manages a growing list of buffers and can hold any size of buffered bytes. It's used both for buffering request body, and for buffering the response. The response pipe is written to by the servlet and read from by the non-blocking code that transfers its data to the socket channel; so in the response phase the pipe buffers that part of the output that the servlet has already generated, but that still couldn't get pushed through the network interface. Thus, it eliminates I/O block in response.getOutputStream().write(...) calls. As an additional memory optimization, it starts buffering in memory, but if the buffered content gets large it switches to buffering in a memory-mapped file (also new in JDK 1.4).
> That requirements makes nbio much less interesting over the current since
> you would need a big thread pool.
With my design, you still need one thread/request but only for the time required to process container.invoke() (that is, while the filters and servlets run). Since the design also eliminates I/O blocking in those threads, they can complete as quickly as their code otherwise allows, so the overall requirement for threads is substantially lower than in a blocking implementation.
>
> > , and could
> > use a helping hand here and there. In fact, I already have much of
> > the code for building the HttpServletRequest object, but to keep
> > it really elegant I'll need some help from the Tomcat community.
> > Most notably, much of the code duplications could go away if Catalina
> > code in org.apache.catalina.connectors.http was more reusable
> > (some classes are package-private and some public classes take
> > package-private classes in a constructor argument etc.).
> > I'll return to these issues on this list after I've properly set
> > up the project on SourceForge.
>
> Don't use that code, as it is not very efficient / memory intensive, etc.
> Something more useful would be to write a nbio selector for the HTTP/1.1
> connector I'm writing in j-t-c.
> For the connector, nbio is the last optimization which should be done. There
> are much bigger hotspots than the time spent in java.net in the connector
> which ships with 4.0.x ;-)
>
> Also, why not contributing the thing to the core project (if it manages to
> get around the problem mentioned above, of course), instead of developing it
> separately ?
There are several reasons:
1. It needs JDK 1.4 to compile, and I believe Tomcat should not rely on the not-yet-final 1.4 JDK.
I know, this can be dealt with using conditional compilation in the build.xml file, as it does now with optional components.
2. My framework as it stands now is a generic framework for implementing arbitrary TCP/IP services in a non-blocking fashion. Having a Tomcat connector built upon it is only one possible application. SMTP, FTP, etc. servers can also be written upon this framework.
3. It would be tedious for me to send in CVS patches over long run. While I have no problem with this approach in general (I have already contributed to both Velocity and Ant by sending patches to committers), on SourceForge I have my own CVS I can commit to right now, and I need this level of control. I don't doubt, that the community would grant me committer status on Tomcat over time if I contribute regularly, however I plan to work intensively on this code and I fear the inability to have a CVS to which I can commit to right from the start would somewhat bog me down.
If the Tomcat HTTP connector comes to life as part of the nbserver project and works fine, I'll have no objections to migrating it into Tomcat source tree. In fact, since I'm publishing the code under Apache-style license, I *CAN NOT* have any objections if it ever gets transferred into Tomcat source tree - either with or without my assistance. :-)
Cheers,
Attila.
>
> Remy
>
>
Re: nbio connector
Posted by Remy Maucherat <re...@apache.org>.
> My primary goal for bringing the code in the public is that I want
> to build a HTTP/1.1 connector for Tomcat 4.0 based on it
I don't think you can have a nbio connector with a compliant servlet
container, because the J2EE spec mandates at the moment: 1 thread <-> 1
request.
That requirements makes nbio much less interesting over the current since
you would need a big thread pool.
> , and could
> use a helping hand here and there. In fact, I already have much of
> the code for building the HttpServletRequest object, but to keep
> it really elegant I'll need some help from the Tomcat community.
> Most notably, much of the code duplications could go away if Catalina
> code in org.apache.catalina.connectors.http was more reusable
> (some classes are package-private and some public classes take
> package-private classes in a constructor argument etc.).
> I'll return to these issues on this list after I've properly set
> up the project on SourceForge.
Don't use that code, as it is not very efficient / memory intensive, etc.
Something more useful would be to write a nbio selector for the HTTP/1.1
connector I'm writing in j-t-c.
For the connector, nbio is the last optimization which should be done. There
are much bigger hotspots than the time spent in java.net in the connector
which ships with 4.0.x ;-)
Also, why not contributing the thing to the core project (if it manages to
get around the problem mentioned above, of course), instead of developing it
separately ?
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: nbio connector
Posted by Attila Szegedi <sz...@freemail.hu>.
Well I didn't want to bring this to the sunlight so soon but since you brought up the issue:
I'm developing a generic non-blocking server framework for JDK 1.4.
It handles all subtleties of the non-blocking server's life, such as non-blocking pipes for servlet output buffering, thread scaling (since a single selector handles at most 63 channels, you still need multiple threads), channel registration from a different thread than the thread that listens to the selector via a mechanism similar to Swing's invokeAndWait(), etc, etc. Using this framework, all you need to do is implement a single ProtocolHandler interface and focus your attention on the protocol implementation and not on the details of non-blocking I/O.
The initial code is practically done, there is only one JUnit test left to pass. I've created a project for it on SourceForge (http://sourceforge.net/projects/nbserver) *today* (see, it's really fresh) and will upload the code there soon. The project is pretty much empty now, but I'll populate the CVS and the introductory website in the following days. It will be distributed under Apache-style license.
My primary goal for bringing the code in the public is that I want to build a HTTP/1.1 connector for Tomcat 4.0 based on it, and could use a helping hand here and there. In fact, I already have much of the code for building the HttpServletRequest object, but to keep it really elegant I'll need some help from the Tomcat community. Most notably, much of the code duplications could go away if Catalina code in org.apache.catalina.connectors.http was more reusable (some classes are package-private and some public classes take package-private classes in a constructor argument etc.). I'll return to these issues on this list after I've properly set up the project on SourceForge.
I wanted to keep the whole project out of the spotlight until it's up and running, but since you mentioned the issue, I thought you should know about similar efforts. Maybe you will want to take a look at it; I can notify you when the SourceForge project site is ready.
Cheers,
Attila.
----- Original Message -----
From: "Mauricio Nuñez" <mn...@maxmedia.cl>
To: <to...@jakarta.apache.org>
Sent: 2002. január 8. 0:27
Subject: nbio connector
> Hi everybody!
>
> I'm using Tomcat 3.3 and Apache on Linux, and i want to develop a connector
> based on nbio (http://www.cs.berkeley.edu/~mdw/proj/java-nbio).
>
> My focus is get a multiplex connector based on few threads waiting
> connections, reducing the overhead on my server.
>
> Firstly, i will port the ajp13 interceptor, based on only 2 threads (i guess)
>
> The next step will be to migrate to java.nio, when Sun release a stable
> jdk1.4.
>
> Any comment?
> Can i submit my code here?
>
> I need to get a stable and scalable Tomcat on Linux. This work try to get
> that.
>
> Bye
>
> Mauricio Nuñez
> mnunez@maxmedia.cl
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>