You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Oleg Kalnichevski <ol...@apache.org> on 2006/10/19 13:05:47 UTC
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP
transport
Folks,
A short update just to keep you all in the loop.
The event-driven HTTP transport I have been working in the past days is
still too flaky to be committed to SVN. It may take me another while
before I am (more or less) sure the new code in NIO extensions is ready
for the first round of reviews. However I feel the public interfaces
(API) are reasonably complete and _should_ stay stable while I am still
hacking on the implementation. There is already enough substance to see
how things are shaping up and give me some feedback. If any of you want
to take an early peek at the API please let me know and I'll check the
interfaces in.
Cheers
Oleg
On Thu, 2006-10-12 at 22:58 +0200, Oleg Kalnichevski wrote:
> On Mon, 2006-10-09 at 15:38 -0400, Sam Berlin wrote:
> > I can't argue with that. :)
> >
> > Perhaps, while you're developing the API, keep the door open in your
> > mind about this blocking model being just one kind of interface to the
> > data? That is, considering reading the data via a pluggable
> > 'interpreter' of some sort that's installed over the socket and parses
> > the data to set headers (and other things). That way, in the future
> > when more time is available for everyone, it won't require breaking
> > backwards compatibility in order to retrofit non-blocking I/O.
> >
>
> Folks,
>
> After having spent some time evaluating the two options, I finally
> concluded we should take time and develop a even driven API for the
> non-blocking HTTP transport as a first step.
>
> All existing NIO HTTP transport implementations that rely on the
> InputStream / OutputStream abstraction I have seen so far suffer from
> the same fatal flaw. They tend to do excessive amount of intermediate
> buffer coping and are very prone to 'out of memory' conditions under
> heavy load. My goal is to make sure that the NIO based transport is not
> only non-blocking but is also memory efficient. In those cases where no
> content encoding/decoding is involved I want to make sure that the HTTP
> service can _directly_ write to / read from the underlying socket
> channel without any intermediate buffering.
>
> This decision also entails some refactoring in HttpCore proper, as I
> need certain classes decoupled from InputStream / OutputStream in order
> to be usable in NIO extensions. I will have to factor out the content
> length strategy code from the DefaultEntityDeserializer and
> DefaultEntitySerializer classes.
>
> Any major objections to that?
>
> Cheers
>
> Evil Comrade Oleg
>
> > I also apologize for only providing my thoughts and not actual code
> > (as I've said I would a few times over the past few years), and am
> > very grateful to you and Roland and the others who do provide code.
> > As I'm sure you all know, it's difficult to find the time around a day
> > job that is constantly shifting directions.
> >
> > Sam
> >
> > On 10/9/06, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > On Mon, 2006-10-09 at 13:26 -0400, Sam Berlin wrote:
> > > > I agree with Robert that it is much easier to go from an event driven
> > > > model to a blocking model. If the first layer that HttpNIO exposes is
> > > > blocking, there'd need to be additional hacking below that in order to
> > > > remove the blocking / thread-based layer. On the other hand, if the
> > > > first layer it exposes is non-blocking, it's relatively trivial to add
> > > > a thread ontop of that and expose an additional blocking layer.
> > > >
> > > > It is difficult to think of many scenarios that require (or a better
> > > > with) non-blocking I/O, but I would caution against excluding them
> > > > from HttpClient's scope. If HttpClient 4.0 had been ready a year or
> > > > so ago (with an exposed non-blocking layer), we would definitely have
> > > > used it in LimeWire as the basis of file-transfers. As-is, we
> > > > invented our own minimalistic non-blocking state-based http transport
> > > > for downloads.
> > > >
> > > > If the non-blocking layer is there, I guarantee that folks will be
> > > > able to find a use for it. Whereas if only a blocking layer is there,
> > > > those developers looking for the high-performance asyncronous model
> > > > will have to go elsewhere.
> > > >
> > >
> > > Sam, Robert, et al
> > >
> > > Simply for practical reasons while there are only two guys hacking (me
> > > and Roland) we ought not spread out efforts too thin. A full-blown
> > > even-driven API will take time to get right. I think it is more
> > > important to get HttpCore ALPHA3 release that covers 95% of use cases
> > > out the door rather sooner than later. Beyond that it is just a matter
> > > of priorities and available time.
> > >
> > > Oleg
> > >
> > >
> > > > Sam
> > > >
> > > > On 10/9/06, Robert Olofsson <ro...@khelekore.org> wrote:
> > > > >
> > > > > Since my proxy is almost fully nio/event based I would like to share
> > > > > a few comments.
> > > > >
> > > > > Oleg Kalnichevski wrote:
> > > > > > I am still quite skeptical about usefulness of a fully event-driven HTTP
> > > > > > transport for one simple reason: asynchronous (non-blocking) I/O
> > > > > > transport makes no sense of what so ever if the process of content
> > > > > > generation or content consumption is asynchronous (blocking).
> > > > >
> > > > > There are many things that may block here are a few examples:
> > > > > *) DNS-lookup
> > > > > *) File reading writing
> > > > > *) Database access
> > > > > *) All higher level api:s that only give you a stream.
> > > > > *) Calls to Runtime.exec
> > > > >
> > > > > That DNS lookups are also totally single threaded in native code in
> > > > > some systems does not make things better.
> > > > >
> > > > > > If one
> > > > > > needs a worker thread to generate / process content anyways, what is the
> > > > > > point of having an even driven transport?
> > > > >
> > > > > Agreed.
> > > > > One objection here may be that you do not need one worker thread for all
> > > > > of the content generation, but that usually does not make things better.
> > > > > You will still need the worker thread for the _slow_and_blocking_
> > > > > operation.
> > > > > So if content generation/modification uses any of the above then using
> > > > > workers simplify things a lot.
> > > > >
> > > > > > I see only a few scenarios
> > > > > > where the third choice (event callbacks) may prove advantageous,
> > > > > > primarily in HTTP proxies and gateways.
> > > > >
> > > > > Except that http proxies does lots of dns lookups so they will block
> > > > > a lot. My proxy spawns worker thread only when they need to, but it
> > > > > complicates some part of the code.
> > > > >
> > > > > That my proxy also modifies the content and caches the data will mean
> > > > > lots of other blocking calls in some of the code paths.
> > > > >
> > > > > > I think ultimately we need both options. I suggest we start with the
> > > > > > second option, release ALPHA3 and then consider implementing the third
> > > > > > option before ALPHA4 / BETA1.
> > > > >
> > > > > One thing to keep in mind:
> > > > > It is easier to go from event driven to a blocking model than to do the
> > > > > reverse. This may be an argument to go for number 3 (full nio).
> > > > > If you go for 3 then make it easy to use a few selector-threads
> > > > > otherwise the system will use only 1 cpu (or 1 core).
> > > > >
> > > > > /robo
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> > > >
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by Sam Berlin <sb...@gmail.com>.
> Hmmm, so whom do the dns lookups? or have you found a dns library that
> is nio as well?
That's a great question -- if you're able to find a library that does
asynchronous DNS lookups, please share it here!
Sam
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by Robert Olofsson <ro...@khelekore.org>.
Oleg Kalnichevski wrote:
> The I/O reactor works with abstract SocketAddress-es. HttpCore makes no
> assumption whether a particular SocketAddress represents a DNS host name
> or an IP address. The process of DNS name resolution is out of HttpCore
> scope. In practical terms that does mean the I/O thread can be blocked
> while a specific DNS name is being resolved. As this problem cannot be
> solved without an external dependency, I am personally prepared to live
> with this limitation.
This is a bit tricky. It would be good to have an interface that does
the lookup asynchronous. I do something like that in my proxy to spawn a
worker thread that does the lookup. DNS timeouts can easily be 2
minutes, handling many connections with that may be acceptable for
a simple client, but can easily be very bad if you want to build a web
spider or a web proxy or ...
Sam:
I use the dnsjava package to do dns lookups in rabbit. It is a dns
library in pure java. Internally it uses some parts of nio, but the
interfaces it provides are blocking calls.
I would like to have time to fix dnsjava to provide an asynchronous
interface, but as it is now I do not have the time.
http://www.xbill.org/dnsjava/
> Absolutely. I/O operations over NHttpConnections can be executed in full duplex.
Glad to hear that.
/robo
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP
transport
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2006-10-19 at 20:29 +0200, Robert Olofsson wrote:
> Oleg Kalnichevski wrote:
> > The API is
> > filly event driven. There is no need for worker threads at any point of
> > time. All content processing takes place at the I/O reactor thread.
>
> Hmmm, so whom do the dns lookups? or have you found a dns library that
> is nio as well?
>
Hi Robert,
The I/O reactor works with abstract SocketAddress-es. HttpCore makes no
assumption whether a particular SocketAddress represents a DNS host name
or an IP address. The process of DNS name resolution is out of HttpCore
scope. In practical terms that does mean the I/O thread can be blocked
while a specific DNS name is being resolved. As this problem cannot be
solved without an external dependency, I am personally prepared to live
with this limitation.
> Suns version will call native code that will be very single threaded
> on some systems and have long timeouts on some missing names.
>
> The typos seems to have been fixed already..
>
> One question: is it possible to use requestInput and requestOutput at
> the same time? That is needed to handle pipelined requests in a fast
> and efficent way.
> From reading the javadoc I would say that it is possible, but just want
> to check.
>
Absolutely. I/O operations over NHttpConnections can be executed in full duplex.
Oleg
>
> /robo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by Robert Olofsson <ro...@khelekore.org>.
Oleg Kalnichevski wrote:
> The API is
> filly event driven. There is no need for worker threads at any point of
> time. All content processing takes place at the I/O reactor thread.
Hmmm, so whom do the dns lookups? or have you found a dns library that
is nio as well?
Suns version will call native code that will be very single threaded
on some systems and have long timeouts on some missing names.
The typos seems to have been fixed already..
One question: is it possible to use requestInput and requestOutput at
the same time? That is needed to handle pipelined requests in a fast
and efficent way.
From reading the javadoc I would say that it is possible, but just want
to check.
/robo
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP
transport
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2006-10-19 at 13:17 +0100, ant elder wrote:
> I'd be interested in having a peek.
>
> ...ant
>
All right. Here it is.
http://svn.apache.org/viewvc?view=rev&revision=465656
The main goal of this API is to provide a non-blocking HTTP transport
with a minimal memory footprint and without excessive content buffering,
thus avoiding 'out of memory' conditions under heavy load. The API is
filly event driven. There is no need for worker threads at any point of
time. All content processing takes place at the I/O reactor thread.
When the implementation is reasonably stable I would like to 'bolt' our
standard HttpCore protocol layer on top of this API in order to provide
a convenient and consistent HTTP service interface for those
applications that rely on RequestInterceptor / ResponseInterceptor
interfaces for protocol handling and InputStream / OutputStream
interfaces for content manipulation and can afford a worker thread per
request / response.
Comments, suggestions, critique will be highly appreciated.
Evil Comrade Oleg
==============================
Abstract non-blocking HTTP connection interface. It can be used to
request or temporarily suspend event notifications that are triggered
when the underlying channel is ready for input / output operations
------------------
public interface NHttpConnection extends HttpConnection {
void requestInput();
void suspendInput();
void requestOutput();
void suspendOutput();
HttpContext getContext();
}
Abstract non-blocking client-side HTTP connection. It can be used to
submit HTTP requests and asynchronously receive HTTP responses
------------------
public interface NHttpClientConnection extends NHttpConnection {
HttpResponse getHttpResponse();
void submitRequest(HttpRequest request) throws HttpException;
boolean isRequestSubmitted();
}
Abstract non-blocking server-side HTTP connection. It can be used to
receive HTTP requests and asynchronously submit HTTP responses.
------------------
public interface NHttpServerConnection extends NHttpConnection {
HttpRequest getHttpRequest();
void submitContinue() throws HttpException;
void submitResponse(HttpResponse response) throws HttpException;
boolean isResponseSubmitted();
}
Abstract server-side HTTP event handler
------------------
public interface NHttpServiceHandler {
void requestReceived(NHttpServerConnection conn);
void inputReady(NHttpServerConnection conn,
ContentDecoder decoder);
void outputReady(NHttpServerConnection conn,
ContentEncoder encoder);
void exception(NHttpServerConnection conn, IOException ex);
void exception(NHttpServerConnection conn, HttpException ex);
void timeout(NHttpServerConnection conn);
void closed(NHttpServerConnection conn);
}
Abstract HTTP content decoder. HTTP content decoders can be used to read
entity content from the underlying channel in small chunks and apply the
required coding transformation.
------------------
public interface ContentDecoder {
void read(ByteBuffer dst) throws IOException;
boolean isCompeted();
}
Abstract HTTP content encoder. HTTP content encoders can be used to
apply the required coding transformation and write entity content to the
underlying channel in small chunks.
------------------
public interface ContentEncoder {
void write(ByteBuffer src) throws IOException;
void complete() throws IOException;
boolean isCompeted();
}
========================================
> On 10/19/06, Oleg Kalnichevski <ol...@apache.org> wrote:
> >
> > Folks,
> >
> > A short update just to keep you all in the loop.
> >
> > The event-driven HTTP transport I have been working in the past days is
> > still too flaky to be committed to SVN. It may take me another while
> > before I am (more or less) sure the new code in NIO extensions is ready
> > for the first round of reviews. However I feel the public interfaces
> > (API) are reasonably complete and _should_ stay stable while I am still
> > hacking on the implementation. There is already enough substance to see
> > how things are shaping up and give me some feedback. If any of you want
> > to take an early peek at the API please let me know and I'll check the
> > interfaces in.
> >
> > Cheers
> >
> > Oleg
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by ant elder <an...@gmail.com>.
I'd be interested in having a peek.
...ant
On 10/19/06, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> Folks,
>
> A short update just to keep you all in the loop.
>
> The event-driven HTTP transport I have been working in the past days is
> still too flaky to be committed to SVN. It may take me another while
> before I am (more or less) sure the new code in NIO extensions is ready
> for the first round of reviews. However I feel the public interfaces
> (API) are reasonably complete and _should_ stay stable while I am still
> hacking on the implementation. There is already enough substance to see
> how things are shaping up and give me some feedback. If any of you want
> to take an early peek at the API please let me know and I'll check the
> interfaces in.
>
> Cheers
>
> Oleg
>
>
> On Thu, 2006-10-12 at 22:58 +0200, Oleg Kalnichevski wrote:
> > On Mon, 2006-10-09 at 15:38 -0400, Sam Berlin wrote:
> > > I can't argue with that. :)
> > >
> > > Perhaps, while you're developing the API, keep the door open in your
> > > mind about this blocking model being just one kind of interface to the
> > > data? That is, considering reading the data via a pluggable
> > > 'interpreter' of some sort that's installed over the socket and parses
> > > the data to set headers (and other things). That way, in the future
> > > when more time is available for everyone, it won't require breaking
> > > backwards compatibility in order to retrofit non-blocking I/O.
> > >
> >
> > Folks,
> >
> > After having spent some time evaluating the two options, I finally
> > concluded we should take time and develop a even driven API for the
> > non-blocking HTTP transport as a first step.
> >
> > All existing NIO HTTP transport implementations that rely on the
> > InputStream / OutputStream abstraction I have seen so far suffer from
> > the same fatal flaw. They tend to do excessive amount of intermediate
> > buffer coping and are very prone to 'out of memory' conditions under
> > heavy load. My goal is to make sure that the NIO based transport is not
> > only non-blocking but is also memory efficient. In those cases where no
> > content encoding/decoding is involved I want to make sure that the HTTP
> > service can _directly_ write to / read from the underlying socket
> > channel without any intermediate buffering.
> >
> > This decision also entails some refactoring in HttpCore proper, as I
> > need certain classes decoupled from InputStream / OutputStream in order
> > to be usable in NIO extensions. I will have to factor out the content
> > length strategy code from the DefaultEntityDeserializer and
> > DefaultEntitySerializer classes.
> >
> > Any major objections to that?
> >
> > Cheers
> >
> > Evil Comrade Oleg
> >
> > > I also apologize for only providing my thoughts and not actual code
> > > (as I've said I would a few times over the past few years), and am
> > > very grateful to you and Roland and the others who do provide code.
> > > As I'm sure you all know, it's difficult to find the time around a day
> > > job that is constantly shifting directions.
> > >
> > > Sam
> > >
> > > On 10/9/06, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > > On Mon, 2006-10-09 at 13:26 -0400, Sam Berlin wrote:
> > > > > I agree with Robert that it is much easier to go from an event
> driven
> > > > > model to a blocking model. If the first layer that HttpNIO
> exposes is
> > > > > blocking, there'd need to be additional hacking below that in
> order to
> > > > > remove the blocking / thread-based layer. On the other hand, if
> the
> > > > > first layer it exposes is non-blocking, it's relatively trivial to
> add
> > > > > a thread ontop of that and expose an additional blocking layer.
> > > > >
> > > > > It is difficult to think of many scenarios that require (or a
> better
> > > > > with) non-blocking I/O, but I would caution against excluding them
> > > > > from HttpClient's scope. If HttpClient 4.0 had been ready a year
> or
> > > > > so ago (with an exposed non-blocking layer), we would definitely
> have
> > > > > used it in LimeWire as the basis of file-transfers. As-is, we
> > > > > invented our own minimalistic non-blocking state-based http
> transport
> > > > > for downloads.
> > > > >
> > > > > If the non-blocking layer is there, I guarantee that folks will be
> > > > > able to find a use for it. Whereas if only a blocking layer is
> there,
> > > > > those developers looking for the high-performance asyncronous
> model
> > > > > will have to go elsewhere.
> > > > >
> > > >
> > > > Sam, Robert, et al
> > > >
> > > > Simply for practical reasons while there are only two guys hacking
> (me
> > > > and Roland) we ought not spread out efforts too thin. A full-blown
> > > > even-driven API will take time to get right. I think it is more
> > > > important to get HttpCore ALPHA3 release that covers 95% of use
> cases
> > > > out the door rather sooner than later. Beyond that it is just a
> matter
> > > > of priorities and available time.
> > > >
> > > > Oleg
> > > >
> > > >
> > > > > Sam
> > > > >
> > > > > On 10/9/06, Robert Olofsson <ro...@khelekore.org> wrote:
> > > > > >
> > > > > > Since my proxy is almost fully nio/event based I would like to
> share
> > > > > > a few comments.
> > > > > >
> > > > > > Oleg Kalnichevski wrote:
> > > > > > > I am still quite skeptical about usefulness of a fully
> event-driven HTTP
> > > > > > > transport for one simple reason: asynchronous (non-blocking)
> I/O
> > > > > > > transport makes no sense of what so ever if the process of
> content
> > > > > > > generation or content consumption is asynchronous (blocking).
> > > > > >
> > > > > > There are many things that may block here are a few examples:
> > > > > > *) DNS-lookup
> > > > > > *) File reading writing
> > > > > > *) Database access
> > > > > > *) All higher level api:s that only give you a stream.
> > > > > > *) Calls to Runtime.exec
> > > > > >
> > > > > > That DNS lookups are also totally single threaded in native code
> in
> > > > > > some systems does not make things better.
> > > > > >
> > > > > > > If one
> > > > > > > needs a worker thread to generate / process content anyways,
> what is the
> > > > > > > point of having an even driven transport?
> > > > > >
> > > > > > Agreed.
> > > > > > One objection here may be that you do not need one worker thread
> for all
> > > > > > of the content generation, but that usually does not make things
> better.
> > > > > > You will still need the worker thread for the
> _slow_and_blocking_
> > > > > > operation.
> > > > > > So if content generation/modification uses any of the above then
> using
> > > > > > workers simplify things a lot.
> > > > > >
> > > > > > > I see only a few scenarios
> > > > > > > where the third choice (event callbacks) may prove
> advantageous,
> > > > > > > primarily in HTTP proxies and gateways.
> > > > > >
> > > > > > Except that http proxies does lots of dns lookups so they will
> block
> > > > > > a lot. My proxy spawns worker thread only when they need to, but
> it
> > > > > > complicates some part of the code.
> > > > > >
> > > > > > That my proxy also modifies the content and caches the data will
> mean
> > > > > > lots of other blocking calls in some of the code paths.
> > > > > >
> > > > > > > I think ultimately we need both options. I suggest we start
> with the
> > > > > > > second option, release ALPHA3 and then consider implementing
> the third
> > > > > > > option before ALPHA4 / BETA1.
> > > > > >
> > > > > > One thing to keep in mind:
> > > > > > It is easier to go from event driven to a blocking model than to
> do the
> > > > > > reverse. This may be an argument to go for number 3 (full nio).
> > > > > > If you go for 3 then make it easy to use a few selector-threads
> > > > > > otherwise the system will use only 1 cpu (or 1 core).
> > > > > >
> > > > > > /robo
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail:
> httpclient-dev-unsubscribe@jakarta.apache.org
> > > > > > For additional commands, e-mail:
> httpclient-dev-help@jakarta.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail:
> httpclient-dev-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail:
> httpclient-dev-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail:
> httpclient-dev-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> httpclient-dev-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> httpclient-dev-help@jakarta.apache.org
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP
transport
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2006-10-19 at 14:20 -0400, Sam Berlin wrote:
> > I think it is pointless to expect a high level protocol handler to be
> > able to recover from runtime exceptions and IOException and
> > HttpException are the only two types of checked exceptions that can be
> > thrown. Anyways, I can live with it either way.
>
> I'm not sure if I interpreted this correctly, but the thread that
> manages the selector must be prepared to handle all kinds of
> exceptions (Throwables & RuntimeExceptions included), otherwise any
> stray error would halt all I/O processing. I suggest having some kind
> of callback to inform the user of an arbitrary exception. Normally I
> would be a staunch advocate for letting any exception propogate as far
> up the stack as possible, but when the entire I/O subsystem can be
> halted as a result of any error, I think it's worthwhile to disregard
> that rule and catch any error.
>
> Sam
>
Sam, et al
I believe the situation should be looked at from a somewhat different
angle. I still think it makes no sense to propagate runtime exceptions
from the I/O thread to individual handlers, whereas safeguards must be
in place to ensure that a stray runtime exception thrown by a protocol
handler cannot take down the whole I/O reactor by killing its thread.
Oleg
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by Sam Berlin <sb...@gmail.com>.
> I think it is pointless to expect a high level protocol handler to be
> able to recover from runtime exceptions and IOException and
> HttpException are the only two types of checked exceptions that can be
> thrown. Anyways, I can live with it either way.
I'm not sure if I interpreted this correctly, but the thread that
manages the selector must be prepared to handle all kinds of
exceptions (Throwables & RuntimeExceptions included), otherwise any
stray error would halt all I/O processing. I suggest having some kind
of callback to inform the user of an arbitrary exception. Normally I
would be a staunch advocate for letting any exception propogate as far
up the stack as possible, but when the entire I/O subsystem can be
halted as a result of any error, I think it's worthwhile to disregard
that rule and catch any error.
Sam
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking
HTTP transport
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2006-10-19 at 17:48 +0200, Roland Weber wrote:
> Hi Oleg,
>
> just a short glimpse...
>
> ContentDecoder.isCompeted should be .isCompleted
> ContentEncoder.isCompeted should be .isCompleted
>
Oopsie.
> NHttpClientConnection.getHttpResponse()
> NHttpServerConnection.getHttpRequest()
> We've avoided tying the request and response directly so far.
> But it's probably not a big issue, so leave it that way.
>
> NHttpServerConnection.submitContinue()
> I don't like this at all. You are assuming that 100 continue
> is the only preliminary response that will ever be sent.
> That is true for the current HTTP specifications, but HTTP
> explicitly allows for extensions. I don't see why we should
> restrict our API in this case. Alternatives:
>
> 1. NHttpServerConnection.submitPreliminary(HttpResponse)
> 2. NHttpServerConnection.submitResponse(HttpResponse, boolean preliminary)
>
It just hit me. We do not a special method for 1xx responses at all. We
simply treat 1xx responses as intermediate (preliminary) ones. The
connection will remain in 'response uncommitted' state until a non 1xx
response has been committed. That is it.
> NHttpClientHandler.exception(...)
> Will the client handler get a chance to handle other kinds
> of exceptions, too? I usually prefer a generic error handling
> method, where the handler decides what to do (if anything):
>
> /** @return true if handled, false otherwise */
> boolean NHttpClientHandler.exception(conn, Throwable)
>
I think it is pointless to expect a high level protocol handler to be
able to recover from runtime exceptions and IOException and
HttpException are the only two types of checked exceptions that can be
thrown. Anyways, I can live with it either way.
> NHttpServerHandler.exception(...)
> See comments for client.
>
> Your JavaDocing is much improved!
Working on it ;-)
Cheers
Oleg
> I noticed a typo in
> NHttpConnection.request*put: "oprtations".
>
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCore] NIO extensions: event-driven non-blocking HTTP transport
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
just a short glimpse...
ContentDecoder.isCompeted should be .isCompleted
ContentEncoder.isCompeted should be .isCompleted
NHttpClientConnection.getHttpResponse()
NHttpServerConnection.getHttpRequest()
We've avoided tying the request and response directly so far.
But it's probably not a big issue, so leave it that way.
NHttpServerConnection.submitContinue()
I don't like this at all. You are assuming that 100 continue
is the only preliminary response that will ever be sent.
That is true for the current HTTP specifications, but HTTP
explicitly allows for extensions. I don't see why we should
restrict our API in this case. Alternatives:
1. NHttpServerConnection.submitPreliminary(HttpResponse)
2. NHttpServerConnection.submitResponse(HttpResponse, boolean preliminary)
NHttpClientHandler.exception(...)
Will the client handler get a chance to handle other kinds
of exceptions, too? I usually prefer a generic error handling
method, where the handler decides what to do (if anything):
/** @return true if handled, false otherwise */
boolean NHttpClientHandler.exception(conn, Throwable)
NHttpServerHandler.exception(...)
See comments for client.
Your JavaDocing is much improved! I noticed a typo in
NHttpConnection.request*put: "oprtations".
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org