You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Mark Thornton <mt...@optrak.com> on 2012/02/28 11:50:05 UTC
HttpCore 4.1.4 and pipelined client
I am trying to implement a pipelined client using 4.1.4 (Async client
and core 4.2 aren't yet GA). It would appear that
AsyncNHttpClientHandler does not permit pipelining. Is it sufficient to
replace this class or are there other problems lurking elsewhere?
Note that the client code is talking to my own server not general hosts.
The server performs somewhat time consuming calculations which are a
good fit with pipelined operation (though it would be nice to return
results out of order).
Mark Thornton
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2012-02-29 at 18:34 +0000, Mark Thornton wrote:
> On 29/02/12 10:16, Oleg Kalnichevski wrote:
> > By the way, is anyone interested in seeing my handlers with (in order)
> > pipeline support?
> >
> > Mark
> >
> > You are also welcome to contribute such implementation to the project,
> > but ideally in this case it should be consistent with the 4.2 API
> >
> > Cheers
> >
> > Oleg
>
> One difficulty with the api is that I don't think a non idempotent
> request (e.g. POST) should be sent while there any outstanding responses
> in the pipeline. Doing so would make it difficult to identify the cause
> of a failure. A failure in one of the pipelined methods ahead of the
> post would also result in failure of the POST. However the callback
> methods (HttpAsyncRequestProducer.generateRequest,
> NHttpRequestExecutionHandler.submitRequest) in both 4.1 and 4.2 api
> don't provide any information on the state of any pipeline. So
> controlling submission of non idempotent methods has to be done on top
> which isn't ideal.
>
I am not sure individual message producers / consumers need to be aware
of pipelining aspects, but please do not feel constrained by the
existing API. Get your code working with the API you feel is the best.
Should you decide to contribute your code to the project, we'll see how
your code can be integrated into the 4.2 code line.
Oleg
> I now have basic pipelining working, but still need to attend to various
> error conditions.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by Mark Thornton <mt...@optrak.com>.
On 29/02/12 10:16, Oleg Kalnichevski wrote:
> By the way, is anyone interested in seeing my handlers with (in order)
> pipeline support?
>
> Mark
>
> You are also welcome to contribute such implementation to the project,
> but ideally in this case it should be consistent with the 4.2 API
>
> Cheers
>
> Oleg
One difficulty with the api is that I don't think a non idempotent
request (e.g. POST) should be sent while there any outstanding responses
in the pipeline. Doing so would make it difficult to identify the cause
of a failure. A failure in one of the pipelined methods ahead of the
post would also result in failure of the POST. However the callback
methods (HttpAsyncRequestProducer.generateRequest,
NHttpRequestExecutionHandler.submitRequest) in both 4.1 and 4.2 api
don't provide any information on the state of any pipeline. So
controlling submission of non idempotent methods has to be done on top
which isn't ideal.
I now have basic pipelining working, but still need to attend to various
error conditions.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2012-02-28 at 21:48 +0000, Mark Thornton wrote:
> On 28/02/12 21:07, William Speirs wrote:
> > Mark-
> >
> > Have you looked at SPDY? It handles the "multiplexing" that you're looking
> > for with respect to out-of-order requests.
> >
> > I started to try and implement SPDY inside of HttpComponents, but didn't
> > get super far (that probably lacks my latest changes):
> > https://github.com/wspeirs/SPDYComponents
> >
> > If you'd be interested in working on such a thing, let me know.
> >
> > Bill-
> I haven't looked at it yet. I am hopeful of getting the pipelining
> working sometime tomorrow. Doing a replacement for
> AsyncNHttpServiceHandler at the moment. In this case it seems straight
> forward to support both pipelining and CONTINUE, at least with my
> interpretation of the rfc. That should suffice to get my code working.
>
> Our reason for using http in the first place was the ease of using a
> browser for testing. While it doesn't test the pipelining it does make
> it easy to very the basic results. I will have a look at SPDY once my
> immediate task is complete.
>
> By the way, is anyone interested in seeing my handlers with (in order)
> pipeline support?
>
Mark
You are also welcome to contribute such implementation to the project,
but ideally in this case it should be consistent with the 4.2 API
Cheers
Oleg
> Thanks for your interest
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by kim young ill <kh...@googlemail.com>.
+
i'm using httpclient (io) but would like to give nio a chance.
would be happy to learn from your work
On Tue, Feb 28, 2012 at 10:48 PM, Mark Thornton <mt...@optrak.com>wrote:
> On 28/02/12 21:07, William Speirs wrote:
>
>> Mark-
>>
>> Have you looked at SPDY? It handles the "multiplexing" that you're looking
>> for with respect to out-of-order requests.
>>
>> I started to try and implement SPDY inside of HttpComponents, but didn't
>> get super far (that probably lacks my latest changes):
>> https://github.com/wspeirs/**SPDYComponents<https://github.com/wspeirs/SPDYComponents>
>>
>> If you'd be interested in working on such a thing, let me know.
>>
>> Bill-
>>
> I haven't looked at it yet. I am hopeful of getting the pipelining working
> sometime tomorrow. Doing a replacement for AsyncNHttpServiceHandler at the
> moment. In this case it seems straight forward to support both pipelining
> and CONTINUE, at least with my interpretation of the rfc. That should
> suffice to get my code working.
>
> Our reason for using http in the first place was the ease of using a
> browser for testing. While it doesn't test the pipelining it does make it
> easy to very the basic results. I will have a look at SPDY once my
> immediate task is complete.
>
> By the way, is anyone interested in seeing my handlers with (in order)
> pipeline support?
>
> Thanks for your interest
>
> Mark
>
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: HttpCore 4.1.4 and pipelined client
Posted by Mark Thornton <mt...@optrak.com>.
On 28/02/12 21:07, William Speirs wrote:
> Mark-
>
> Have you looked at SPDY? It handles the "multiplexing" that you're looking
> for with respect to out-of-order requests.
>
> I started to try and implement SPDY inside of HttpComponents, but didn't
> get super far (that probably lacks my latest changes):
> https://github.com/wspeirs/SPDYComponents
>
> If you'd be interested in working on such a thing, let me know.
>
> Bill-
I haven't looked at it yet. I am hopeful of getting the pipelining
working sometime tomorrow. Doing a replacement for
AsyncNHttpServiceHandler at the moment. In this case it seems straight
forward to support both pipelining and CONTINUE, at least with my
interpretation of the rfc. That should suffice to get my code working.
Our reason for using http in the first place was the ease of using a
browser for testing. While it doesn't test the pipelining it does make
it easy to very the basic results. I will have a look at SPDY once my
immediate task is complete.
By the way, is anyone interested in seeing my handlers with (in order)
pipeline support?
Thanks for your interest
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by William Speirs <ws...@apache.org>.
Mark-
Have you looked at SPDY? It handles the "multiplexing" that you're looking
for with respect to out-of-order requests.
I started to try and implement SPDY inside of HttpComponents, but didn't
get super far (that probably lacks my latest changes):
https://github.com/wspeirs/SPDYComponents
If you'd be interested in working on such a thing, let me know.
Bill-
On Tue, Feb 28, 2012 at 9:47 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Tue, 2012-02-28 at 13:45 +0000, Mark Thornton wrote:
> > On 28/02/12 12:31, Oleg Kalnichevski wrote:
> > > On Tue, 2012-02-28 at 10:50 +0000, Mark Thornton wrote:
> > >> I am trying to implement a pipelined client using 4.1.4 (Async client
> > >> and core 4.2 aren't yet GA). It would appear that
> > >> AsyncNHttpClientHandler does not permit pipelining.
> > > No, it does not. The expect-continue handshake support and request
> > > pipelining are pretty much mutually exclusive. Default protocol
> handlers
> > > shipped with HttpCore implement expect-continue handshaking as it is
> > > required by the HTTP/1.1 specification, while request pipelining is
> > > optional.
> >
> > Not quite exclusive --- the expect handshake will force a pipeline flush
> > but pipelining can be used when there is no expect-continue outstanding.
> > Obviously it requires more work and additional synchronization between
> > send and receive parts. Will this change in 4.2?
> >
>
> I am not sure it is that simple. Sending a response to a pipelined
> request while the client is expecting a 100-continue _or_ a 4xx status
> code in case of a failed expectation would directly violate the
> specification in my opinion. Personally I do not see a non-ambiguous way
> of supporting both the expect-continue and request pipelining at the
> same time, but I'd be enormously happy to see someone prove me wrong by
> submitting such a protocol handler ;-)
>
> > > Hope this helps Oleg
> >
> > Thanks very much, it does give me confidence to proceed. I am slightly
> > tempted to extend the protocol to permit out of order responses.
> >
>
> I would advise against bastardizing the HTTP protocol. You probably
> should consider using WebSockets or Google SPDY if you want to support
> fully asynchronous message exchanges.
>
> Hope this helps
>
> Oleg
>
> > regards,
> > Mark Thornton
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: HttpCore 4.1.4 and pipelined client
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2012-02-28 at 13:45 +0000, Mark Thornton wrote:
> On 28/02/12 12:31, Oleg Kalnichevski wrote:
> > On Tue, 2012-02-28 at 10:50 +0000, Mark Thornton wrote:
> >> I am trying to implement a pipelined client using 4.1.4 (Async client
> >> and core 4.2 aren't yet GA). It would appear that
> >> AsyncNHttpClientHandler does not permit pipelining.
> > No, it does not. The expect-continue handshake support and request
> > pipelining are pretty much mutually exclusive. Default protocol handlers
> > shipped with HttpCore implement expect-continue handshaking as it is
> > required by the HTTP/1.1 specification, while request pipelining is
> > optional.
>
> Not quite exclusive --- the expect handshake will force a pipeline flush
> but pipelining can be used when there is no expect-continue outstanding.
> Obviously it requires more work and additional synchronization between
> send and receive parts. Will this change in 4.2?
>
I am not sure it is that simple. Sending a response to a pipelined
request while the client is expecting a 100-continue _or_ a 4xx status
code in case of a failed expectation would directly violate the
specification in my opinion. Personally I do not see a non-ambiguous way
of supporting both the expect-continue and request pipelining at the
same time, but I'd be enormously happy to see someone prove me wrong by
submitting such a protocol handler ;-)
> > Hope this helps Oleg
>
> Thanks very much, it does give me confidence to proceed. I am slightly
> tempted to extend the protocol to permit out of order responses.
>
I would advise against bastardizing the HTTP protocol. You probably
should consider using WebSockets or Google SPDY if you want to support
fully asynchronous message exchanges.
Hope this helps
Oleg
> regards,
> Mark Thornton
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by Mark Thornton <mt...@optrak.com>.
On 28/02/12 12:31, Oleg Kalnichevski wrote:
> On Tue, 2012-02-28 at 10:50 +0000, Mark Thornton wrote:
>> I am trying to implement a pipelined client using 4.1.4 (Async client
>> and core 4.2 aren't yet GA). It would appear that
>> AsyncNHttpClientHandler does not permit pipelining.
> No, it does not. The expect-continue handshake support and request
> pipelining are pretty much mutually exclusive. Default protocol handlers
> shipped with HttpCore implement expect-continue handshaking as it is
> required by the HTTP/1.1 specification, while request pipelining is
> optional.
Not quite exclusive --- the expect handshake will force a pipeline flush
but pipelining can be used when there is no expect-continue outstanding.
Obviously it requires more work and additional synchronization between
send and receive parts. Will this change in 4.2?
> Hope this helps Oleg
Thanks very much, it does give me confidence to proceed. I am slightly
tempted to extend the protocol to permit out of order responses.
regards,
Mark Thornton
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: HttpCore 4.1.4 and pipelined client
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2012-02-28 at 10:50 +0000, Mark Thornton wrote:
> I am trying to implement a pipelined client using 4.1.4 (Async client
> and core 4.2 aren't yet GA). It would appear that
> AsyncNHttpClientHandler does not permit pipelining.
No, it does not. The expect-continue handshake support and request
pipelining are pretty much mutually exclusive. Default protocol handlers
shipped with HttpCore implement expect-continue handshaking as it is
required by the HTTP/1.1 specification, while request pipelining is
optional.
> Is it sufficient to
> replace this class or are there other problems lurking elsewhere?
>
One can always to choose to implement a custom protocol handler instead
of using the default one. It should be perfectly ok to replace
AsyncNHttpClientHandler.
Hope this helps
Oleg
> Note that the client code is talking to my own server not general hosts.
> The server performs somewhat time consuming calculations which are a
> good fit with pipelined operation (though it would be nice to return
> results out of order).
>
> Mark Thornton
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org