You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Mike Heath <mh...@apache.org> on 2008/03/18 21:56:24 UTC
[AsyncWeb] Proposed client API in sandbox
The proposed new AsyncWeb client API is available here for your viewing
pleasure:
http://svn.apache.org/repos/asf/mina/sandbox/mheath/asyncweb/client/
I've taken into account a lot of the feedback I've received so far.
Some of the big changes include:
- I've separated the HttpClientFactory and the HttpConnector. The
HttpConnector provides HttpConnection objects which can be thought of as
a step above a MINA IoSession. The HttpClientFactory produces
HttpClient instances that provide a friendlier interface to issuing
asynchronous HTTP requests. The configuration options have been
separated as well
- The HttpFuture object has get/setPayload methods that allow the user
to arbitrarily associate an object with the future (this is an idea
borrow from the JSR 203 IoFuture).
- I added an HttpRequestFuture interface that extends HttpFuture and
returns the HttpRequest object used to initiate the request
- I added support for sending HttpRequest objects directly
- I added PartialResponseListener for dealing with large responses.
This has the same name as an interface proposed by Julien for the server
API so we may need to change the name and/or discover overlap with the
server side of the framework
- I added some configuration options for HTTP proxy settings
Some of the things that we need to work on still
- We still need to add something analogous to the IoServiceListener in
MINA or the Event mechanism in AHC
- We need some way for configuring different encoding mechanisms (i.e.
gzip encoding)
- Caching - good caching and support for ETag would be a big plus
I'm thinking that a HttpClientFactory would use a HttpConnector under
the hood. This would allow us to implement HttpClientFactory and
HttpClient once and be able to plug in any transport that implements
HttpConnector. We can also implement connection pooling by creating a
HttpConnector implementation that proxies another HttpConnector that
does the actual network communication.
It's still a work in progress so please tell me what you don't like and
what could be improved.
-Mike
Re: [AsyncWeb] Proposed client API in sandbox
Posted by "Alan D. Cabrera" <li...@toolazydogs.com>.
A bit more detail..
On Mar 18, 2008, at 2:16 PM, Alan D. Cabrera wrote:
> This is great!
>
> If I might make a suggestion. Can we have a hierarchy like:
>
> client/core
> client/examples
The examples POM would be a simple "empty" POM so that we could
client/examples/largeFile
client/examples/scatterGather
Regards,
Alan
Re: [AsyncWeb] Proposed client API in sandbox
Posted by "Alan D. Cabrera" <li...@toolazydogs.com>.
This is great!
If I might make a suggestion. Can we have a hierarchy like:
client/core
client/examples
So that people can submit patches to add their examples which
illustrate their use cases. Then people could also submit changes to
the API, which might be needed to support their use case, for us to
all discuss.
Regards,
Alan
On Mar 18, 2008, at 1:56 PM, Mike Heath wrote:
> The proposed new AsyncWeb client API is available here for your
> viewing
> pleasure:
> http://svn.apache.org/repos/asf/mina/sandbox/mheath/asyncweb/client/
>
> I've taken into account a lot of the feedback I've received so far.
> Some of the big changes include:
>
> - I've separated the HttpClientFactory and the HttpConnector. The
> HttpConnector provides HttpConnection objects which can be thought
> of as
> a step above a MINA IoSession. The HttpClientFactory produces
> HttpClient instances that provide a friendlier interface to issuing
> asynchronous HTTP requests. The configuration options have been
> separated as well
> - The HttpFuture object has get/setPayload methods that allow the user
> to arbitrarily associate an object with the future (this is an idea
> borrow from the JSR 203 IoFuture).
> - I added an HttpRequestFuture interface that extends HttpFuture and
> returns the HttpRequest object used to initiate the request
> - I added support for sending HttpRequest objects directly
> - I added PartialResponseListener for dealing with large responses.
> This has the same name as an interface proposed by Julien for the
> server
> API so we may need to change the name and/or discover overlap with the
> server side of the framework
> - I added some configuration options for HTTP proxy settings
>
> Some of the things that we need to work on still
> - We still need to add something analogous to the IoServiceListener in
> MINA or the Event mechanism in AHC
> - We need some way for configuring different encoding mechanisms (i.e.
> gzip encoding)
> - Caching - good caching and support for ETag would be a big plus
>
> I'm thinking that a HttpClientFactory would use a HttpConnector under
> the hood. This would allow us to implement HttpClientFactory and
> HttpClient once and be able to plug in any transport that implements
> HttpConnector. We can also implement connection pooling by creating a
> HttpConnector implementation that proxies another HttpConnector that
> does the actual network communication.
>
> It's still a work in progress so please tell me what you don't like
> and
> what could be improved.
>
> -Mike
>
Re: [AsyncWeb] Proposed client API in sandbox
Posted by "Alan D. Cabrera" <li...@toolazydogs.com>.
On Apr 4, 2008, at 4:36 PM, Sangjin Lee wrote:
> On Fri, Apr 4, 2008 at 9:36 AM, Alan D. Cabrera <li...@toolazydogs.com>
> wrote:
>>
>> - Create an abstract implementation of HttpListener that has callback
>>> methods such as "onSuccess, onCancel, onError, onTimeout".
>>> - Add methods as you suggested (isSuccessful, isTimedOut, etc.)
>>>
>>> Perhaps we should support both approaches.
>>>
>>
>> I think that supporting both approaches would lead to a confusing and
>> cluttered API. I like the idea of Future.get() and, if you want more
>> detail, you listen for the instrumentation events.
>
>
> The way mina handles futures is actually pretty similar to what Mike
> suggested: ConnectFuture<http://mina.apache.org/report/trunk/apidocs/org/apache/mina/common/ConnectFuture.html
> >.
> For example, ConnectFuture provides getSession() which will throw an
> exception if the connect failed. It also provides isConnected().
> Just
> because mina does it this way doesn't mean we'd want to do it the
> same way,
> but it is consistent.
My personal preference is to follow what ever paradigm that Mina set
forth if all things are roughly equal.
>> - Monitoring support based on key events: might be useful for
>>>> instrumentation and/or monitoring
>>>>
>>>
>>> Yes, this needs to be added.
>>>
>>
>> I looked at this. About the connect event. Would it make sense to
>> have
>> two events
>>
>> - Connect
>> - SSL Negotiation Completed
>>
>> I think that I would also like a response started event.
>
>
> Those would be great.
>
>
>>
>>
>> If this makes sense to you all, I'll add them to the wiki.
>
>
> This might be a newbie question, but how can I get edit access to
> the wiki
> pages?
Just sign up. If you have permissions problems then just ping me and
I'll fix you up.
Regards,
Alan
Re: [AsyncWeb] Proposed client API in sandbox
Posted by Sangjin Lee <sj...@gmail.com>.
On Fri, Apr 4, 2008 at 9:36 AM, Alan D. Cabrera <li...@toolazydogs.com>
wrote:
>
> - Create an abstract implementation of HttpListener that has callback
> > methods such as "onSuccess, onCancel, onError, onTimeout".
> > - Add methods as you suggested (isSuccessful, isTimedOut, etc.)
> >
> > Perhaps we should support both approaches.
> >
>
> I think that supporting both approaches would lead to a confusing and
> cluttered API. I like the idea of Future.get() and, if you want more
> detail, you listen for the instrumentation events.
The way mina handles futures is actually pretty similar to what Mike
suggested: ConnectFuture<http://mina.apache.org/report/trunk/apidocs/org/apache/mina/common/ConnectFuture.html>.
For example, ConnectFuture provides getSession() which will throw an
exception if the connect failed. It also provides isConnected(). Just
because mina does it this way doesn't mean we'd want to do it the same way,
but it is consistent.
> - Monitoring support based on key events: might be useful for
> > > instrumentation and/or monitoring
> > >
> >
> > Yes, this needs to be added.
> >
>
> I looked at this. About the connect event. Would it make sense to have
> two events
>
> - Connect
> - SSL Negotiation Completed
>
> I think that I would also like a response started event.
Those would be great.
>
>
> If this makes sense to you all, I'll add them to the wiki.
This might be a newbie question, but how can I get edit access to the wiki
pages?
Thanks,
Sangjin
Re: [AsyncWeb] Proposed client API in sandbox
Posted by "Alan D. Cabrera" <li...@toolazydogs.com>.
Agreed. This is really nice.
On Mar 19, 2008, at 11:39 AM, Mike Heath wrote:
> Sangjin Lee wrote:
>> It looks great! I went through the APIs, and here are my early
>> thoughts:
>>
>> [HttpConnectorConfig]
>> - It would be good to support configuring socket parameters
>> (tcpNoDelay,
>> soLinger, bufsize, ...).
>
> IMO these parameters should be configured through the underlying
> transport implementation. Because we're passing around buffers, it
> doesn't make a lot of sense to turn off tcpNoDelay and soLinger is
> something that should be managed by the transport. With MINA 2.0 (and
> perhaps 1.x, I don't remember) the buffer size is dynamic so I don't
> think we would want to expose this in the AsyncWeb API.
Agreed.
>> [HttpListener]
>> - onComplete() would be invoked not only if there is a response
>> ready but
>> also if there was an exception or a timeout on the response,
>> correct? How
>> would one determine one case against another? Perhaps it might be
>> a good
>> idea to provide a few convenience methods that let you determine
>> exactly
>> which case (not unlike mina's IoFuture implementations). Some
>> suggestions
>> include
>>
>> boolean isSuccessful(): terrible name for lack of a better
>> alternative
>> boolean hasException()
>> boolean isTimedOut()
>> Throwable getException()
>>
>> We could simply say that one should call Future.get() and react
>> according to
>> the outcome (whether it returns a result as opposed to throwing an
>> executor
>> exception) but the above seems more direct and more convenient.
>
> In ADBCJ, you have to invoke Future.get() to determine if the request
> succeeded or not. In same cases it works well, in others it doesn't.
In what cases does it not work well?
> I see two ways of solving the problem:
>
> - Create an abstract implementation of HttpListener that has callback
> methods such as "onSuccess, onCancel, onError, onTimeout".
> - Add methods as you suggested (isSuccessful, isTimedOut, etc.)
>
> Perhaps we should support both approaches.
I think that supporting both approaches would lead to a confusing and
cluttered API. I like the idea of Future.get() and, if you want more
detail, you listen for the instrumentation events.
>> [CookieManager]
>> - Would it also encapsulate the logic of accepting and validating
>> cookies
>> coming from the server (i.e. handling the "Set-Cookie" header)? For
>> example, validation on the domain attribute needs to be done
>> against the
>> request host name. Also, default values for the domain and the path
>> attributes need to be determined per spec before storing the cookies.
>
> I think the client should be responsible for validating the cookies in
> the HttpResponse so that neither the CM nor the code that invokes the
> AsyncWeb client have to worry about getting invalid cookies.
Agreed.
> We could have an event that is triggered when we receive an invalid
> cookie.
>
>> [Some things that exist in the old client but not here]
>> - Authentication support: we could migrate them with minimal
>> modifications?
>
> We could put the username and password in the URI (i.e.
> http://USER_NAME:SECRET@somehost.foo/) for simple cases. I would like
> to add support for some type of AuthenticationManager that could be
> used
> for managing authentication credentials. The AuthenticationManager
> could also be used to provide a callback for when a request fails
> due to
> not authenticating (so, in the case of a GUI, for example, a dialog
> box
> could be displayed prompting the user to enter a username and
> password).
I like that idea.
>> - Monitoring support based on key events: might be useful for
>> instrumentation and/or monitoring
>
> Yes, this needs to be added.
I looked at this. About the connect event. Would it make sense to
have two events
- Connect
- SSL Negotiation Completed
I think that I would also like a response started event.
If this makes sense to you all, I'll add them to the wiki.
About getHttpClientFactory(). Will all HttpClients be created only by
HttpClientFactories? If not maybe we should have an
HttpClientFactoryGenerated interface that has that method.
Maybe we should have a root interface HttpClientConfigured which
provides an R/O configuration and HttpClientConfigurable which
provides an R/W configuration. We you can see, I favor simple
disconnected interfaces that implementations would mix and match.
Should HttpClients inherit from HttpClientConfigurable? Maybe it
might be a good idea to disconnect and have implementations implement
both HttpClient and HttpClientConfigurable for those implementations
where clients are not configurable.
Looking for symmetries between HttpClient and HttpConnector. If we
can get an HttpClient's factory, should we not also be able to get
HttpConnector's factory as well? HttpClients have
HttpClientConfigurable, should HttpConnectors not also have one as well?
Again, really nice.
Regards,
Alan
Re: [AsyncWeb] Proposed client API in sandbox
Posted by Mike Heath <mh...@apache.org>.
Sangjin Lee wrote:
> It looks great! I went through the APIs, and here are my early thoughts:
> [HttpClient]
> - Would a common usage be one HttpClient instance per host? It's not very
> clear to me whether one instance can be used against multiple hosts or it
> can be used only for a given host. Specifically, for the following method
>
> List<HttpRequestFuture> get(String... uris)
>
> must all uris be against the same host, or can they go to various hosts?
> The javadoc "pipelining" suggests they must go all against the same host...
A single HttpClient instance may be used to connect to any number of
hosts. With the List<HttpRequestFuture> get(String... uris), all the
hosts and ports would have to be the same to pipeline the requests.
> [HttpConnectorConfig]
> - It would be good to support configuring socket parameters (tcpNoDelay,
> soLinger, bufsize, ...).
IMO these parameters should be configured through the underlying
transport implementation. Because we're passing around buffers, it
doesn't make a lot of sense to turn off tcpNoDelay and soLinger is
something that should be managed by the transport. With MINA 2.0 (and
perhaps 1.x, I don't remember) the buffer size is dynamic so I don't
think we would want to expose this in the AsyncWeb API.
> - One thing that we may want to see is supporting optional connection
> retries.
Agreed.
> [HttpFuture]
> - Should we support Future.cancel()? Whether we can truly cancel these
> asynchronous operations here is bit tricky...
If the operation gets queued up (because of maxConnections being
exceeded, for example), it's trivial to cancel the request. Once the
request gets sent, it really can't be cancelled in which case
Future.cancel(boolean) would have to return false.
But I agree, adding support for cancel is tricky. In ADBCJ I have
support for cancel and it has been a real pain. In most cases I would
be fine with just return false and not even trying to cancel the operation.
One of the things I've been considering for ADBCJ is that if
Future.cancel(true) is invoked, it closes the socket connection. This
is similar to JSR 203 where calling IoFuture.cancel(true) on an
asynchronous file request may close the file (this may be the same for
asynchronous sockets too but I don't remember exactly.) We could use
this same approach in AsyncWeb client.
> [HttpListener]
> - onComplete() would be invoked not only if there is a response ready but
> also if there was an exception or a timeout on the response, correct? How
> would one determine one case against another? Perhaps it might be a good
> idea to provide a few convenience methods that let you determine exactly
> which case (not unlike mina's IoFuture implementations). Some suggestions
> include
>
> boolean isSuccessful(): terrible name for lack of a better alternative
> boolean hasException()
> boolean isTimedOut()
> Throwable getException()
>
> We could simply say that one should call Future.get() and react according to
> the outcome (whether it returns a result as opposed to throwing an executor
> exception) but the above seems more direct and more convenient.
In ADBCJ, you have to invoke Future.get() to determine if the request
succeeded or not. In same cases it works well, in others it doesn't.
I see two ways of solving the problem:
- Create an abstract implementation of HttpListener that has callback
methods such as "onSuccess, onCancel, onError, onTimeout".
- Add methods as you suggested (isSuccessful, isTimedOut, etc.)
Perhaps we should support both approaches.
> [CookieManager]
> - Would it also encapsulate the logic of accepting and validating cookies
> coming from the server (i.e. handling the "Set-Cookie" header)? For
> example, validation on the domain attribute needs to be done against the
> request host name. Also, default values for the domain and the path
> attributes need to be determined per spec before storing the cookies.
I think the client should be responsible for validating the cookies in
the HttpResponse so that neither the CM nor the code that invokes the
AsyncWeb client have to worry about getting invalid cookies.
We could have an event that is triggered when we receive an invalid cookie.
> [Some things that exist in the old client but not here]
> - Authentication support: we could migrate them with minimal modifications?
We could put the username and password in the URI (i.e.
http://USER_NAME:SECRET@somehost.foo/) for simple cases. I would like
to add support for some type of AuthenticationManager that could be used
for managing authentication credentials. The AuthenticationManager
could also be used to provide a callback for when a request fails due to
not authenticating (so, in the case of a GUI, for example, a dialog box
could be displayed prompting the user to enter a username and password).
> - Monitoring support based on key events: might be useful for
> instrumentation and/or monitoring
Yes, this needs to be added.
> - Being able to insert an executor filter for isolating the codec and
> post-operations
> - Proxy configuration: support for exclusion lists
Both good ideas.
>
> Thanks,
> Sangjin
Excellent suggestions. Thanks for the feedback. I've create a wiki
page to track these ideas:
http://cwiki.apache.org/confluence/display/AWEB/Client+Ideas
-Mike
> On Tue, Mar 18, 2008 at 1:56 PM, Mike Heath <mh...@apache.org> wrote:
>
>> The proposed new AsyncWeb client API is available here for your viewing
>> pleasure:
>> http://svn.apache.org/repos/asf/mina/sandbox/mheath/asyncweb/client/
>>
>> I've taken into account a lot of the feedback I've received so far.
>> Some of the big changes include:
>>
>> - I've separated the HttpClientFactory and the HttpConnector. The
>> HttpConnector provides HttpConnection objects which can be thought of as
>> a step above a MINA IoSession. The HttpClientFactory produces
>> HttpClient instances that provide a friendlier interface to issuing
>> asynchronous HTTP requests. The configuration options have been
>> separated as well
>> - The HttpFuture object has get/setPayload methods that allow the user
>> to arbitrarily associate an object with the future (this is an idea
>> borrow from the JSR 203 IoFuture).
>> - I added an HttpRequestFuture interface that extends HttpFuture and
>> returns the HttpRequest object used to initiate the request
>> - I added support for sending HttpRequest objects directly
>> - I added PartialResponseListener for dealing with large responses.
>> This has the same name as an interface proposed by Julien for the server
>> API so we may need to change the name and/or discover overlap with the
>> server side of the framework
>> - I added some configuration options for HTTP proxy settings
>>
>> Some of the things that we need to work on still
>> - We still need to add something analogous to the IoServiceListener in
>> MINA or the Event mechanism in AHC
>> - We need some way for configuring different encoding mechanisms (i.e.
>> gzip encoding)
>> - Caching - good caching and support for ETag would be a big plus
>>
>> I'm thinking that a HttpClientFactory would use a HttpConnector under
>> the hood. This would allow us to implement HttpClientFactory and
>> HttpClient once and be able to plug in any transport that implements
>> HttpConnector. We can also implement connection pooling by creating a
>> HttpConnector implementation that proxies another HttpConnector that
>> does the actual network communication.
>>
>> It's still a work in progress so please tell me what you don't like and
>> what could be improved.
>>
>> -Mike
>>
>
Re: [AsyncWeb] Proposed client API in sandbox
Posted by Sangjin Lee <sj...@gmail.com>.
It looks great! I went through the APIs, and here are my early thoughts:
[HttpClient]
- Would a common usage be one HttpClient instance per host? It's not very
clear to me whether one instance can be used against multiple hosts or it
can be used only for a given host. Specifically, for the following method
List<HttpRequestFuture> get(String... uris)
must all uris be against the same host, or can they go to various hosts?
The javadoc "pipelining" suggests they must go all against the same host...
[HttpConnectorConfig]
- It would be good to support configuring socket parameters (tcpNoDelay,
soLinger, bufsize, ...).
- One thing that we may want to see is supporting optional connection
retries.
[HttpFuture]
- Should we support Future.cancel()? Whether we can truly cancel these
asynchronous operations here is bit tricky...
[HttpListener]
- onComplete() would be invoked not only if there is a response ready but
also if there was an exception or a timeout on the response, correct? How
would one determine one case against another? Perhaps it might be a good
idea to provide a few convenience methods that let you determine exactly
which case (not unlike mina's IoFuture implementations). Some suggestions
include
boolean isSuccessful(): terrible name for lack of a better alternative
boolean hasException()
boolean isTimedOut()
Throwable getException()
We could simply say that one should call Future.get() and react according to
the outcome (whether it returns a result as opposed to throwing an executor
exception) but the above seems more direct and more convenient.
[CookieManager]
- Would it also encapsulate the logic of accepting and validating cookies
coming from the server (i.e. handling the "Set-Cookie" header)? For
example, validation on the domain attribute needs to be done against the
request host name. Also, default values for the domain and the path
attributes need to be determined per spec before storing the cookies.
[Some things that exist in the old client but not here]
- Authentication support: we could migrate them with minimal modifications?
- Monitoring support based on key events: might be useful for
instrumentation and/or monitoring
- Being able to insert an executor filter for isolating the codec and
post-operations
- Proxy configuration: support for exclusion lists
Thanks,
Sangjin
On Tue, Mar 18, 2008 at 1:56 PM, Mike Heath <mh...@apache.org> wrote:
> The proposed new AsyncWeb client API is available here for your viewing
> pleasure:
> http://svn.apache.org/repos/asf/mina/sandbox/mheath/asyncweb/client/
>
> I've taken into account a lot of the feedback I've received so far.
> Some of the big changes include:
>
> - I've separated the HttpClientFactory and the HttpConnector. The
> HttpConnector provides HttpConnection objects which can be thought of as
> a step above a MINA IoSession. The HttpClientFactory produces
> HttpClient instances that provide a friendlier interface to issuing
> asynchronous HTTP requests. The configuration options have been
> separated as well
> - The HttpFuture object has get/setPayload methods that allow the user
> to arbitrarily associate an object with the future (this is an idea
> borrow from the JSR 203 IoFuture).
> - I added an HttpRequestFuture interface that extends HttpFuture and
> returns the HttpRequest object used to initiate the request
> - I added support for sending HttpRequest objects directly
> - I added PartialResponseListener for dealing with large responses.
> This has the same name as an interface proposed by Julien for the server
> API so we may need to change the name and/or discover overlap with the
> server side of the framework
> - I added some configuration options for HTTP proxy settings
>
> Some of the things that we need to work on still
> - We still need to add something analogous to the IoServiceListener in
> MINA or the Event mechanism in AHC
> - We need some way for configuring different encoding mechanisms (i.e.
> gzip encoding)
> - Caching - good caching and support for ETag would be a big plus
>
> I'm thinking that a HttpClientFactory would use a HttpConnector under
> the hood. This would allow us to implement HttpClientFactory and
> HttpClient once and be able to plug in any transport that implements
> HttpConnector. We can also implement connection pooling by creating a
> HttpConnector implementation that proxies another HttpConnector that
> does the actual network communication.
>
> It's still a work in progress so please tell me what you don't like and
> what could be improved.
>
> -Mike
>