You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Roland Weber <ht...@dubioso.net> on 2006/09/16 20:54:20 UTC
HttpCore NIO questions
Hi Oleg,
I just did a little code dive to verify my ideas about what
NIO is doing. As was to be expected, there is a mismatch.
Maybe you can confirm or correct my understanding of a few
points:
AbstractIOReactor has code that deals with a server socket
channel. So it is exclusively server-side? If that is the case,
I would suggest a name change to AbstractServerIOReactor.
I was a bit puzzled that IOSession has a channel() method
that returns a ByteChannel, but I just realized that
java.nio.channels.SocketChannel does implement that interface.
So IOSession can be used for non-blocking connects on the
client side.
I found it impossible to understand the roles and relations
between the interfaces defined in o.a.h.nio without looking
not only into the implementation classes, but into the examples.
They seem a little disjointed at the interface level. I've gone
through Doug Lea's presentation that I linked as PDF in the
package JavaDocs, but I didn't find the IOSession mentioned
there.
The chances that I will come up with a patch for HTTPCORE-8
this weekend have dropped to virtually 0.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
> I believe strongly when it comes to libraries sometimes less is more. I
> very much rather prefer a library that does one thing and does it well
> to one that tries to be all sorts of things to all sorts of people.
I agree. But I wonder how much connection opening is needed for HttpCore
in order for it to work well.
> We
> cannot deliver a complete SSL tunneling solution in HttpCore without
> authentication, so probably we should not even try.
Yes, I started to think along those lines when I wrote the last mail.
>>Opening a plain socket to an HTTP host or proxy is easily enough achieved
>>using standard Java functionality. But that means that applications will
>>have responsibilities on the socket level, while the API works on the
>>HttpHost/ProxyHost level, and we'll have no way to ensure consistency.
>
> They do not necessarily have to be consistent as HttpHost may well
> represent a virtual host, which does not directly correspond to the
> underlying physical socket. I do not see this as an issue.
Maybe. Maybe HttpCore doesn't even have to know whether it is connected
to a proxy or directly to a server. I need to think about this.
>>Maybe we can come up with a small module-conn for opening connections,
>>while HttpConn deals with real connection management, that is pools of
>>connections?
>
> Then we rather keep this code in HttpCore base module, if you feel
> strongly about it.
I don't feel strongly about it at this time.
> Let us step back for now and think things over.
I definitely agree with that :-)
> Maybe others will throw
> in their ideas and opinions as well.
That would be so welcome. I'm a bit tired of our stalemates with
nobody joining the discussion to break the tie. Considering how often
that happens, I am surprised about how much progress we're making :-)
Two is a majority, as a friend of mine once observed. He was referring
to a department of maybe 20 or 25 people. How many are on this mailing
list? ;-)
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2006-09-18 at 15:12 +0200, Roland Weber wrote:
> Hi Oleg,
>
> >>We have to bear in mind that HttpCore is supposed to be useable without
> >>any other module. Moving default implementations for connections out
> >>of the core seems to clash with this goal.
> >
> > The default implementations should stay in HttpCore, but they should be
> > (at least in my opinion) only implement a simple method allowing the
> > connection to be bound to an open socket. Code aspects that define how
> > exactly that socket gets initialized (directly or via a proxy) belongs
> > to HttpConn in my opinion. This is just a proposal. If you object to
> > such a change it will not happen.
>
> I don't feel well telling our users that HttpCore is useable by itself,
> but not giving them the infrastructure to open an SSL connection over
> a proxy (without authentication). Let me ponder this for a while.
>
Hi Roland,
I believe strongly when it comes to libraries sometimes less is more. I
very much rather prefer a library that does one thing and does it well
to one that tries to be all sorts of things to all sorts of people. We
cannot deliver a complete SSL tunneling solution in HttpCore without
authentication, so probably we should not even try.
> Opening a plain socket to an HTTP host or proxy is easily enough achieved
> using standard Java functionality. But that means that applications will
> have responsibilities on the socket level, while the API works on the
> HttpHost/ProxyHost level, and we'll have no way to ensure consistency.
They do not necessarily have to be consistent as HttpHost may well
represent a virtual host, which does not directly correspond to the
underlying physical socket. I do not see this as an issue.
> Maybe we can come up with a small module-conn for opening connections,
> while HttpConn deals with real connection management, that is pools of
> connections?
>
Then we rather keep this code in HttpCore base module, if you feel
strongly about it.
Let us step back for now and think things over. Maybe others will throw
in their ideas and opinions as well.
Cheers,
Oleg
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
>>We have to bear in mind that HttpCore is supposed to be useable without
>>any other module. Moving default implementations for connections out
>>of the core seems to clash with this goal.
>
> The default implementations should stay in HttpCore, but they should be
> (at least in my opinion) only implement a simple method allowing the
> connection to be bound to an open socket. Code aspects that define how
> exactly that socket gets initialized (directly or via a proxy) belongs
> to HttpConn in my opinion. This is just a proposal. If you object to
> such a change it will not happen.
I don't feel well telling our users that HttpCore is useable by itself,
but not giving them the infrastructure to open an SSL connection over
a proxy (without authentication). Let me ponder this for a while.
Opening a plain socket to an HTTP host or proxy is easily enough achieved
using standard Java functionality. But that means that applications will
have responsibilities on the socket level, while the API works on the
HttpHost/ProxyHost level, and we'll have no way to ensure consistency.
Maybe we can come up with a small module-conn for opening connections,
while HttpConn deals with real connection management, that is pools of
connections?
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: Lessons Learned
Posted by Roland Weber <ht...@dubioso.net>.
Hello Asankha,
> Synapse would be expected to work with many messages concurrently
> (acting as an ESB) we do not want a huge number of threads to be doing
> IO - and hence the requirement for an async http client.
Thanks for the update, I will consider this around christmas time.
For the records:
Let m be the number of messages processed concurrently and c the
number of connections opened concurrently. The original idea of
HttpAsync - without NIO - was to have O(c) threads for handling I/O,
with a maximum of 2*c for maximum throughput and a minimum of c.
With NIO, we can get that number below c as we don't need one
thread per connection for receiving messages. But even without NIO,
2*c or c could be a significant improvement over O(m).
Still, there is no point in pushing HttpAsync until we've figured
out HttpCore and how to deal with opening connections.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: Lessons Learned
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2006-09-19 at 11:53 +0530, Asankha C. Perera wrote:
> Sorry I haven't been able to follow through all messages in this
> thread.. but yes, Synapse is very much interested in an async/NIO based
> http/s client. Basically this is required as an http/s transport
> implementation for Axis2 (and not directly for Synapse). However, as
> Synapse would be expected to work with many messages concurrently
> (acting as an ESB) we do not want a huge number of threads to be doing
> IO - and hence the requirement for an async http client. So far we
> haven't made any progress on this front...
>
Asankha,
Are full-blown HTTP state management and authentication hard
requirements for such a client? If not, a simple async HTTP
client can be hacked up within a reasonable period of time, provided
Apache Synapse is prepared to act as a testbed for it.
Oleg
> thanks
> asankha
> >> I believe Apache Synapse expressed interest in an asynchronous
> >> HTTP client of a sort. Probably you should get in touch with those guys,
> >> see if they are still interested and get them articulate requirements
> >> for such a client.
> >>
> >
> > I don't see when I would be able to do something with those requirements.
> > I know that I won't have time to do anything until December, and the
> > items lining up so far have already taken months not to be implemented.
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: Lessons Learned
Posted by "Asankha C. Perera" <as...@wso2.com>.
Sorry I haven't been able to follow through all messages in this
thread.. but yes, Synapse is very much interested in an async/NIO based
http/s client. Basically this is required as an http/s transport
implementation for Axis2 (and not directly for Synapse). However, as
Synapse would be expected to work with many messages concurrently
(acting as an ESB) we do not want a huge number of threads to be doing
IO - and hence the requirement for an async http client. So far we
haven't made any progress on this front...
thanks
asankha
>> I believe Apache Synapse expressed interest in an asynchronous
>> HTTP client of a sort. Probably you should get in touch with those guys,
>> see if they are still interested and get them articulate requirements
>> for such a client.
>>
>
> I don't see when I would be able to do something with those requirements.
> I know that I won't have time to do anything until December, and the
> items lining up so far have already taken months not to be implemented.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: Lessons Learned
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
> I think you are over-reacting a little.
Well, the frustration has been building up for quite a while.
HttpComponents is just one of the things which I have to schedule,
and right now there's a lot of them rushing at me. I need to make
a cut at some point in time.
> First off, HttpComponents is a
> VERY small and unimportant project compared to many of its open-source
> peers. Such is the dynamics of a small-time open-source project: it is
> dog slow and boring. We just have to live with that. No one cares unless
> they are bitten by an annoying bug that directly affects their project.
> And to make matters worse, our flagship product HttpClient 3.x is not
> very buggy.
Regarding HttpAsync, I'm not really bothered by the lack of additional
contributors. I'm bother by my unability to deliver something that is
rather clearly shaped in my mind and should not take long to implement.
If there weren't popping up higher priority items left, right and center.
> Secondly, I felt HttpAsync was bound to hit a bumpy spot at some point
> of time because it lacked a clearly defined scope and a target user
> base.
The functionality for the client side is rather clearly defined. Even
if there is room for improvement using NIO and such: there's a few
threads sending requests while a few others are receiving responses.
I also expected a rough time waiting for users, but I expected that
I could at least implement something that's useable. In time. I can't.
> I believe Apache Synapse expressed interest in an asynchronous
> HTTP client of a sort. Probably you should get in touch with those guys,
> see if they are still interested and get them articulate requirements
> for such a client.
I don't see when I would be able to do something with those requirements.
I know that I won't have time to do anything until December, and the
items lining up so far have already taken months not to be implemented.
> I want to see HttpCore 4.0 alpha3 and HttpClient 4.0 alpha1 released
> first, at which point I want to take a serious look at HttpAsync. Let us
> re-focus for a little while. That's all.
Agreed. I would expect the core alpha3 at the end of this year, and
a client alpha 1 maybe next summer?
> There is no need for burials and resurrections.
Well, then let's call it re-focussing for a little while. There is a
russian proverb saying "Call me a pot just do not put me in an oven" ;-)
When I am taking a break from a project, and I know that the break will
last for many months, and I can't even give an estimate when it will be
over, then I feel the need to express that this project is no longer
under active development from my part. Without that, I would still feel
responsible for working on it, and I would constantly feel bad about not
making progress. I feel much better after saying: Whack! *puff* :-)
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: Lessons Learned (was: HttpCore NIO questions)
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2006-09-18 at 17:36 +0200, Roland Weber wrote:
> Hi Oleg, everyone,
>
Hi Roland.
I think you are over-reacting a little. First off, HttpComponents is a
VERY small and unimportant project compared to many of its open-source
peers. Such is the dynamics of a small-time open-source project: it is
dog slow and boring. We just have to live with that. No one cares unless
they are bitten by an annoying bug that directly affects their project.
And to make matters worse, our flagship product HttpClient 3.x is not
very buggy.
Secondly, I felt HttpAsync was bound to hit a bumpy spot at some point
of time because it lacked a clearly defined scope and a target user
base. I believe Apache Synapse expressed interest in an asynchronous
HTTP client of a sort. Probably you should get in touch with those guys,
see if they are still interested and get them articulate requirements
for such a client.
>
> > I have always been trying to say that controlled code duplication in
> > HttpAsync is a lesser evil because it can shield HttpAsync from API
> > instability in HttpCore. Go ahead and roll out an HttpAsync specific
> > extension of HttpClientConnection interface and its default
> > implementation.
>
> If I have to work on connection management code anyway, I prefer to
> do it for HttpConn. Let HttpAsync/CS rest in peace until the time is
> ripe for resurrection.
>
I want to see HttpCore 4.0 alpha3 and HttpClient 4.0 alpha1 released
first, at which point I want to take a serious look at HttpAsync. Let us
re-focus for a little while. That's all. There is no need for burials
and resurrections.
Oleg
>
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Lessons Learned (was: HttpCore NIO questions)
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg, everyone,
it is a good and usually neglected practice to summarize and discuss
the "lessons learned" during a project at some time. I planned to do
that later this year, but since the opportunity presents itself, I'll
do it now. The project in question is HttpAsync, or rather the client
side thereof.
I set out to contribute an API and initial implementation of HttpAsync
last december. Since then, that has been the primary goal of my
contributions to HttpComponents. When challenged with a task such as
this at work, I would estimate an effort of about two weeks, excluding
test cases. The result would not be as good as here due to the missing
peer reviews - a lot of overdesigning and some "smelly code". Still,
within two weeks, I could deliver something that works and is a good
starting point for improvements in the next iteration. Since I'm not
developing for HttpComponents full time, let's translate the two weeks
to 10 weekends. That's a net effort of about three months.
Obviously, there were some points I did not take into account.
First, I needed a lot more time than expected to familiarize myself
with the environment. Setting up accounts, learning new tools, all that
took quite some time. I'm still not finished, but I'm at least at a
point where I am no longer having urgent issues with the environment.
Next, I had to realize that the peer review process slowed me down
considerably. It's not so much coming up with a modified patch after
some feedback, but more the waiting for the feedback. If I create a
patch on one weekend and put it up on Sunday, the review will have to
wait until the following weekend, and the next iteration of the patch
maybe until the weekend after that. Estimated effort 1 weekend, elapsed
time 3 weekends. Even worse when Oleg and I disagree and multiple
iterations are needed. One way to deal with this is to work on several
isolated items in parallel. But HttpAsync is too narrow a codebase
for that, and males are reportedly bad at multitasking anyway :-)
Then, I expected about half of my weekends to be blocked by my
lectures, while it was actually more like two thirds.
And finally, I spent considerable time working on HttpComponents
other than HttpAsync.
So where is HttpAsync now? It's a pity, but HttpAsync is still at
the same point where it was when we concluded the discussion about
the threading design. When I entered a new coding phase several
weeks back, I wanted to deal with HTTPASYNC-1 and HTTPASYNC-2:
https://issues.apache.org/jira/browse/HTTPASYNC-1
https://issues.apache.org/jira/browse/HTTPASYNC-1
The idea was to get these things out of the way to prepare for
another coding phase around christmas in which I wanted to tackle
the real issues. Now the current coding phase is basically over,
and I have not achieved a single item on my list. What's worse,
I don't see the situation changing for the next coding phase
around christmas. Maybe, just maybe, I'll get HTTPASYNC-1/2
finished then, and start thinking about the next step. Several
months behind the revised plan, which is already several months
behind the previous plan.
Sure enough, I've achieved other things. We have a new Ant based
build process, and a lot of work has gone into HTTPCORE-11 and
some into HTTPCORE-8. These are important items, that's why I gave
them priority over HttpAsync. It's not like I wasted my efforts.
And still: my declared goal is to contribute HttpAsync, and I have
not made any measurable progress towards that goal. With all the
activities necessarily going on in HttpComponents, HttpAsync is
constantly dangling at the end of my to-do list. Meanwhile, it is
also a constant drag on my mind, and not only during the coding
phases.
And that's why I want it gone from my to-do list. I want to drive
a stake through HttpAsync/CS, now. Let's revisit the ashes some
time in the future, to see whether a phoenix is hiding in there.
When the prerequisites are in place.
With the HttpAsync monkey off my back, I can set new goals like
improving the other components, and fixing the problems we meet on
the way. Accomplishable short-term goals. Goals that allow me to
see some progress. This way, I will get a little more satisfaction
and a lot less frustration out of my activities.
> HttpAsync has not seen a single official release. There is no need to
> make any declarations.
Fair enough. No official declaration needed. Whack! *puff*
> I have always been trying to say that controlled code duplication in
> HttpAsync is a lesser evil because it can shield HttpAsync from API
> instability in HttpCore. Go ahead and roll out an HttpAsync specific
> extension of HttpClientConnection interface and its default
> implementation.
If I have to work on connection management code anyway, I prefer to
do it for HttpConn. Let HttpAsync/CS rest in peace until the time is
ripe for resurrection.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2006-09-17 at 22:07 +0200, Roland Weber wrote:
> Hi Oleg,
>
> >>We could start with an HttpConn module. The decision whether we'll
> >>release it independently or only as part of HttpClient is still
> >>pending, after all. See also below.
> >
> > This is exactly what I see as a proper place for this kind of code
>
> We have to bear in mind that HttpCore is supposed to be useable without
> any other module. Moving default implementations for connections out
> of the core seems to clash with this goal.
The default implementations should stay in HttpCore, but they should be
(at least in my opinion) only implement a simple method allowing the
connection to be bound to an open socket. Code aspects that define how
exactly that socket gets initialized (directly or via a proxy) belongs
to HttpConn in my opinion. This is just a proposal. If you object to
such a change it will not happen.
>
> >>I am considering to officially declare HttpAsync defunct and pulling it
> >>from the website. [...]
> >
> > I disagree rather strongly. It is true that HttpAsync still lacks a
> > clearly defined scope and it represents an experimental branch in
> > HttpComponents development, but I certainly see HttpAsync as a very
> > valuable and important development effort. I personally see it as a set
> > of extensions to HttpCore and HttpClient targeted at those use cases
> > where scalability and efficient thread management is a more important
> > that raw performance delivered by blocking I/O. I also think it should
> > be making an extensive use of NIO extensions to achieve those goals.
> >
> > I personally am very interested in working on an async equivalent of the
> > HttpService class capable of handling a large number of high latency
> > connections with a fixed number of threads.
> >
> > So, overall, it is very premature to declare HttpAsync as defunct. It
> > may take a lower priority to delivering the first ALPHA of HttpClient
> > 4.0, but it by no means a less interesting project.
>
> Well maybe not defunct, but at least suspended. The point is that most
> of the time I wanted to invest into implementing the pending changes to
> HttpAsync (client side) has gone into a new build process and redesigning
> connection handling in HttpCore, and now we're at a point where the
> HttpCore API is not even sufficient to implement HttpAsync/CS anymore.
> I'm tired of being delayed again and again. Every time I want to pick up
> on HttpAsync/CS development, I need to spend an hour with the code to even
> understand where I left it the last time. I'd rather call it quits until
> such time that the required base APIs are stabilized and I can concentrate
> on delivering HttpAsync/CS in one go. It's also fairer to the potential
> users to tell them that they have to wait for months, if not a year or so,
> until there's something to consider for use. Last December, I wrote that I
> hoped to contribute an initial implementation over the course of this year:
> http://mail-archives.apache.org/mod_mbox/jakarta-httpclient-dev/200512.mbox/%3c43948DD9.8060905@dubioso.net%3e
> I am not much closer to this goal than at that time, and I don't see
> that changing for the coming months (6? 9? more?). Giving not only HttpCore
> but also HttpClient and related components a higher priority means that
> there will be none of my time left for the client part of HttpAsync that
> I wanted to contribute. And not giving them a higher priority would mean
> that I have to shoot at a moving target, spending considerable time to
> compensate for code changes in the base APIs, or to implement provisional
> versions of stuff that will most likely have to be rewritten completely
> later on. I consider my time too precious for that. It's better invested
> into HttpCore & Co. Which means that HttpAsync/CS will lie dead until
> further notice.
>
HttpAsync has not seen a single official release. There is no need to
make any declarations.
Client side connection management is the last hurdle that need to be
removed before HttpCore API is ready to be stabilized. The simpler and
the more generic it is going to be the better it is for all the modules
that sit on top of it.
I have always been trying to say that controlled code duplication in
HttpAsync is a lesser evil because it can shield HttpAsync from API
instability in HttpCore. Go ahead and roll out an HttpAsync specific
extension of HttpClientConnection interface and its default
implementation. Once we start seeing code commonalities in HttpAsync,
HttpConn and HttpClient, we can always push common aspects downstream to
HttpCore. It is easier to observe commonalities in code then to predict
them.
Oleg
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
>>We could start with an HttpConn module. The decision whether we'll
>>release it independently or only as part of HttpClient is still
>>pending, after all. See also below.
>
> This is exactly what I see as a proper place for this kind of code
We have to bear in mind that HttpCore is supposed to be useable without
any other module. Moving default implementations for connections out
of the core seems to clash with this goal.
>>I am considering to officially declare HttpAsync defunct and pulling it
>>from the website. [...]
>
> I disagree rather strongly. It is true that HttpAsync still lacks a
> clearly defined scope and it represents an experimental branch in
> HttpComponents development, but I certainly see HttpAsync as a very
> valuable and important development effort. I personally see it as a set
> of extensions to HttpCore and HttpClient targeted at those use cases
> where scalability and efficient thread management is a more important
> that raw performance delivered by blocking I/O. I also think it should
> be making an extensive use of NIO extensions to achieve those goals.
>
> I personally am very interested in working on an async equivalent of the
> HttpService class capable of handling a large number of high latency
> connections with a fixed number of threads.
>
> So, overall, it is very premature to declare HttpAsync as defunct. It
> may take a lower priority to delivering the first ALPHA of HttpClient
> 4.0, but it by no means a less interesting project.
Well maybe not defunct, but at least suspended. The point is that most
of the time I wanted to invest into implementing the pending changes to
HttpAsync (client side) has gone into a new build process and redesigning
connection handling in HttpCore, and now we're at a point where the
HttpCore API is not even sufficient to implement HttpAsync/CS anymore.
I'm tired of being delayed again and again. Every time I want to pick up
on HttpAsync/CS development, I need to spend an hour with the code to even
understand where I left it the last time. I'd rather call it quits until
such time that the required base APIs are stabilized and I can concentrate
on delivering HttpAsync/CS in one go. It's also fairer to the potential
users to tell them that they have to wait for months, if not a year or so,
until there's something to consider for use. Last December, I wrote that I
hoped to contribute an initial implementation over the course of this year:
http://mail-archives.apache.org/mod_mbox/jakarta-httpclient-dev/200512.mbox/%3c43948DD9.8060905@dubioso.net%3e
I am not much closer to this goal than at that time, and I don't see
that changing for the coming months (6? 9? more?). Giving not only HttpCore
but also HttpClient and related components a higher priority means that
there will be none of my time left for the client part of HttpAsync that
I wanted to contribute. And not giving them a higher priority would mean
that I have to shoot at a moving target, spending considerable time to
compensate for code changes in the base APIs, or to implement provisional
versions of stuff that will most likely have to be rewritten completely
later on. I consider my time too precious for that. It's better invested
into HttpCore & Co. Which means that HttpAsync/CS will lie dead until
further notice.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2006-09-17 at 18:24 +0200, Roland Weber wrote:
> Hi Oleg,
> I hate having the same class/interface names in different packages.
> I remember trying to override an initialization method that was given
> a Properties argument. I failed miserably, until I realized that it
> wasn't java.util.Properties but a project specific Properties class.
>
Same here. But right now for the lack of better ideas this is all I can
think of.
> > However, I understand you may want to use this interface in HttpAsync
> >
> We could start with an HttpConn module. The decision whether we'll
> release it independently or only as part of HttpClient is still
> pending, after all. See also below.
>
This is exactly what I see as a proper place for this kind of code
>
> > To make matters even worse we have the following feature request to take
> > care of:
> >
> > https://issues.apache.org/jira/browse/HTTPCLIENT-475
> >
> > This is yet another reason why I think there is no way around having a
> > HttpClient specific HTTP connection and connection management interfaces
> > and another motivating factor for me to reduce the connection management
> > code in HttpCore to an absolute minimum.
> >
> > SocketFactory in HttpCore is very likely to be my next victim.
>
> I am considering to officially declare HttpAsync defunct and pulling it
> from the website. It was always caught between a rock and a hard place,
> since it has to use HttpCore but also needs to provide functionality
> for which (synchronous) APIs will only be discussed as part of HttpClient.
> Now that we are even removing functionality from HttpCore, the work that
> I would have to accomplish in order to make HttpAsync functional is
> just piling up more and more.
I disagree rather strongly. It is true that HttpAsync still lacks a
clearly defined scope and it represents an experimental branch in
HttpComponents development, but I certainly see HttpAsync as a very
valuable and important development effort. I personally see it as a set
of extensions to HttpCore and HttpClient targeted at those use cases
where scalability and efficient thread management is a more important
that raw performance delivered by blocking I/O. I also think it should
be making an extensive use of NIO extensions to achieve those goals.
I personally am very interested in working on an async equivalent of the
HttpService class capable of handling a large number of high latency
connections with a fixed number of threads.
So, overall, it is very premature to declare HttpAsync as defunct. It
may take a lower priority to delivering the first ALPHA of HttpClient
4.0, but it by no means a less interesting project.
Cheers,
Oleg
> And whatever is put into HttpAsync now
> will need to be reconsidered once we're working on the corresponding
> part of HttpClient or one of the other modules. I can't help thinking
> that any time spent on HttpAsync at this stage is wasted.
>
>
>
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
I'll combine my responses in this mail.
>>> AbstractIOReactor has code that deals with a server socket
>>> channel. So it is exclusively server-side? If that is the case,
>>> I would suggest a name change to AbstractServerIOReactor.
>
> The SelectionKey.OP_CONNECT handling code will be added at some point.
I get it. If there is no server socket channel registered at the
selector, then key.isAcceptable() will never be true and no server
side code will ever be executed.
> Was this document of any use at all?
>
> http://wiki.apache.org/jakarta-httpclient/HttpCoreNioApi
Damn, I had forgotten about that. Yes, it would have been helpful :-(
>> [...HttpClientConnection.open() and where to put it...]
> I _really_ would love to resolve this problem by simply moving the
> extended connection interface and its default impl to the HttpClient
> module
>
> interface o.a.httpclient.HttpClientConnection
> extends o.a.http.HttpClientConnection {
>
> void open(...);
>
> }
I hate having the same class/interface names in different packages.
I remember trying to override an initialization method that was given
a Properties argument. I failed miserably, until I realized that it
wasn't java.util.Properties but a project specific Properties class.
> However, I understand you may want to use this interface in HttpAsync
>
We could start with an HttpConn module. The decision whether we'll
release it independently or only as part of HttpClient is still
pending, after all. See also below.
> To make matters even worse we have the following feature request to take
> care of:
>
> https://issues.apache.org/jira/browse/HTTPCLIENT-475
>
> This is yet another reason why I think there is no way around having a
> HttpClient specific HTTP connection and connection management interfaces
> and another motivating factor for me to reduce the connection management
> code in HttpCore to an absolute minimum.
>
> SocketFactory in HttpCore is very likely to be my next victim.
I am considering to officially declare HttpAsync defunct and pulling it
from the website. It was always caught between a rock and a hard place,
since it has to use HttpCore but also needs to provide functionality
for which (synchronous) APIs will only be discussed as part of HttpClient.
Now that we are even removing functionality from HttpCore, the work that
I would have to accomplish in order to make HttpAsync functional is
just piling up more and more. And whatever is put into HttpAsync now
will need to be reconsidered once we're working on the corresponding
part of HttpClient or one of the other modules. I can't help thinking
that any time spent on HttpAsync at this stage is wasted.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2006-09-17 at 03:18 +0200, Roland Weber wrote:
> Hi Oleg,
>
> all my ideas about how to open a socket NIO style have gone down
> when I remembered the [Secure]SocketFactory. NIO socket opening
> is based on a SocketChannel and unconnected sockets. But the
> SocketFactory returns a Socket, and core-main is JDK 1.3 compatible
> while unconnected sockets were not introduced until JDK 1.4.
> I am giving up on suggesting anything here.
>
To make matters even worse we have the following feature request to take
care of:
https://issues.apache.org/jira/browse/HTTPCLIENT-475
This is yet another reason why I think there is no way around having a
HttpClient specific HTTP connection and connection management interfaces
and another motivating factor for me to reduce the connection management
code in HttpCore to an absolute minimum.
SocketFactory in HttpCore is very likely to be my next victim.
Oleg
> cheers,
> Roland
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Roland Weber <ht...@dubioso.net>.
Hi Oleg,
all my ideas about how to open a socket NIO style have gone down
when I remembered the [Secure]SocketFactory. NIO socket opening
is based on a SocketChannel and unconnected sockets. But the
SocketFactory returns a Socket, and core-main is JDK 1.3 compatible
while unconnected sockets were not introduced until JDK 1.4.
I am giving up on suggesting anything here.
cheers,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: HttpCore NIO questions
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2006-09-16 at 20:54 +0200, Roland Weber wrote:
> Hi Oleg,
>
Hi Roland,
> I just did a little code dive to verify my ideas about what
> NIO is doing. As was to be expected, there is a mismatch.
> Maybe you can confirm or correct my understanding of a few
> points:
>
> AbstractIOReactor has code that deals with a server socket
> channel. So it is exclusively server-side? If that is the case,
> I would suggest a name change to AbstractServerIOReactor.
>
The SelectionKey.OP_CONNECT handling code will be added at some point.
The premise here is that there is no clear cut distinction between a
server side and a client side I/O reactor responsibilities, as the same
I/O reactor can act as a listener for incoming connections and as a
connector for outgoing connections when, for instance, used in a
proxy.
> I was a bit puzzled that IOSession has a channel() method
> that returns a ByteChannel, but I just realized that
> java.nio.channels.SocketChannel does implement that interface.
> So IOSession can be used for non-blocking connects on the
> client side.
>
ByteChannel is perfectly sufficient for what IOSession is supposed to
do, so there is no point coupling IOSession with SocketChannel. Besides,
there can be other non-blocking ByteChannels that are not bound to a
Socket (at least theoretically).
> I found it impossible to understand the roles and relations
> between the interfaces defined in o.a.h.nio without looking
> not only into the implementation classes, but into the examples.
> They seem a little disjointed at the interface level. I've gone
> through Doug Lea's presentation that I linked as PDF in the
> package JavaDocs, but I didn't find the IOSession mentioned
> there.
>
Was this document of any use at all?
http://wiki.apache.org/jakarta-httpclient/HttpCoreNioApi
HttpCore NIO is based on Doug Lea's ideas but there are certain
deviations. I did my best to express them in the aforementioned
documents but I do admit I am not much of a writer.
>
>
> The chances that I will come up with a patch for HTTPCORE-8
> this weekend have dropped to virtually 0.
>
HTTPCORE-8 is a difficult one to get right. So, there is no point rushing it.
Cheers
Oleg
>
>
> cheers,
> Roland
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org