You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "J. D." <j1...@yahoo.com> on 2009/06/24 03:11:41 UTC
Http Core NIO Client Using non-socket Channel
I am trying to use HTTP Core NIO module to build highly scalable HTTP clients that do not directly use sockets to connect to peer HTTP servers. The network topology involves a tunnel and hence not amenable to a direct socket to the peer.
[ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
My first thought was to simply use an instance of pair of java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up with my tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink channel will enable HttpClient to write data to my tunnel and pipeIn.source will enable HttpClient to read data from my tunnel. Tunnel module in conjunction with Users interaction would be responsible for channel setup and teardown. Note that Channel setup and teardown entails special tunnelling protocol events and hence cannot be straight Selector event driven like current implementation in DefaultConnectingIOReactor.
The current implementation of HTTP Core expects a SocketChannel and hence it does not seem feasible to directly hook up any other types of channels. Do you recommend an integration strategy that would be inline with future direction?
BaseIOReactor is being created by DefaultConnectingIOReactor and expects a ChannelEntry that requires a SocketChannel. Instead an abstract ChannelEntry that would assume channel is any nio channel could be used. This would however require current code that expects channel to be a socket - e.g. setting socket preferences can check if it is indeed a SocketChannel. I am not sure how pervasive is such a change and if there is a better idea that can prevent such change.
-V.B.
Re: Http Core NIO Client Using non-socket Channel
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Jun 23, 2009 at 06:11:41PM -0700, J. D. wrote:
> I am trying to use HTTP Core NIO module to build highly scalable HTTP clients that do not directly use sockets to connect to peer HTTP servers. The network topology involves a tunnel and hence not amenable to a direct socket to the peer.
>
> [ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
>
> My first thought was to simply use an instance of pair of java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up with my tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink channel will enable HttpClient to write data to my tunnel and pipeIn.source will enable HttpClient to read data from my tunnel. Tunnel module in conjunction with Users interaction would be responsible for channel setup and teardown. Note that Channel setup and teardown entails special tunnelling protocol events and hence cannot be straight Selector event driven like current implementation in DefaultConnectingIOReactor.
>
> The current implementation of HTTP Core expects a SocketChannel and hence it does not seem feasible to directly hook up any other types of channels. Do you recommend an integration strategy that would be inline with future direction?
>
> BaseIOReactor is being created by DefaultConnectingIOReactor and expects a ChannelEntry that requires a SocketChannel. Instead an abstract ChannelEntry that would assume channel is any nio channel could be used. This would however require current code that expects channel to be a socket - e.g. setting socket preferences can check if it is indeed a SocketChannel. I am not sure how pervasive is such a change and if there is a better idea that can prevent such change.
>
> -V.B.
>
V.B.
I see no way around implementing a custom IOReactor. It will be difficult to
change the contract of BaseIOReactor without breaking the API compatibility.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by Oleg Kalnichevski <ol...@apache.org>.
J. D. wrote:
> Thanx Oleg once again for your prompt response.
>>
> olegk wrote:
>>> J. D. wrote:
>>>> Thanx Oleg for your response. I have an NIO as well as non-NIO tunnel
>>> impls.
>>>> Either one can be integrated first and will suffice initially but
>>>> ultimately, I will need to integrate both.
>>>>
>>> J. D.
>>>
>>> The reason I am asking is that trying to mix blocking and non-blocking
>>> I/O models usually produces very suboptimal results. If the tunneling
>>> module were based on blocking I/O, you should have simply chosen
>>> HttpCore blocking I/O module or HttpClient.
>> I am not trying to hookup a blocking tunneling module directly to HTTP
>> NIO.
>> There are adapter(s) to tunneling module(s) that provides the necessary
>> non-blocking abstraction. The tunneling module is decoupled completely
>> from the
>> HTTP module and hence this is not an issue. There are various tunneling
>> modules based on peer's capabilities and it is really not a trivial
>> endeavor to just
>> make its interfacing look like a socket - it is more of a stream i.e. data
>> is
>> sent and received. The connectivity semantics must be decoupled as it goes
>> through a lot more complex processing. I am afraid that will be the case
>> with
>> most complex tunnels.
>>
V.B.,
What I am trying to say is that using non-blocking HTTP transport on top
of an inherently blocking tunnel implementation will give you all the
disadvantages of blocking I/O such as as poorer scalability coupled with
all the disadvantages of NIO such as poorer data throughput without any
advantages of either. In my opinion you should seriously consider using
blocking HttpCore unless the tunneling module is capable of doing NIO
natively.
>>>> I am exploring to see if BaseIOReactor can be updated/extended to
>>> accomodate
>>>> the ability to perform non-socket channel control:
>>>>
>>>> 1) Currently, BaseIOReactor is handed over either an accepted socket
>>> channel
>>>> (from DefaultListeningIOReactor) or a connected socket channel (from
>>>> DefaultConnectingIOReactor) and the channel henceforth performs IO and
>>> close
>>>> within the scope of a BaseIOReactor selector. If current BaseIOReactor
>>>> cannot be altered due to other non-technical reasons, a non-socket
>>> channel
>>>> aware BaseIOReactor can extend current BaseIOReactor with following
>>> changes.
>>>> A factory can be used by
>>>> DefaultConnectingIOReactor/DefaultListeningIOReactor to instantiate the
>>>> appropriate AbstractIOReactor impl.
>>>>
>>>> 2) If processNewChannels (and for future extensibility
>>>> processClosedSessions) is made protected, then a non-socket channel
>>> aware
>>>> BaseIOReactor can skip socket related option configurations, add the
>>> channel
>>>> to the selector, create an IOSession. A better option would be to use a
>>>> ChannelLifecycleController that can be set to perform these operations
>>> for
>>>> more elegant protocol controls in future. Obviously, a custom
>>> ChannelEntry
>>>> that expects a Channel instead of SocketChannel is required.
>>> While it does not look entirely impossible, it may be very hard to
>>> decouple ChannelEntry and all classes dependent on it from SocketChannel.
>> I am not entirely sure if this is the case as the only class that seem to
>> depend on
>> ChannelEntry.getChannel() is
>> org.apache.http.impl.nio.reactor.AbstractIOReactor.
>> This dependency is in processNewChannels and closeNewChannels. It is easy
>> to change processNewChannels() as mentioned above. closeNewChannel
>> obviously need not
>> depend on a SocketChannel - all it needs is a Channel to close it. Please
>> clarify if I am
>> missing something.
>>
Right. It might be very easy to do as long as you do not mind forking
some of the low level reactor classes. It may be significantly more
difficult to pull off without breaking API compatibility with the stock
version of HttpCore.
>>> As far as I understand the tunnel module should be able to expose the
>>> socket it is connected to, so you might want to consider wrapping the
>>> tunnel code with a custom SocketChannel impl instead of trying to fight
>>> your way through the HttpCore NIO code.
>> I need to separate the connectivity semantics from data transfer semantics
>> for
>> tunnel as I explained earlier and it does not seem pragmatic to do so due
>> to the
>> aforesaid reasons.
>>
>> The reason I am persistent on doing it in HTTP NIO modules is because,
>> as I had stated in an earlier mail, the utility of HTTP component is
>> significantly improved if a non-socket channel can be used to perform IO
>> as the
>> HTTP messaging can go through through various communication
>> techs/topologies.
>>
This should be doable as long as you do not mind maintaining your own
implementation of ConnectingIOReactor.
Oleg
>>>> 3) Since IOSession provides access to channel, it is necessary to still
>>>> identify if the IOEventDispatch impls or its users care whether the
>>> channel
>>>> is a socket. A brief look into codebase does not reveal anything else
>>> but
>>>> please advice if that is not the case. At many places, socket specific
>>> ops
>>>> already check whether channel is a socket channel e.g.
>>>> IOSessionImpl.getLocalAddress().
>>>>
>>>> Please advice.
>>>>
>>> I can confirm the IOSession does not depend on any SocketChannel
>>> specific functionality except for #getRemoteAddress() and
>>> #getLocalAddress(). It should work just fine with any selectable channel.
>>>
>>> Hope this helps somewhat.
>>>
>>> Oleg
>>>
>>>
>>>> -V.B.
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: dev-help@hc.apache.org
>>>
>>>
>>>
>
> Please advise.
>
> -V.B.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by "J. D." <j1...@yahoo.com>.
Thanx Oleg once again for your prompt response.
>
>
olegk wrote:
>
>> J. D. wrote:
>> > Thanx Oleg for your response. I have an NIO as well as non-NIO tunnel
>> impls.
>> > Either one can be integrated first and will suffice initially but
>> > ultimately, I will need to integrate both.
>> >
>>
>> J. D.
>>
>> The reason I am asking is that trying to mix blocking and non-blocking
>> I/O models usually produces very suboptimal results. If the tunneling
>> module were based on blocking I/O, you should have simply chosen
>> HttpCore blocking I/O module or HttpClient.
>
> I am not trying to hookup a blocking tunneling module directly to HTTP
> NIO.
> There are adapter(s) to tunneling module(s) that provides the necessary
> non-blocking abstraction. The tunneling module is decoupled completely
> from the
> HTTP module and hence this is not an issue. There are various tunneling
> modules based on peer's capabilities and it is really not a trivial
> endeavor to just
> make its interfacing look like a socket - it is more of a stream i.e. data
> is
> sent and received. The connectivity semantics must be decoupled as it goes
> through a lot more complex processing. I am afraid that will be the case
> with
> most complex tunnels.
>
>> > I am exploring to see if BaseIOReactor can be updated/extended to
>> accomodate
>> > the ability to perform non-socket channel control:
>> >
>> > 1) Currently, BaseIOReactor is handed over either an accepted socket
>> channel
>> > (from DefaultListeningIOReactor) or a connected socket channel (from
>> > DefaultConnectingIOReactor) and the channel henceforth performs IO and
>> close
>> > within the scope of a BaseIOReactor selector. If current BaseIOReactor
>> > cannot be altered due to other non-technical reasons, a non-socket
>> channel
>> > aware BaseIOReactor can extend current BaseIOReactor with following
>> changes.
>> > A factory can be used by
>> > DefaultConnectingIOReactor/DefaultListeningIOReactor to instantiate the
>> > appropriate AbstractIOReactor impl.
>> >
>> > 2) If processNewChannels (and for future extensibility
>> > processClosedSessions) is made protected, then a non-socket channel
>> aware
>> > BaseIOReactor can skip socket related option configurations, add the
>> channel
>> > to the selector, create an IOSession. A better option would be to use a
>> > ChannelLifecycleController that can be set to perform these operations
>> for
>> > more elegant protocol controls in future. Obviously, a custom
>> ChannelEntry
>> > that expects a Channel instead of SocketChannel is required.
>>
>> While it does not look entirely impossible, it may be very hard to
>> decouple ChannelEntry and all classes dependent on it from SocketChannel.
>
> I am not entirely sure if this is the case as the only class that seem to
> depend on
> ChannelEntry.getChannel() is
> org.apache.http.impl.nio.reactor.AbstractIOReactor.
> This dependency is in processNewChannels and closeNewChannels. It is easy
> to change processNewChannels() as mentioned above. closeNewChannel
> obviously need not
> depend on a SocketChannel - all it needs is a Channel to close it. Please
> clarify if I am
> missing something.
>
>> As far as I understand the tunnel module should be able to expose the
>> socket it is connected to, so you might want to consider wrapping the
>> tunnel code with a custom SocketChannel impl instead of trying to fight
>> your way through the HttpCore NIO code.
>
> I need to separate the connectivity semantics from data transfer semantics
> for
> tunnel as I explained earlier and it does not seem pragmatic to do so due
> to the
> aforesaid reasons.
>
> The reason I am persistent on doing it in HTTP NIO modules is because,
> as I had stated in an earlier mail, the utility of HTTP component is
> significantly improved if a non-socket channel can be used to perform IO
> as the
> HTTP messaging can go through through various communication
> techs/topologies.
>
>> >
>> > 3) Since IOSession provides access to channel, it is necessary to still
>> > identify if the IOEventDispatch impls or its users care whether the
>> channel
>> > is a socket. A brief look into codebase does not reveal anything else
>> but
>> > please advice if that is not the case. At many places, socket specific
>> ops
>> > already check whether channel is a socket channel e.g.
>> > IOSessionImpl.getLocalAddress().
>> >
>> > Please advice.
>> >
>>
>> I can confirm the IOSession does not depend on any SocketChannel
>> specific functionality except for #getRemoteAddress() and
>> #getLocalAddress(). It should work just fine with any selectable channel.
>>
>> Hope this helps somewhat.
>>
>> Oleg
>>
>>
>> > -V.B.
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>>
>>
>
Please advise.
-V.B.
--
View this message in context: http://www.nabble.com/Http-Core-NIO-Client-Using-non-socket-Channel-tp24179229p24260709.html
Sent from the HttpComponents-Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by Oleg Kalnichevski <ol...@apache.org>.
J. D. wrote:
> Thanx Oleg for your response. I have an NIO as well as non-NIO tunnel impls.
> Either one can be integrated first and will suffice initially but
> ultimately, I will need to integrate both.
>
J. D.
The reason I am asking is that trying to mix blocking and non-blocking
I/O models usually produces very suboptimal results. If the tunneling
module were based on blocking I/O, you should have simply chosen
HttpCore blocking I/O module or HttpClient.
> I am exploring to see if BaseIOReactor can be updated/extended to accomodate
> the ability to perform non-socket channel control:
>
> 1) Currently, BaseIOReactor is handed over either an accepted socket channel
> (from DefaultListeningIOReactor) or a connected socket channel (from
> DefaultConnectingIOReactor) and the channel henceforth performs IO and close
> within the scope of a BaseIOReactor selector. If current BaseIOReactor
> cannot be altered due to other non-technical reasons, a non-socket channel
> aware BaseIOReactor can extend current BaseIOReactor with following changes.
> A factory can be used by
> DefaultConnectingIOReactor/DefaultListeningIOReactor to instantiate the
> appropriate AbstractIOReactor impl.
>
> 2) If processNewChannels (and for future extensibility
> processClosedSessions) is made protected, then a non-socket channel aware
> BaseIOReactor can skip socket related option configurations, add the channel
> to the selector, create an IOSession. A better option would be to use a
> ChannelLifecycleController that can be set to perform these operations for
> more elegant protocol controls in future. Obviously, a custom ChannelEntry
> that expects a Channel instead of SocketChannel is required.
While it does not look entirely impossible, it may be very hard to
decouple ChannelEntry and all classes dependent on it from SocketChannel.
As far as I understand the tunnel module should be able to expose the
socket it is connected to, so you might want to consider wrapping the
tunnel code with a custom SocketChannel impl instead of trying to fight
your way through the HttpCore NIO code.
>
> 3) Since IOSession provides access to channel, it is necessary to still
> identify if the IOEventDispatch impls or its users care whether the channel
> is a socket. A brief look into codebase does not reveal anything else but
> please advice if that is not the case. At many places, socket specific ops
> already check whether channel is a socket channel e.g.
> IOSessionImpl.getLocalAddress().
>
> Please advice.
>
I can confirm the IOSession does not depend on any SocketChannel
specific functionality except for #getRemoteAddress() and
#getLocalAddress(). It should work just fine with any selectable channel.
Hope this helps somewhat.
Oleg
> -V.B.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by "J. D." <j1...@yahoo.com>.
Thanx Oleg for your response. I have an NIO as well as non-NIO tunnel impls.
Either one can be integrated first and will suffice initially but
ultimately, I will need to integrate both.
I am exploring to see if BaseIOReactor can be updated/extended to accomodate
the ability to perform non-socket channel control:
1) Currently, BaseIOReactor is handed over either an accepted socket channel
(from DefaultListeningIOReactor) or a connected socket channel (from
DefaultConnectingIOReactor) and the channel henceforth performs IO and close
within the scope of a BaseIOReactor selector. If current BaseIOReactor
cannot be altered due to other non-technical reasons, a non-socket channel
aware BaseIOReactor can extend current BaseIOReactor with following changes.
A factory can be used by
DefaultConnectingIOReactor/DefaultListeningIOReactor to instantiate the
appropriate AbstractIOReactor impl.
2) If processNewChannels (and for future extensibility
processClosedSessions) is made protected, then a non-socket channel aware
BaseIOReactor can skip socket related option configurations, add the channel
to the selector, create an IOSession. A better option would be to use a
ChannelLifecycleController that can be set to perform these operations for
more elegant protocol controls in future. Obviously, a custom ChannelEntry
that expects a Channel instead of SocketChannel is required.
3) Since IOSession provides access to channel, it is necessary to still
identify if the IOEventDispatch impls or its users care whether the channel
is a socket. A brief look into codebase does not reveal anything else but
please advice if that is not the case. At many places, socket specific ops
already check whether channel is a socket channel e.g.
IOSessionImpl.getLocalAddress().
Please advice.
-V.B.
olegk wrote:
>
> On Thu, Jun 25, 2009 at 04:55:38PM -0700, J. D. wrote:
>>
>> Thanks Ortwin for your SocketFactory idea. I did explore that idea
>> earlier
>> and found it to be a fairly cumbersome effort.
>>
>> Problem is with tunnel, you also have to deal with multiplexing multiple
>> streams onto a tunnel stream. A socket factory would hand you a socket -
>> which can be some abstraction of a single logical stream and therefore
>> you
>> end up writing your own SocketChannel impl that would end up handling
>> tunnel
>> protocol and the socket streaming specifics - which if not an optimal
>> design
>> would atleast be a fairly intricate development effort.
>>
>> I realize that BaseIOReactor depends on a SocketChannel which in itself
>> poses this problem. However, in reality, what HTTPCore provides is a HTTP
>> protocol tier that can/should abstract itself from the transport
>> specifics
>> and therein I started contemplating that maybe BaseIOReactor should not
>> depend on a SocketChannel but instead should depend on an abstraction of
>> a
>> communication pathway that could be a SocketChannel or any other Channel.
>>
>> -V.B.
>>
>>
>
> V.B.
>
> This may be difficult or even impossible to do without breaking API
> compatibility, which cannot afford until the 5.0 time frame. Please by all
> of
> means do feel free to submit a patch with the intended changes, though.
>
> By the way, is the tunneling module you are using based on NIO?
>
> Oleg
>
>
>
>>
>> Ortwin Gl??ck wrote:
>> >
>> > V.B,
>> >
>> > To me this looks similar to TLS. Can you thus not just implement a
>> custom
>> > SocketFactory? That's usually the way how you would integrate a TLS
>> > library that
>> > doesn't support JSSE as well.
>> >
>> > Ortwin
>> >
>> >
>> > J. D. wrote:
>> >> I am trying to use HTTP Core NIO module to build highly scalable HTTP
>> >> clients
>> >> that do not directly use sockets to connect to peer HTTP servers. The
>> >> network
>> >> topology involves a tunnel and hence not amenable to a direct socket
>> to
>> >> the
>> >> peer.
>> >>
>> >> [ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
>> >>
>> >> My first thought was to simply use an instance of pair of
>> >> java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up
>> >> with my
>> >> tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink
>> channel
>> >> will
>> >> enable HttpClient to write data to my tunnel and pipeIn.source will
>> >> enable
>> >> HttpClient to read data from my tunnel. Tunnel module in conjunction
>> with
>> >> Users interaction would be responsible for channel setup and teardown.
>> >> Note
>> >> that Channel setup and teardown entails special tunnelling protocol
>> >> events
>> >> and hence cannot be straight Selector event driven like current
>> >> implementation in DefaultConnectingIOReactor.
>> >>
>> >> The current implementation of HTTP Core expects a SocketChannel and
>> hence
>> >> it
>> >> does not seem feasible to directly hook up any other types of
>> channels.
>> >> Do
>> >> you recommend an integration strategy that would be inline with future
>> >> direction?
>> >>
>> >> BaseIOReactor is being created by DefaultConnectingIOReactor and
>> expects
>> >> a
>> >> ChannelEntry that requires a SocketChannel. Instead an abstract
>> >> ChannelEntry
>> >> that would assume channel is any nio channel could be used. This would
>> >> however require current code that expects channel to be a socket -
>> e.g.
>> >> setting socket preferences can check if it is indeed a SocketChannel.
>> I
>> >> am
>> >> not sure how pervasive is such a change and if there is a better idea
>> >> that
>> >> can prevent such change.
>> >>
>> >> -V.B.
>> >
>> > --
>> > [web] http://www.odi.ch/
>> > [blog] http://www.odi.ch/weblog/
>> > [pgp] key 0x81CF3416
>> > finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> > For additional commands, e-mail: dev-help@hc.apache.org
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Http-Core-NIO-Client-Using-non-socket-Channel-tp24179229p24212890.html
>> Sent from the HttpComponents-Dev mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Http-Core-NIO-Client-Using-non-socket-Channel-tp24179229p24234943.html
Sent from the HttpComponents-Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, Jun 25, 2009 at 04:55:38PM -0700, J. D. wrote:
>
> Thanks Ortwin for your SocketFactory idea. I did explore that idea earlier
> and found it to be a fairly cumbersome effort.
>
> Problem is with tunnel, you also have to deal with multiplexing multiple
> streams onto a tunnel stream. A socket factory would hand you a socket -
> which can be some abstraction of a single logical stream and therefore you
> end up writing your own SocketChannel impl that would end up handling tunnel
> protocol and the socket streaming specifics - which if not an optimal design
> would atleast be a fairly intricate development effort.
>
> I realize that BaseIOReactor depends on a SocketChannel which in itself
> poses this problem. However, in reality, what HTTPCore provides is a HTTP
> protocol tier that can/should abstract itself from the transport specifics
> and therein I started contemplating that maybe BaseIOReactor should not
> depend on a SocketChannel but instead should depend on an abstraction of a
> communication pathway that could be a SocketChannel or any other Channel.
>
> -V.B.
>
>
V.B.
This may be difficult or even impossible to do without breaking API
compatibility, which cannot afford until the 5.0 time frame. Please by all of
means do feel free to submit a patch with the intended changes, though.
By the way, is the tunneling module you are using based on NIO?
Oleg
>
> Ortwin Gl??ck wrote:
> >
> > V.B,
> >
> > To me this looks similar to TLS. Can you thus not just implement a custom
> > SocketFactory? That's usually the way how you would integrate a TLS
> > library that
> > doesn't support JSSE as well.
> >
> > Ortwin
> >
> >
> > J. D. wrote:
> >> I am trying to use HTTP Core NIO module to build highly scalable HTTP
> >> clients
> >> that do not directly use sockets to connect to peer HTTP servers. The
> >> network
> >> topology involves a tunnel and hence not amenable to a direct socket to
> >> the
> >> peer.
> >>
> >> [ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
> >>
> >> My first thought was to simply use an instance of pair of
> >> java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up
> >> with my
> >> tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink channel
> >> will
> >> enable HttpClient to write data to my tunnel and pipeIn.source will
> >> enable
> >> HttpClient to read data from my tunnel. Tunnel module in conjunction with
> >> Users interaction would be responsible for channel setup and teardown.
> >> Note
> >> that Channel setup and teardown entails special tunnelling protocol
> >> events
> >> and hence cannot be straight Selector event driven like current
> >> implementation in DefaultConnectingIOReactor.
> >>
> >> The current implementation of HTTP Core expects a SocketChannel and hence
> >> it
> >> does not seem feasible to directly hook up any other types of channels.
> >> Do
> >> you recommend an integration strategy that would be inline with future
> >> direction?
> >>
> >> BaseIOReactor is being created by DefaultConnectingIOReactor and expects
> >> a
> >> ChannelEntry that requires a SocketChannel. Instead an abstract
> >> ChannelEntry
> >> that would assume channel is any nio channel could be used. This would
> >> however require current code that expects channel to be a socket - e.g.
> >> setting socket preferences can check if it is indeed a SocketChannel. I
> >> am
> >> not sure how pervasive is such a change and if there is a better idea
> >> that
> >> can prevent such change.
> >>
> >> -V.B.
> >
> > --
> > [web] http://www.odi.ch/
> > [blog] http://www.odi.ch/weblog/
> > [pgp] key 0x81CF3416
> > finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Http-Core-NIO-Client-Using-non-socket-Channel-tp24179229p24212890.html
> Sent from the HttpComponents-Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by "J. D." <j1...@yahoo.com>.
Thanks Ortwin for your SocketFactory idea. I did explore that idea earlier
and found it to be a fairly cumbersome effort.
Problem is with tunnel, you also have to deal with multiplexing multiple
streams onto a tunnel stream. A socket factory would hand you a socket -
which can be some abstraction of a single logical stream and therefore you
end up writing your own SocketChannel impl that would end up handling tunnel
protocol and the socket streaming specifics - which if not an optimal design
would atleast be a fairly intricate development effort.
I realize that BaseIOReactor depends on a SocketChannel which in itself
poses this problem. However, in reality, what HTTPCore provides is a HTTP
protocol tier that can/should abstract itself from the transport specifics
and therein I started contemplating that maybe BaseIOReactor should not
depend on a SocketChannel but instead should depend on an abstraction of a
communication pathway that could be a SocketChannel or any other Channel.
-V.B.
Ortwin Glück wrote:
>
> V.B,
>
> To me this looks similar to TLS. Can you thus not just implement a custom
> SocketFactory? That's usually the way how you would integrate a TLS
> library that
> doesn't support JSSE as well.
>
> Ortwin
>
>
> J. D. wrote:
>> I am trying to use HTTP Core NIO module to build highly scalable HTTP
>> clients
>> that do not directly use sockets to connect to peer HTTP servers. The
>> network
>> topology involves a tunnel and hence not amenable to a direct socket to
>> the
>> peer.
>>
>> [ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
>>
>> My first thought was to simply use an instance of pair of
>> java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up
>> with my
>> tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink channel
>> will
>> enable HttpClient to write data to my tunnel and pipeIn.source will
>> enable
>> HttpClient to read data from my tunnel. Tunnel module in conjunction with
>> Users interaction would be responsible for channel setup and teardown.
>> Note
>> that Channel setup and teardown entails special tunnelling protocol
>> events
>> and hence cannot be straight Selector event driven like current
>> implementation in DefaultConnectingIOReactor.
>>
>> The current implementation of HTTP Core expects a SocketChannel and hence
>> it
>> does not seem feasible to directly hook up any other types of channels.
>> Do
>> you recommend an integration strategy that would be inline with future
>> direction?
>>
>> BaseIOReactor is being created by DefaultConnectingIOReactor and expects
>> a
>> ChannelEntry that requires a SocketChannel. Instead an abstract
>> ChannelEntry
>> that would assume channel is any nio channel could be used. This would
>> however require current code that expects channel to be a socket - e.g.
>> setting socket preferences can check if it is indeed a SocketChannel. I
>> am
>> not sure how pervasive is such a change and if there is a better idea
>> that
>> can prevent such change.
>>
>> -V.B.
>
> --
> [web] http://www.odi.ch/
> [blog] http://www.odi.ch/weblog/
> [pgp] key 0x81CF3416
> finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Http-Core-NIO-Client-Using-non-socket-Channel-tp24179229p24212890.html
Sent from the HttpComponents-Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Http Core NIO Client Using non-socket Channel
Posted by Ortwin Glück <od...@odi.ch>.
V.B,
To me this looks similar to TLS. Can you thus not just implement a custom
SocketFactory? That's usually the way how you would integrate a TLS library that
doesn't support JSSE as well.
Ortwin
J. D. wrote:
> I am trying to use HTTP Core NIO module to build highly scalable HTTP clients
> that do not directly use sockets to connect to peer HTTP servers. The network
> topology involves a tunnel and hence not amenable to a direct socket to the
> peer.
>
> [ Users <==> HttpClients <==> Tunnel(s) ] <==> [ Peer HttpServers ]
>
> My first thought was to simply use an instance of pair of
> java.nio.channels.Pipe pipeIn, pipeOut and use these pipes to hook up with my
> tunnel IO module. For outcoming (w.r.t. HttpClient) pipeOut.sink channel will
> enable HttpClient to write data to my tunnel and pipeIn.source will enable
> HttpClient to read data from my tunnel. Tunnel module in conjunction with
> Users interaction would be responsible for channel setup and teardown. Note
> that Channel setup and teardown entails special tunnelling protocol events
> and hence cannot be straight Selector event driven like current
> implementation in DefaultConnectingIOReactor.
>
> The current implementation of HTTP Core expects a SocketChannel and hence it
> does not seem feasible to directly hook up any other types of channels. Do
> you recommend an integration strategy that would be inline with future
> direction?
>
> BaseIOReactor is being created by DefaultConnectingIOReactor and expects a
> ChannelEntry that requires a SocketChannel. Instead an abstract ChannelEntry
> that would assume channel is any nio channel could be used. This would
> however require current code that expects channel to be a socket - e.g.
> setting socket preferences can check if it is indeed a SocketChannel. I am
> not sure how pervasive is such a change and if there is a better idea that
> can prevent such change.
>
> -V.B.
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org