You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Thomas <ma...@apache.org> on 2015/05/01 11:55:16 UTC
Tomcat 9 connector refactoring: NIO vs NIO2
Next up on my list is the NIO2 connector.
NIO and APR/native both use a polling approach to non-blocking I/O. You
add the socket to the poller, tell it what operation (read/write) you
want to perform and then you wait for the poller to tell you the socket
is ready to perform that operation.
NIO2 uses an asynchronous approach to non-blocking I/O. You perform the
read/write and then wait to be told it has finished via either a Future
or a CompletionHandler.
Servlet 3.1 non-blocking I/O is closest to the polling style (you get a
callback when you are allowed to read/write).
WebSocket non-blocking I/O uses the asynchronous style.
In short, regardless of the underlying approach to non-blocking I/O, we
have to support JavaEE APIs that use both styles. Therefore there is no
'obvious' advantage for either NIO or NIO2.
As far as I can tell, the performance of NIO and NIO2 are comparable.
That raises the question why do we need both NIO and NIO2? And I don't
have an answer to that. If I had to pick one, I'd pick NIO because:
- it has been around longer and is more stable
- it uses the same style as APR/native which may allow further
refactoring to reduce duplication.
So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
yes, which one?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Mark Thomas <ma...@apache.org>.
On 01/05/2015 14:02, Mark Thomas wrote:
> On 01/05/2015 13:42, Jess Holle wrote:
>> Naively, looking at the Tomcat documentation and note the comparison
>> table there makes it appear that:
>>
>> 1. NIO2 offers everything NIO does
>> 2. NIO cannot offer true blocking IO, whereas NIO2 can
>>
>> If that's not true, then the documentation should be updated at least....
>
> It depends how you define "true blocking IO". None of the three
> connectors in trunk provide blocking I/O via a single read or write
> call. For all of them you call read/write and then you need to call
> something else that blocks until the read/write has finished. Only BIO
> offered "true" blocking I/O in my view.
I take that back. APR does offer true blocking.
Mark
> I'll update the docs.
>
> Mark
>
>
>>
>> And, of course, NIO2 just sounds newer/better/fancier than NIO :-)
>>
>> On 5/1/2015 4:55 AM, Mark Thomas wrote:
>>> Next up on my list is the NIO2 connector.
>>>
>>> NIO and APR/native both use a polling approach to non-blocking I/O. You
>>> add the socket to the poller, tell it what operation (read/write) you
>>> want to perform and then you wait for the poller to tell you the socket
>>> is ready to perform that operation.
>>>
>>> NIO2 uses an asynchronous approach to non-blocking I/O. You perform the
>>> read/write and then wait to be told it has finished via either a Future
>>> or a CompletionHandler.
>>>
>>> Servlet 3.1 non-blocking I/O is closest to the polling style (you get a
>>> callback when you are allowed to read/write).
>>>
>>> WebSocket non-blocking I/O uses the asynchronous style.
>>>
>>> In short, regardless of the underlying approach to non-blocking I/O, we
>>> have to support JavaEE APIs that use both styles. Therefore there is no
>>> 'obvious' advantage for either NIO or NIO2.
>>>
>>> As far as I can tell, the performance of NIO and NIO2 are comparable.
>>>
>>> That raises the question why do we need both NIO and NIO2? And I don't
>>> have an answer to that. If I had to pick one, I'd pick NIO because:
>>> - it has been around longer and is more stable
>>> - it uses the same style as APR/native which may allow further
>>> refactoring to reduce duplication.
>>>
>>> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
>>> yes, which one?
>>>
>>> Mark
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>>
>>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Mark Thomas <ma...@apache.org>.
On 01/05/2015 13:42, Jess Holle wrote:
> Naively, looking at the Tomcat documentation and note the comparison
> table there makes it appear that:
>
> 1. NIO2 offers everything NIO does
> 2. NIO cannot offer true blocking IO, whereas NIO2 can
>
> If that's not true, then the documentation should be updated at least....
It depends how you define "true blocking IO". None of the three
connectors in trunk provide blocking I/O via a single read or write
call. For all of them you call read/write and then you need to call
something else that blocks until the read/write has finished. Only BIO
offered "true" blocking I/O in my view.
I'll update the docs.
Mark
>
> And, of course, NIO2 just sounds newer/better/fancier than NIO :-)
>
> On 5/1/2015 4:55 AM, Mark Thomas wrote:
>> Next up on my list is the NIO2 connector.
>>
>> NIO and APR/native both use a polling approach to non-blocking I/O. You
>> add the socket to the poller, tell it what operation (read/write) you
>> want to perform and then you wait for the poller to tell you the socket
>> is ready to perform that operation.
>>
>> NIO2 uses an asynchronous approach to non-blocking I/O. You perform the
>> read/write and then wait to be told it has finished via either a Future
>> or a CompletionHandler.
>>
>> Servlet 3.1 non-blocking I/O is closest to the polling style (you get a
>> callback when you are allowed to read/write).
>>
>> WebSocket non-blocking I/O uses the asynchronous style.
>>
>> In short, regardless of the underlying approach to non-blocking I/O, we
>> have to support JavaEE APIs that use both styles. Therefore there is no
>> 'obvious' advantage for either NIO or NIO2.
>>
>> As far as I can tell, the performance of NIO and NIO2 are comparable.
>>
>> That raises the question why do we need both NIO and NIO2? And I don't
>> have an answer to that. If I had to pick one, I'd pick NIO because:
>> - it has been around longer and is more stable
>> - it uses the same style as APR/native which may allow further
>> refactoring to reduce duplication.
>>
>> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
>> yes, which one?
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Jess Holle <je...@ptc.com>.
Naively, looking at the Tomcat documentation and note the comparison
table there makes it appear that:
1. NIO2 offers everything NIO does
2. NIO cannot offer true blocking IO, whereas NIO2 can
If that's not true, then the documentation should be updated at least....
And, of course, NIO2 just sounds newer/better/fancier than NIO :-)
On 5/1/2015 4:55 AM, Mark Thomas wrote:
> Next up on my list is the NIO2 connector.
>
> NIO and APR/native both use a polling approach to non-blocking I/O. You
> add the socket to the poller, tell it what operation (read/write) you
> want to perform and then you wait for the poller to tell you the socket
> is ready to perform that operation.
>
> NIO2 uses an asynchronous approach to non-blocking I/O. You perform the
> read/write and then wait to be told it has finished via either a Future
> or a CompletionHandler.
>
> Servlet 3.1 non-blocking I/O is closest to the polling style (you get a
> callback when you are allowed to read/write).
>
> WebSocket non-blocking I/O uses the asynchronous style.
>
> In short, regardless of the underlying approach to non-blocking I/O, we
> have to support JavaEE APIs that use both styles. Therefore there is no
> 'obvious' advantage for either NIO or NIO2.
>
> As far as I can tell, the performance of NIO and NIO2 are comparable.
>
> That raises the question why do we need both NIO and NIO2? And I don't
> have an answer to that. If I had to pick one, I'd pick NIO because:
> - it has been around longer and is more stable
> - it uses the same style as APR/native which may allow further
> refactoring to reduce duplication.
>
> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
> yes, which one?
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark,
On 5/1/15 1:28 PM, Mark Thomas wrote:
> On 01/05/2015 15:03, Rémy Maucherat wrote:
>> 2015-05-01 11:55 GMT+02:00 Mark Thomas <ma...@apache.org>:
>
>>> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
>>> yes, which one?
>>>
>>> I volunteer to help maintain NIO2 for the time being, so I am not in favor
>> of removing it at the moment. Unfortunately, I think all three connectors
>> have some good points.
>>
>> APR:
>> - Blocking IO
>> - OpenSSL (for the time being)
>
> If you can get OpenSSL working with the Java connectors then that opens
> up the question why keep the APR/native connector. But we aren't there yet.
It looks like Harmony had an OpenSSL crypto provider for Java, and it
seems that Android might still be caring for it. Jean-Frederic expressed
some interest in the past in having a JSSE provider based upon OpenSSL
as well. I'm not sure if there's corporate will behind that or not.
One of these days, the JRE is going to fix all their bugs preventing
hardware-accelerated crypto, and the difference between JSSE and OpenSSL
is going to be reduced.
>> - Sendfile
>>
>> NIO:
>> - It's been around longer :)
>> - Sendfile
>>
>> NIO2:
>> - Modern async IO
>> - Scatter / gather IO that can be exposed and taken advantage (see the new
>> IO calls I added; implementing them with APR and NIO is going to be a whole
>> lot more convoluted ...)
>> - Probably HTTP/2 and Servlet.next will take advantage of it just like
>> websockets did
>
> WebSockets didn't take advantage of it scatter/gather. Neither did it
> take advantage of the async style of API.
>
>> I don't think NIO has gotten any better, it's still the most horrendous IO
>> API imaginable as far as I am concerned. Of course, you can use frameworks
>> and stuff but ... I agree to keep it as well, since as you say it's more
>> mature and stable, but that's about it.
>
> The main driver for this thinking is reducing complexity in the
> connectors. Having to support both poller style and async style basic
> I/O creates complexity. If we only supported one style the I/O code
> could be a lot cleaner.
That suggests that getting rid of NIO also implies getting rid of APR.
IS that your thinking?
> Overall, I think I prefer the async style. It is much easier to simulate
> blocking and requires less supporting code (no pollers etc).
>
> But, APR uses the poller style and it has much better SSL performance.
>
> Thinking ahead (not sure how far, maybe Tomcat 9 at the outside) but if
> we had OpenSSL working with NIO.2 and performance was similar to the
> APR/native connector I'd be all for dropping NIO and APR/native in
> favour of NIO2 and with the option to use OpenSSL or JSSE for SSL.
>
> Is that too extreme?
It's going to need a huge amount of testing, obviously. Ognjen and Felix
found some problems with the NIO2 implementation in the latest 8.0.x
candidate (though NIO2 is still considered "experimental" in Tomcat 8),
so it's obviously not ready for prime-time just yet.
One connector to rule them all. It definitely sounds like a good goal.
-chris
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Rémy Maucherat <re...@apache.org>.
2015-05-04 21:54 GMT+02:00 Mark Thomas <ma...@apache.org>:
> Looking some time further ahead where NIO2 is as stable as NIO and there
> is an OpenSSL option for SSL/TLS with NIO2 I don't see any advantages of
> NIO or APR/native over NIO2 which raises the possibility - at that point
> - of just having NIO2.
>
> At this point I'm not proposing anything - just putting the idea out
> there for discussion. I think we are a long way off being in a position
> to drop any of the connectors at this point.
>
> Looking further ahead at the SSL configuration, SSLHostConfig is a good
help for OpenSSL with NIO, since it contains both the JSSE config and the
OpenSSL config.
Rémy
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Mark Thomas <ma...@apache.org>.
On 02/05/2015 21:52, Rémy Maucherat wrote:
> 2015-05-01 19:28 GMT+02:00 Mark Thomas <ma...@apache.org>:
>
>> On 01/05/2015 15:03, Rémy Maucherat wrote:
>>> 2015-05-01 11:55 GMT+02:00 Mark Thomas <ma...@apache.org>:
>>
>>>> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
>>>> yes, which one?
>>>>
>>>> I volunteer to help maintain NIO2 for the time being, so I am not in
>> favor
>>> of removing it at the moment. Unfortunately, I think all three connectors
>>> have some good points.
>>>
>>> APR:
>>> - Blocking IO
>>> - OpenSSL (for the time being)
>>
>> If you can get OpenSSL working with the Java connectors then that opens
>> up the question why keep the APR/native connector. But we aren't there
>> yet.
>>
>>> - Sendfile
>>>
>>> NIO:
>>> - It's been around longer :)
>>> - Sendfile
>>>
>>> NIO2:
>>> - Modern async IO
>>> - Scatter / gather IO that can be exposed and taken advantage (see the
>> new
>>> IO calls I added; implementing them with APR and NIO is going to be a
>> whole
>>> lot more convoluted ...)
>>> - Probably HTTP/2 and Servlet.next will take advantage of it just like
>>> websockets did
>>
>> WebSockets didn't take advantage of it scatter/gather. Neither did it
>> take advantage of the async style of API.
>>
>
> Websockets could easily take advantage of it. For example the code in
> WsRemoteEndpointImplServer and all its for loops over arrays of buffers to
> write them looks rather close to a gather write to me and could probably
> all be replaced with a single write.
I agree it could. My point was that, contrary to your original
statement, it doesn't currently.
APR/native looks to have support for gathered write as well. There is
definitely scope to simplify WebSocket code. NIO would still need the
loop over the arrays but that would move to the NIOSocketWrapper.
>>> I don't think NIO has gotten any better, it's still the most horrendous
>> IO
>>> API imaginable as far as I am concerned. Of course, you can use
>> frameworks
>>> and stuff but ... I agree to keep it as well, since as you say it's more
>>> mature and stable, but that's about it.
>>
>> The main driver for this thinking is reducing complexity in the
>> connectors. Having to support both poller style and async style basic
>> I/O creates complexity. If we only supported one style the I/O code
>> could be a lot cleaner.
>
> Of course, things can always be cleaner, but this causes issues. For
> example, the one I dislike the most is NIO, but as you said it is the most
> stable, so removing it is probably not a good idea.
Right now, I agree with that assessment.
>> Overall, I think I prefer the async style. It is much easier to simulate
>> blocking and requires less supporting code (no pollers etc).
>>
>> But, APR uses the poller style and it has much better SSL performance.
>>
>> Thinking ahead (not sure how far, maybe Tomcat 9 at the outside) but if
>> we had OpenSSL working with NIO.2 and performance was similar to the
>> APR/native connector I'd be all for dropping NIO and APR/native in
>> favour of NIO2 and with the option to use OpenSSL or JSSE for SSL.
>
> I was planning to send an email about that on Monday, so I guess I'm
> sending it now instead.
>
>> Is that too extreme?
>>
> The list of "pros" for each connector is not empty, so I don't see why it
> helps to remove one.
Looking some time further ahead where NIO2 is as stable as NIO and there
is an OpenSSL option for SSL/TLS with NIO2 I don't see any advantages of
NIO or APR/native over NIO2 which raises the possibility - at that point
- of just having NIO2.
At this point I'm not proposing anything - just putting the idea out
there for discussion. I think we are a long way off being in a position
to drop any of the connectors at this point.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Rémy Maucherat <re...@apache.org>.
2015-05-01 19:28 GMT+02:00 Mark Thomas <ma...@apache.org>:
> On 01/05/2015 15:03, Rémy Maucherat wrote:
> > 2015-05-01 11:55 GMT+02:00 Mark Thomas <ma...@apache.org>:
>
> >> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
> >> yes, which one?
> >>
> >> I volunteer to help maintain NIO2 for the time being, so I am not in
> favor
> > of removing it at the moment. Unfortunately, I think all three connectors
> > have some good points.
> >
> > APR:
> > - Blocking IO
> > - OpenSSL (for the time being)
>
> If you can get OpenSSL working with the Java connectors then that opens
> up the question why keep the APR/native connector. But we aren't there
> yet.
>
> > - Sendfile
> >
> > NIO:
> > - It's been around longer :)
> > - Sendfile
> >
> > NIO2:
> > - Modern async IO
> > - Scatter / gather IO that can be exposed and taken advantage (see the
> new
> > IO calls I added; implementing them with APR and NIO is going to be a
> whole
> > lot more convoluted ...)
> > - Probably HTTP/2 and Servlet.next will take advantage of it just like
> > websockets did
>
> WebSockets didn't take advantage of it scatter/gather. Neither did it
> take advantage of the async style of API.
>
Websockets could easily take advantage of it. For example the code in
WsRemoteEndpointImplServer and all its for loops over arrays of buffers to
write them looks rather close to a gather write to me and could probably
all be replaced with a single write.
> > I don't think NIO has gotten any better, it's still the most horrendous
> IO
> > API imaginable as far as I am concerned. Of course, you can use
> frameworks
> > and stuff but ... I agree to keep it as well, since as you say it's more
> > mature and stable, but that's about it.
>
> The main driver for this thinking is reducing complexity in the
> connectors. Having to support both poller style and async style basic
> I/O creates complexity. If we only supported one style the I/O code
> could be a lot cleaner.
>
Of course, things can always be cleaner, but this causes issues. For
example, the one I dislike the most is NIO, but as you said it is the most
stable, so removing it is probably not a good idea.
>
> Overall, I think I prefer the async style. It is much easier to simulate
> blocking and requires less supporting code (no pollers etc).
>
> But, APR uses the poller style and it has much better SSL performance.
>
> Thinking ahead (not sure how far, maybe Tomcat 9 at the outside) but if
> we had OpenSSL working with NIO.2 and performance was similar to the
> APR/native connector I'd be all for dropping NIO and APR/native in
> favour of NIO2 and with the option to use OpenSSL or JSSE for SSL.
>
I was planning to send an email about that on Monday, so I guess I'm
sending it now instead.
>
> Is that too extreme?
>
> The list of "pros" for each connector is not empty, so I don't see why it
helps to remove one.
Rémy
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Mark Thomas <ma...@apache.org>.
On 01/05/2015 15:03, Rémy Maucherat wrote:
> 2015-05-01 11:55 GMT+02:00 Mark Thomas <ma...@apache.org>:
>> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
>> yes, which one?
>>
>> I volunteer to help maintain NIO2 for the time being, so I am not in favor
> of removing it at the moment. Unfortunately, I think all three connectors
> have some good points.
>
> APR:
> - Blocking IO
> - OpenSSL (for the time being)
If you can get OpenSSL working with the Java connectors then that opens
up the question why keep the APR/native connector. But we aren't there yet.
> - Sendfile
>
> NIO:
> - It's been around longer :)
> - Sendfile
>
> NIO2:
> - Modern async IO
> - Scatter / gather IO that can be exposed and taken advantage (see the new
> IO calls I added; implementing them with APR and NIO is going to be a whole
> lot more convoluted ...)
> - Probably HTTP/2 and Servlet.next will take advantage of it just like
> websockets did
WebSockets didn't take advantage of it scatter/gather. Neither did it
take advantage of the async style of API.
> I don't think NIO has gotten any better, it's still the most horrendous IO
> API imaginable as far as I am concerned. Of course, you can use frameworks
> and stuff but ... I agree to keep it as well, since as you say it's more
> mature and stable, but that's about it.
The main driver for this thinking is reducing complexity in the
connectors. Having to support both poller style and async style basic
I/O creates complexity. If we only supported one style the I/O code
could be a lot cleaner.
Overall, I think I prefer the async style. It is much easier to simulate
blocking and requires less supporting code (no pollers etc).
But, APR uses the poller style and it has much better SSL performance.
Thinking ahead (not sure how far, maybe Tomcat 9 at the outside) but if
we had OpenSSL working with NIO.2 and performance was similar to the
APR/native connector I'd be all for dropping NIO and APR/native in
favour of NIO2 and with the option to use OpenSSL or JSSE for SSL.
Is that too extreme?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Posted by Rémy Maucherat <re...@apache.org>.
2015-05-01 11:55 GMT+02:00 Mark Thomas <ma...@apache.org>:
> Next up on my list is the NIO2 connector.
>
> NIO and APR/native both use a polling approach to non-blocking I/O. You
> add the socket to the poller, tell it what operation (read/write) you
> want to perform and then you wait for the poller to tell you the socket
> is ready to perform that operation.
>
> NIO2 uses an asynchronous approach to non-blocking I/O. You perform the
> read/write and then wait to be told it has finished via either a Future
> or a CompletionHandler.
>
> Servlet 3.1 non-blocking I/O is closest to the polling style (you get a
> callback when you are allowed to read/write).
>
> WebSocket non-blocking I/O uses the asynchronous style.
>
> In short, regardless of the underlying approach to non-blocking I/O, we
> have to support JavaEE APIs that use both styles. Therefore there is no
> 'obvious' advantage for either NIO or NIO2.
>
> As far as I can tell, the performance of NIO and NIO2 are comparable.
>
> That raises the question why do we need both NIO and NIO2? And I don't
> have an answer to that. If I had to pick one, I'd pick NIO because:
> - it has been around longer and is more stable
> - it uses the same style as APR/native which may allow further
> refactoring to reduce duplication.
>
> So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If
> yes, which one?
>
> I volunteer to help maintain NIO2 for the time being, so I am not in favor
of removing it at the moment. Unfortunately, I think all three connectors
have some good points.
APR:
- Blocking IO
- OpenSSL (for the time being)
- Sendfile
NIO:
- It's been around longer :)
- Sendfile
NIO2:
- Modern async IO
- Scatter / gather IO that can be exposed and taken advantage (see the new
IO calls I added; implementing them with APR and NIO is going to be a whole
lot more convoluted ...)
- Probably HTTP/2 and Servlet.next will take advantage of it just like
websockets did
I don't think NIO has gotten any better, it's still the most horrendous IO
API imaginable as far as I am concerned. Of course, you can use frameworks
and stuff but ... I agree to keep it as well, since as you say it's more
mature and stable, but that's about it.
Rémy