You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Thomas <ma...@apache.org> on 2013/01/15 14:29:29 UTC
APR, WebSocket and Servlet 3.1 NIO refactoring
Hi,
I think I am going to need to refactor the APR Poller code. Before I get
to far down that road I wanted to get a sanity check from those that
understand the APR/native connector better than I do.
The main problem:
- I need to be able to update the flags for a socket current in the
poller. For example, adding FLAGS_WRITE to a socket that currently has
FLAGS_READ
Side issues:
- APR connections don't track number of HTTP keep-alives left (it gets
reset every time the socket goes back to the Poller)
- Currently we have two completely separate sets of Pollers. Once for
Comet and one for non-Comet.
The reasoning for the refactoring:
1. Sockets should only be added to a pollset once.
2. Because of 1) to update the flags the socket needs to be removed from
the pollset and re-added with the new flags.
3. Currently, we have no record of which Poller a socket is registered
with. This makes 2) more expensive as we have to iterate through the
Pollers to find the socket. Tracking the current poller used by a socket
would make 2) a cheaper operation.
4. Implementing 3) requires maintaining a mapping from socket to
SocketWrapper (like KeyAttachment does in NIO)
5. A simple Map of Socket->SocketWrapper should give us what we need.
In addition to enabling me to continue with the Servlet 3.1 NIO work,
the other advantages to the proposed refactoring are:
- Will be able to eliminate the CometPollers since the SocketWrapper
knows if a request is Comet or not.
- Proper tracking of HTTP keep-alive requests left per connection.
If anyone can see a good reason not to start down this path, please
speak up.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
On 15/01/2013 21:26, Remy Maucherat wrote:
> On Tue, 2013-01-15 at 19:26 +0000, Mark Thomas wrote:
>>> I had to solve each of these issues in the web container of JBoss.
>>> Although this is not going to be quite 100% compatible with Tomcat, it
>>> is very similar.
>>>
>>> I recommend looking at the branch that is in AS 7 trunk:
>>> http://anonsvn.jboss.org/repos/jbossweb/branches/7.2.x/
>>
>> Thanks. The more I look into this the more strangeness I find. I'll keep
>> the above in mind if I need some pointers.
>
> Ok. With APR, dealing with the poller is not very forgiving, so I think
> you should use that, it is well tested too (plus the license is ASL
> now).
The non-forgiving nature of APR is why I prefer to make small changes
from the current Tomcat version rather than the huge change to the JBoss
one. It appears that they have diverged quite a lot.
As I said, I'll certainly use it as a guide for specific features if I
need some pointers.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Tue, 2013-01-15 at 19:26 +0000, Mark Thomas wrote:
> >I had to solve each of these issues in the web container of JBoss.
> >Although this is not going to be quite 100% compatible with Tomcat, it
> >is very similar.
> >
> >I recommend looking at the branch that is in AS 7 trunk:
> >http://anonsvn.jboss.org/repos/jbossweb/branches/7.2.x/
>
> Thanks. The more I look into this the more strangeness I find. I'll keep
> the above in mind if I need some pointers.
Ok. With APR, dealing with the poller is not very forgiving, so I think
you should use that, it is well tested too (plus the license is ASL
now).
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
Remy Maucherat <re...@apache.org> wrote:
>On Tue, 2013-01-15 at 13:29 +0000, Mark Thomas wrote:
>> Hi,
>>
>> I think I am going to need to refactor the APR Poller code. Before I
>get
>> to far down that road I wanted to get a sanity check from those that
>> understand the APR/native connector better than I do.
>>
>> The main problem:
>> - I need to be able to update the flags for a socket current in the
>> poller. For example, adding FLAGS_WRITE to a socket that currently
>has
>> FLAGS_READ
>>
>> Side issues:
>> - APR connections don't track number of HTTP keep-alives left (it
>gets
>> reset every time the socket goes back to the Poller)
>> - Currently we have two completely separate sets of Pollers. Once for
>> Comet and one for non-Comet.
>>
>> The reasoning for the refactoring:
>>
>> 1. Sockets should only be added to a pollset once.
>> 2. Because of 1) to update the flags the socket needs to be removed
>from
>> the pollset and re-added with the new flags.
>> 3. Currently, we have no record of which Poller a socket is
>registered
>> with. This makes 2) more expensive as we have to iterate through the
>> Pollers to find the socket. Tracking the current poller used by a
>socket
>> would make 2) a cheaper operation.
>> 4. Implementing 3) requires maintaining a mapping from socket to
>> SocketWrapper (like KeyAttachment does in NIO)
>> 5. A simple Map of Socket->SocketWrapper should give us what we need.
>
>I had to solve each of these issues in the web container of JBoss.
>Although this is not going to be quite 100% compatible with Tomcat, it
>is very similar.
>
>I recommend looking at the branch that is in AS 7 trunk:
>http://anonsvn.jboss.org/repos/jbossweb/branches/7.2.x/
Thanks. The more I look into this the more strangeness I find. I'll keep
the above in mind if I need some pointers.
>> In addition to enabling me to continue with the Servlet 3.1 NIO work,
>> the other advantages to the proposed refactoring are:
>> - Will be able to eliminate the CometPollers since the SocketWrapper
>> knows if a request is Comet or not.
>> - Proper tracking of HTTP keep-alive requests left per connection.
>>
>> If anyone can see a good reason not to start down this path, please
>> speak up.
>
>With APR, I would keep a separate poller for events (called comet in
>Tomcat) and keepalive, the code is rather different.
At the moment they are almost identical. I'll see how things develop.
Cheers,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Tue, 2013-01-15 at 13:29 +0000, Mark Thomas wrote:
> Hi,
>
> I think I am going to need to refactor the APR Poller code. Before I get
> to far down that road I wanted to get a sanity check from those that
> understand the APR/native connector better than I do.
>
> The main problem:
> - I need to be able to update the flags for a socket current in the
> poller. For example, adding FLAGS_WRITE to a socket that currently has
> FLAGS_READ
>
> Side issues:
> - APR connections don't track number of HTTP keep-alives left (it gets
> reset every time the socket goes back to the Poller)
> - Currently we have two completely separate sets of Pollers. Once for
> Comet and one for non-Comet.
>
> The reasoning for the refactoring:
>
> 1. Sockets should only be added to a pollset once.
> 2. Because of 1) to update the flags the socket needs to be removed from
> the pollset and re-added with the new flags.
> 3. Currently, we have no record of which Poller a socket is registered
> with. This makes 2) more expensive as we have to iterate through the
> Pollers to find the socket. Tracking the current poller used by a socket
> would make 2) a cheaper operation.
> 4. Implementing 3) requires maintaining a mapping from socket to
> SocketWrapper (like KeyAttachment does in NIO)
> 5. A simple Map of Socket->SocketWrapper should give us what we need.
I had to solve each of these issues in the web container of JBoss.
Although this is not going to be quite 100% compatible with Tomcat, it
is very similar.
I recommend looking at the branch that is in AS 7 trunk:
http://anonsvn.jboss.org/repos/jbossweb/branches/7.2.x/
> In addition to enabling me to continue with the Servlet 3.1 NIO work,
> the other advantages to the proposed refactoring are:
> - Will be able to eliminate the CometPollers since the SocketWrapper
> knows if a request is Comet or not.
> - Proper tracking of HTTP keep-alive requests left per connection.
>
> If anyone can see a good reason not to start down this path, please
> speak up.
With APR, I would keep a separate poller for events (called comet in
Tomcat) and keepalive, the code is rather different.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mladen Turk <mt...@apache.org>.
On 01/16/2013 01:05 PM, Remy Maucherat wrote:
> On Wed, 2013-01-16 at 11:43 +0000, Mark Thomas wrote:
>
> It is not really a bug: only the newer Windows (Vista+) have efficient
> polling. With previous versions, small pollsets have to be used because
> (apparently) it can only poll over 64 sockets at a time. If the size is
> over 1000, there is a big performance impact. It is supposed to be
> handled well in APR (using the new APIs if available).
>
> Source: Mladen some time ago. Hopefully, I understood everything
> correctly ;)
>
Yeah, correct.
Since Vista (2003R2) MS has added a WSAPoll to winsock API
which allows unlimited number of sockets (I tested that with 12K sockets)
Before that select call has to be used, which is limited by a
#define FD_SIZE 1024 at compile time.
Meaning < 1024 sockets per poller. Beyond that point it's just unusable.
Contemporary APR's have code which does automatic winsock API detection,
so 1000 sockets/pollset is not a limit if running on post XP boxes.
I think we even try to create 5000 socket pollset and if it fails
we create 5 pollsets with 1000 sockets. If support for XP is needed
I'm afraid we'll have to have that dual implementation.
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
On 16/01/2013 13:19, Remy Maucherat wrote:
> On Wed, 2013-01-16 at 12:08 +0000, Mark Thomas wrote:
>>> It is not really a bug: only the newer Windows (Vista+) have efficient
>>> polling. With previous versions, small pollsets have to be used because
>>> (apparently) it can only poll over 64 sockets at a time. If the size is
>>> over 1000, there is a big performance impact. It is supposed to be
>>> handled well in APR (using the new APIs if available).
>>>
>>> Source: Mladen some time ago. Hopefully, I understood everything
>>> correctly ;)
>>
>> It looks like lifting your new Poller implementation (and associated
>> supporting classes) will be the quickest way forward at this point. Am I
>> correct in assuming you are happy for me to do that?
>
> Yes, I think you should use that poller code (at least a little bit),
> it's rather tricky.
Thanks. Looking at it now.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Wed, 2013-01-16 at 12:08 +0000, Mark Thomas wrote:
> > It is not really a bug: only the newer Windows (Vista+) have efficient
> > polling. With previous versions, small pollsets have to be used because
> > (apparently) it can only poll over 64 sockets at a time. If the size is
> > over 1000, there is a big performance impact. It is supposed to be
> > handled well in APR (using the new APIs if available).
> >
> > Source: Mladen some time ago. Hopefully, I understood everything
> > correctly ;)
>
> It looks like lifting your new Poller implementation (and associated
> supporting classes) will be the quickest way forward at this point. Am I
> correct in assuming you are happy for me to do that?
Yes, I think you should use that poller code (at least a little bit),
it's rather tricky.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
On 16/01/2013 12:05, Remy Maucherat wrote:
> On Wed, 2013-01-16 at 11:43 +0000, Mark Thomas wrote:
>>> Using multiple pollsets is only needed on some platforms (Windows ...),
>>> and it is possible it is not supposed to be mandatory now (Windows
>>> should be able to handle large pollsets now). So if you accept to drop
>>> support for running production on older things (ex: XP), it can be
>>> simplified. To be confirmed though ;)
>>
>> Tempting...
>>
>> XP is still my primary development environment and only this week I
>> found the solution to the hardware compatibility issue that was stopping
>> me upgrading.
>>
>> If XP has an issue (which doesn't surprise me at all) then I suspect
>> Server 2003 will have the same problem. That is supported until (at
>> least) July 2015 so I think dropping support (as much as I like deleting
>> code) is a little premature.
>
> It is not really a bug: only the newer Windows (Vista+) have efficient
> polling. With previous versions, small pollsets have to be used because
> (apparently) it can only poll over 64 sockets at a time. If the size is
> over 1000, there is a big performance impact. It is supposed to be
> handled well in APR (using the new APIs if available).
>
> Source: Mladen some time ago. Hopefully, I understood everything
> correctly ;)
It looks like lifting your new Poller implementation (and associated
supporting classes) will be the quickest way forward at this point. Am I
correct in assuming you are happy for me to do that?
Cheers,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Wed, 2013-01-16 at 11:43 +0000, Mark Thomas wrote:
> > Using multiple pollsets is only needed on some platforms (Windows ...),
> > and it is possible it is not supposed to be mandatory now (Windows
> > should be able to handle large pollsets now). So if you accept to drop
> > support for running production on older things (ex: XP), it can be
> > simplified. To be confirmed though ;)
>
> Tempting...
>
> XP is still my primary development environment and only this week I
> found the solution to the hardware compatibility issue that was stopping
> me upgrading.
>
> If XP has an issue (which doesn't surprise me at all) then I suspect
> Server 2003 will have the same problem. That is supported until (at
> least) July 2015 so I think dropping support (as much as I like deleting
> code) is a little premature.
It is not really a bug: only the newer Windows (Vista+) have efficient
polling. With previous versions, small pollsets have to be used because
(apparently) it can only poll over 64 sockets at a time. If the size is
over 1000, there is a big performance impact. It is supposed to be
handled well in APR (using the new APIs if available).
Source: Mladen some time ago. Hopefully, I understood everything
correctly ;)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
On 16/01/2013 10:43, Remy Maucherat wrote:
> On Wed, 2013-01-16 at 09:35 +0000, Mark Thomas wrote:
>> That would be really nice. What I'd need is this:
>>
>> int removeFromPollset(long socket)
>>
>> where the return value is either -ve for an error code or +ve for the
>> current flags.
>>
>> Looking at the code, by the time I have finished the other changes the
>> change above would be a nice to have but not essential.
>>
>> One of the changes that will help (Remy already did this in his fork) is
>> to move from:
>> 1 pollset <-> 1 poller, 1 endpoint <-> multiple pollers
>> to:
>> 1 endpoint <-> 1 poller, multiple pollsets <-> 1 poller
>>
>> This is next on my todo list.
>
> Using multiple pollsets is only needed on some platforms (Windows ...),
> and it is possible it is not supposed to be mandatory now (Windows
> should be able to handle large pollsets now). So if you accept to drop
> support for running production on older things (ex: XP), it can be
> simplified. To be confirmed though ;)
Tempting...
XP is still my primary development environment and only this week I
found the solution to the hardware compatibility issue that was stopping
me upgrading.
If XP has an issue (which doesn't surprise me at all) then I suspect
Server 2003 will have the same problem. That is supported until (at
least) July 2015 so I think dropping support (as much as I like deleting
code) is a little premature.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Wed, 2013-01-16 at 09:35 +0000, Mark Thomas wrote:
> That would be really nice. What I'd need is this:
>
> int removeFromPollset(long socket)
>
> where the return value is either -ve for an error code or +ve for the
> current flags.
>
> Looking at the code, by the time I have finished the other changes the
> change above would be a nice to have but not essential.
>
> One of the changes that will help (Remy already did this in his fork) is
> to move from:
> 1 pollset <-> 1 poller, 1 endpoint <-> multiple pollers
> to:
> 1 endpoint <-> 1 poller, multiple pollsets <-> 1 poller
>
> This is next on my todo list.
Using multiple pollsets is only needed on some platforms (Windows ...),
and it is possible it is not supposed to be mandatory now (Windows
should be able to handle large pollsets now). So if you accept to drop
support for running production on older things (ex: XP), it can be
simplified. To be confirmed though ;)
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mark Thomas <ma...@apache.org>.
On 16/01/2013 06:50, Mladen Turk wrote:
> On 01/15/2013 02:29 PM, Mark Thomas wrote:
>> Hi,
>>
>> 3. Currently, we have no record of which Poller a socket is registered
>> with. This makes 2) more expensive as we have to iterate through the
>> Pollers to find the socket. Tracking the current poller used by a socket
>> would make 2) a cheaper operation.
>
> Think this can be done with relative ease.
> I recently made some changes to native pollset performance
> (removing multiple linear scans for sockets if they are in the pollset)
> which can reference the pollset with socket.
>
> In theory this allows to remove the socket from 'any' pollset which
> means that you can remove the socket from pollset without knowing to
> which one it belongs.
>
> If there is a need for reconfigure (different signaling events)
> we can add an extra API call that would allow that, without the need
> to know for which pollset this is.
That would be really nice. What I'd need is this:
int removeFromPollset(long socket)
where the return value is either -ve for an error code or +ve for the
current flags.
Looking at the code, by the time I have finished the other changes the
change above would be a nice to have but not essential.
One of the changes that will help (Remy already did this in his fork) is
to move from:
1 pollset <-> 1 poller, 1 endpoint <-> multiple pollers
to:
1 endpoint <-> 1 poller, multiple pollsets <-> 1 poller
This is next on my todo list.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Remy Maucherat <re...@apache.org>.
On Wed, 2013-01-16 at 07:50 +0100, Mladen Turk wrote:
> On 01/15/2013 02:29 PM, Mark Thomas wrote:
> > Hi,
> >
> > 3. Currently, we have no record of which Poller a socket is registered
> > with. This makes 2) more expensive as we have to iterate through the
> > Pollers to find the socket. Tracking the current poller used by a socket
> > would make 2) a cheaper operation.
>
> Think this can be done with relative ease.
> I recently made some changes to native pollset performance
> (removing multiple linear scans for sockets if they are in the pollset)
> which can reference the pollset with socket.
>
> In theory this allows to remove the socket from 'any' pollset which
> means that you can remove the socket from pollset without knowing to
> which one it belongs.
>
> If there is a need for reconfigure (different signaling events)
> we can add an extra API call that would allow that, without the need
> to know for which pollset this is.
Yes, remove [and add as well, but hopefully this was never a big issue]
performance is essential, because write events mean more events, and
flags also need to be updated. So a lot more operations overall.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: APR, WebSocket and Servlet 3.1 NIO refactoring
Posted by Mladen Turk <mt...@apache.org>.
On 01/15/2013 02:29 PM, Mark Thomas wrote:
> Hi,
>
> 3. Currently, we have no record of which Poller a socket is registered
> with. This makes 2) more expensive as we have to iterate through the
> Pollers to find the socket. Tracking the current poller used by a socket
> would make 2) a cheaper operation.
Think this can be done with relative ease.
I recently made some changes to native pollset performance
(removing multiple linear scans for sockets if they are in the pollset)
which can reference the pollset with socket.
In theory this allows to remove the socket from 'any' pollset which
means that you can remove the socket from pollset without knowing to
which one it belongs.
If there is a need for reconfigure (different signaling events)
we can add an extra API call that would allow that, without the need
to know for which pollset this is.
Regards
--
^TM
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org