[RFC] apr_pollcb api?

[Paul Querna <ch...@force-elite.com>]

I haven't been happy with our apr_pollset api for awhile.

Most of the frustration revolves around how the API requires that data 
is copied from the user, and we therefore have to manage it, keep track 
of it, and recycle it.

This means we added ring structures to track the data, and then we added 
optional mutexes to make it all thread safe.....

The ring structures also mean we must iterate them for _delete() and 
_poll(), wasting more cpu time.

So, I would like to propose a new poll API, I am initially calling 
'apr_pollcb'.  I am not stuck on the name, and better suggestions are 
welcome :)

Basic Concept:  You _add() a whole bunch of sockets, Calling _poll(), 
you pass a callback function.  This callback function is called with the 
same pollfd you used in _add(). (pollfd contains a user_data baton).

The pseudo API:

enum apr_pollcb_type_e {
#if HAS_KQUEUE
    APR_POLLCB_KQUEUE,
#endif
....
    APR_POLLCB_DEFAULT,
};

apr_pollcb_create(apr_pollcb_t **pollcb,
                   apr_pollcb_type_e ptype,
                   apr_pool_t* pool);


apr_pollcb_add(apr_pollcb_t *pollcb, apr_pollfd_t *descriptor);

apr_pollcb_remove(apr_pollcb_t *pollcb, apr_pollfd_t *descriptor);


typedef int(*apr_pollcb_cb_t)(void* baton, apr_pollfd_t *descriptor);

apr_pollcb_poll(apr_pollcb_t *pollcb,
                 apr_interval_time_t timeout,
                 apr_pollcb_cb_t* func,
                 void* baton);


Positives:
- O(1) Insertion & Deletion for epoll/kqueue/eventports
- O(n) iteration of results for epoll/kqueue/eventports
- No memory allocations inside the API. No copying of the user data.
- No lame limits on the number of sockets you can add or get data about 
in on call.
- If KQueue is broken on your platform, you can at runtime select poll() 
or select(). (I believe it would like similar to how process mutexes 
work underneath.)

Negatives:
- Users of the API must maintain the lifetime of the apr_pollfd_t structure.
- Poll/Select Implementations will likely need to keep extra state, and 
copy user data, and maybe even need mutexes to remain thread safe.

Questions:
- How could win32 fit into this type of API?  It would be awesomr if we 
could ditch select() on win32.

Thoughts?

-Paul

Re: [RFC] apr_pollcb api?

["Roy T. Fielding" <fi...@gbiv.com>]

On Mar 19, 2006, at 10:54 AM, Paul Querna wrote:

> Roy T. Fielding wrote:
>> Also, should there be different
>> callback functions for ready read, write, and error, or just one
>> function that is responsible for all three?
>
> But, what happens when there is a socket that is being monitored  
> for all 3, do you call all 3 callbacks?

I'd guess so, if all three are set on the return from poll.  I'm just  
curious what the best design would be.  My guess is that it will  
depend on why
the poll is being done.

> The apr_pollfd_t contains a bit field which we mark with the events  
> it was triggered for.  If the user of the API wants to do different  
> callbacks for each, they can build that on top of the single callback.

Hmm.  I was thinking in terms of async i/o applications, wherein the
caller is actually a multiplexer for many handlers and the ones looking
for read events (input filters) might not be the same as those looking
for write events (output filters).  They might not even be aware of
each other. But there may be a better way to do that.

Also, I have no idea what poll is supposed to do if the array
contains the same fd more than once, or even if it is worthwhile
supporting such a thing in callbacks.

....Roy

Re: [RFC] apr_pollcb api?

[Paul Querna <ch...@force-elite.com>]

Roy T. Fielding wrote:
> Sounds like a good idea to me.  If more than one fd is ready, what is
> the order that they are called?

The order that we get them back from the OS level function.

More or less 'no guarantees'.

> Also, should there be different
> callback functions for ready read, write, and error, or just one
> function that is responsible for all three?

But, what happens when there is a socket that is being monitored for all 
3, do you call all 3 callbacks?

The apr_pollfd_t contains a bit field which we mark with the events it 
was triggered for.  If the user of the API wants to do different 
callbacks for each, they can build that on top of the single callback.

-Paul

Re: [RFC] apr_pollcb api?

["Roy T. Fielding" <fi...@gbiv.com>]

Sounds like a good idea to me.  If more than one fd is ready, what is
the order that they are called?  Also, should there be different
callback functions for ready read, write, and error, or just one
function that is responsible for all three?

....Roy

Re: [RFC] apr_pollcb api?

[Justin Erenkrantz <ju...@erenkrantz.com>]

On 3/19/06, Paul Querna <ch...@force-elite.com> wrote:
> Basic Concept:  You _add() a whole bunch of sockets, Calling _poll(),
> you pass a callback function.  This callback function is called with the
> same pollfd you used in _add(). (pollfd contains a user_data baton).

Serf would be simplified somewhat with this, so +1.  -- justin

RE: [RFC] apr_pollcb api?

[James Mansion <ja...@wgold.demon.co.uk>]

>- How could win32 fit into this type of API?  It would be
> awesomr if we could ditch select() on win32.

This sort of reactive design doesn't fit well in Win32.
Win32 is 'completion' based.  You need to have a callback
structure that handles partial or full completions.  cf the
selector and proactor stuff in ACE.

You can implement the completion-based stuff with select etc,
but not the other way around.  Personally, I find the
completion view much more natural - I hardly ever generate
content because I have buffer space to write to, and buffers
are cheap enough to come by now to have some space allocated
to each socket, even if its 'only' a small buffer that will
handle a fixed message header, or is likely to tell me what
sort of HTTP request I'm going to deal with.