You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Alan Conway <ac...@redhat.com> on 2007/11/13 20:44:37 UTC
c++: getting rid of serializer & driving queue dispatch from the
writer threads.
Andrew, does the IO framework support non-IO wakeup of threads yet? I
think that's the key to eliminating the serializer and
getting message delivery into the thread pool. I'd like to be able to
mark a queue so that it can wake an IO thread when it's got messages.
I think we can drive message delivery from IO wakeups like this:
Producer ready-to-read wakeup: only goes as far as enqueueing the message.
Consumer thread ready-to-write wakeup: visits queues that it consumes
from and
- if it finds messages, write until writing would block, then go back
to sleep.
- if there's not enough data to block writing, mark myself as an
"unsatisfied consumer" on all queues of interest.
- Any queue with unsatisfied consumers enables IO wakeups.
Queue-has-messages wakeup:
- write messages to unsatisfied consumers, writing until write would block.
- remove unsatisfied consumer marker for all consumers that are blocked
in write, leave it for any that are not.
The idea is that every consumer is always in exactly one of these states:
- blocked on write to network.
- blocked waiting for message from a set of queues.
- being serviced by a thread (which could be triggered by
ready-to-write or a queue-has-messages, but not both)
Does that make sense?
Cheers,
Alan.
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Alan Conway <ac...@redhat.com>.
Gordon Sim wrote:
> Alan Conway wrote:
>> Andrew Stitcher wrote:
>>> On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
>>>> Consumer thread ready-to-write wakeup: visits queues that it
>>>> consumes from and
>>>> - if it finds messages, write until writing would block, then go
>>>> back to sleep.
>>>> - if there's not enough data to block writing, mark myself as an
>>>> "unsatisfied consumer" on all queues of interest. - Any queue
>>>> with unsatisfied consumers enables IO wakeups.
>>>
>>> Yes this would be a good way to make it work (I think this the general
>>> scheme that Gordon and I have been pushing for a while now). The IO
>>> framework already has a place to hook this into (via a yet-as-unwritten
>>> callback).
>>
>> Great. So do you or Gordon have plans to implement that in the short
>> term? If not I can look into it.
>
> Yes, this is (more or less) what I'm working on at present.
Excellent. I'll stay with the heap then.
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Gordon Sim <gs...@redhat.com>.
Alan Conway wrote:
> Andrew Stitcher wrote:
>> On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
>>> Consumer thread ready-to-write wakeup: visits queues that it consumes
>>> from and
>>> - if it finds messages, write until writing would block, then go
>>> back to sleep.
>>> - if there's not enough data to block writing, mark myself as an
>>> "unsatisfied consumer" on all queues of interest. - Any queue with
>>> unsatisfied consumers enables IO wakeups.
>>
>> Yes this would be a good way to make it work (I think this the general
>> scheme that Gordon and I have been pushing for a while now). The IO
>> framework already has a place to hook this into (via a yet-as-unwritten
>> callback).
>
> Great. So do you or Gordon have plans to implement that in the short
> term? If not I can look into it.
Yes, this is (more or less) what I'm working on at present.
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Alan Conway <ac...@redhat.com>.
Andrew Stitcher wrote:
> On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
>> Andrew, does the IO framework support non-IO wakeup of threads yet? I
>> think that's the key to eliminating the serializer and
>> getting message delivery into the thread pool. I'd like to be able to
>> mark a queue so that it can wake an IO thread when it's got messages.
>
> I don't think that anything you describe below needs non-IO wakeup of
> the threads! Although time based wakeup might turn out to be useful I
> have no plans to implement it the moment.
Not at all interested in time based wakeup. What I describe below is
wakeup based on presence of messages on a queue. How do we do that with
only IO based wakeup?
>> I think we can drive message delivery from IO wakeups like this:
>>
>> Producer ready-to-read wakeup: only goes as far as enqueueing the message.
>
> this is already how it works.
Currently the producer calls Queue::deliver which calls the Serializer.
I think we agree that we'd like to get rid of the serializer call at
this point.
>> Consumer thread ready-to-write wakeup: visits queues that it consumes
>> from and
>> - if it finds messages, write until writing would block, then go back
>> to sleep.
>> - if there's not enough data to block writing, mark myself as an
>> "unsatisfied consumer" on all queues of interest.
>> - Any queue with unsatisfied consumers enables IO wakeups.
>
> Yes this would be a good way to make it work (I think this the general
> scheme that Gordon and I have been pushing for a while now). The IO
> framework already has a place to hook this into (via a yet-as-unwritten
> callback).
Great. So do you or Gordon have plans to implement that in the short
term? If not I can look into it.
>> Queue-has-messages wakeup:
>> - write messages to unsatisfied consumers, writing until write would block.
>> - remove unsatisfied consumer marker for all consumers that are blocked
>> in write, leave it for any that are not.
>
> What this needs to do is poke the IO framework to tell it that we are
> now interested in writing on a specific connection.
Potentially a set of connections. There may be more than one consumer
interested in the queue.
so that if we
> weren't looking for writeability before we will now. This is already
> possible.
So are you saying that if a consumer is writable but there are no
messages to write it would mark itself "unsatisfied" on the queues it is
interested in and disable it's write trigger (or rather, not re-enable
its write trigger). When a queue has messages again it would enable the
write triggers of unsatified consumers causing their write threads to
wake up and contend for messages to write. That avoids the need for
non-IO wakeup. Is that what you mean? (presumably epoll is level
triggered for this to work)
>
>> The idea is that every consumer is always in exactly one of these states:
>> - blocked on write to network.
>> - blocked waiting for message from a set of queues.
>> - being serviced by a thread (which could be triggered by
>> ready-to-write or a queue-has-messages, but not both)
>>
>> Does that make sense?
>>
>> Cheers,
>> Alan.
>
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Alan Conway <ac...@redhat.com>.
Andrew Stitcher wrote:
[snip]
> So to recast these states:
>
> In queue:
>
> 1. No messages queued
> get callback informing connection writable:
> do nothing (there's nothing to write)
> goto state 1
> receive messages to queue:
> set connections to monitor writability
> goto state 2
>
> 2. Queued messages
> get callback informing connection writable:
> queue all messages for write
> goto state 1
>
> I think that's all you need here.
>
> Is what I'm talking about becoming clear?
Yup, we're in full agreement.
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Andrew Stitcher <as...@redhat.com>.
On Tue, 2007-11-13 at 17:08 -0500, Alan Conway wrote:
> Andrew Stitcher wrote:
> > On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
> >> Andrew, does the IO framework support non-IO wakeup of threads yet? I
> >> think that's the key to eliminating the serializer and
> >> getting message delivery into the thread pool. I'd like to be able to
> >> mark a queue so that it can wake an IO thread when it's got messages.
> >
> > I don't think that anything you describe below needs non-IO wakeup of
> > the threads! Although time based wakeup might turn out to be useful I
> > have no plans to implement it the moment.
> Not at all interested in time based wakeup. What I describe below is
> wakeup based on presence of messages on a queue. How do we do that with
> only IO based wakeup?
As I say below this equates to saying we would like to be notified of
writeability of all connections that are subscribed to the queue. So
that we can then get a callback in the context of the writing thread for
that connection.
As you infer below, if a connection is writeable but there is nothing
presently available to write there would have been a callback notifying
the writeability, if subsequently there was nothing queued for writing
then the connection would no longer be monitoring for writeability (to
do otherwise would impose a high overhead of useless callbacks). As soon
as there is something to write then writeability needs to be monitored
again.
>
> >> I think we can drive message delivery from IO wakeups like this:
> >>
> >> Producer ready-to-read wakeup: only goes as far as enqueueing the message.
> >
> > this is already how it works.
> Currently the producer calls Queue::deliver which calls the Serializer.
> I think we agree that we'd like to get rid of the serializer call at
> this point.
Sorry I was probably thinking more of the previous scheme.
>
> >> Consumer thread ready-to-write wakeup: visits queues that it consumes
> >> from and
> >> - if it finds messages, write until writing would block, then go back
> >> to sleep.
> >> - if there's not enough data to block writing, mark myself as an
> >> "unsatisfied consumer" on all queues of interest.
> >> - Any queue with unsatisfied consumers enables IO wakeups.
> >
> > Yes this would be a good way to make it work (I think this the general
> > scheme that Gordon and I have been pushing for a while now). The IO
> > framework already has a place to hook this into (via a yet-as-unwritten
> > callback).
>
> Great. So do you or Gordon have plans to implement that in the short
> term? If not I can look into it.
I'd like to do this when I'm done with the low level stuff, but that
doesn't seem very short term.
>
> >> Queue-has-messages wakeup:
> >> - write messages to unsatisfied consumers, writing until write would block.
> >> - remove unsatisfied consumer marker for all consumers that are blocked
> >> in write, leave it for any that are not.
> >
> > What this needs to do is poke the IO framework to tell it that we are
> > now interested in writing on a specific connection.
>
> Potentially a set of connections. There may be more than one consumer
> interested in the queue.
Ack
>
> so that if we
> > weren't looking for writeability before we will now. This is already
> > possible.
>
> So are you saying that if a consumer is writable but there are no
> messages to write it would mark itself "unsatisfied" on the queues it is
> interested in and disable it's write trigger (or rather, not re-enable
> its write trigger). When a queue has messages again it would enable the
> write triggers of unsatified consumers causing their write threads to
> wake up and contend for messages to write. That avoids the need for
> non-IO wakeup. Is that what you mean? (presumably epoll is level
> triggered for this to work)
There is no explicit "unsatisfied" mark, it just comes naturally from
informing the connections you have something to write to them, and
writing it in response to a "we're writable, but have nothing queued"
callback.
>
> >
> >> The idea is that every consumer is always in exactly one of these states:
> >> - blocked on write to network.
> >> - blocked waiting for message from a set of queues.
> >> - being serviced by a thread (which could be triggered by
> >> ready-to-write or a queue-has-messages, but not both)
Except that there's no "blocking" as such ("waiting" might be a better
word), and these states are summaries of implicit state in the queue and
asynch IO state of the connection.
Actually I'm not sure what you mean by "consumer" in this context. Is it
the connection, the queue or some combination of the two.
So to recast these states:
In queue:
1. No messages queued
get callback informing connection writable:
do nothing (there's nothing to write)
goto state 1
receive messages to queue:
set connections to monitor writability
goto state 2
2. Queued messages
get callback informing connection writable:
queue all messages for write
goto state 1
I think that's all you need here.
Is what I'm talking about becoming clear?
Andrew
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Alan Conway <ac...@redhat.com>.
Andrew Stitcher wrote:
> On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
>> Andrew, does the IO framework support non-IO wakeup of threads yet? I
>> think that's the key to eliminating the serializer and
>> getting message delivery into the thread pool. I'd like to be able to
>> mark a queue so that it can wake an IO thread when it's got messages.
>
> I don't think that anything you describe below needs non-IO wakeup of
> the threads! Although time based wakeup might turn out to be useful I
> have no plans to implement it the moment.
Not at all interested in time based wakeup. What I describe below is
wakeup based on presence of messages on a queue. How do we do that with
only IO based wakeup?
>> I think we can drive message delivery from IO wakeups like this:
>>
>> Producer ready-to-read wakeup: only goes as far as enqueueing the message.
>
> this is already how it works.
Currently the producer calls Queue::deliver which calls the Serializer.
I think we agree that we'd like to get rid of the serializer call at
this point.
>> Consumer thread ready-to-write wakeup: visits queues that it consumes
>> from and
>> - if it finds messages, write until writing would block, then go back
>> to sleep.
>> - if there's not enough data to block writing, mark myself as an
>> "unsatisfied consumer" on all queues of interest.
>> - Any queue with unsatisfied consumers enables IO wakeups.
>
> Yes this would be a good way to make it work (I think this the general
> scheme that Gordon and I have been pushing for a while now). The IO
> framework already has a place to hook this into (via a yet-as-unwritten
> callback).
Great. So do you or Gordon have plans to implement that in the short
term? If not I can look into it.
>> Queue-has-messages wakeup:
>> - write messages to unsatisfied consumers, writing until write would block.
>> - remove unsatisfied consumer marker for all consumers that are blocked
>> in write, leave it for any that are not.
>
> What this needs to do is poke the IO framework to tell it that we are
> now interested in writing on a specific connection.
Potentially a set of connections. There may be more than one consumer
interested in the queue.
so that if we
> weren't looking for writeability before we will now. This is already
> possible.
So are you saying that if a consumer is writable but there are no
messages to write it would mark itself "unsatisfied" on the queues it is
interested in and disable it's write trigger (or rather, not re-enable
its write trigger). When a queue has messages again it would enable the
write triggers of unsatified consumers causing their write threads to
wake up and contend for messages to write. That avoids the need for
non-IO wakeup. Is that what you mean? (presumably epoll is level
triggered for this to work)
>
>> The idea is that every consumer is always in exactly one of these states:
>> - blocked on write to network.
>> - blocked waiting for message from a set of queues.
>> - being serviced by a thread (which could be triggered by
>> ready-to-write or a queue-has-messages, but not both)
>>
>> Does that make sense?
>>
>> Cheers,
>> Alan.
>
Re: c++: getting rid of serializer & driving queue dispatch from
the writer threads.
Posted by Andrew Stitcher <as...@redhat.com>.
On Tue, 2007-11-13 at 14:44 -0500, Alan Conway wrote:
> Andrew, does the IO framework support non-IO wakeup of threads yet? I
> think that's the key to eliminating the serializer and
> getting message delivery into the thread pool. I'd like to be able to
> mark a queue so that it can wake an IO thread when it's got messages.
I don't think that anything you describe below needs non-IO wakeup of
the threads! Although time based wakeup might turn out to be useful I
have no plans to implement it the moment.
>
> I think we can drive message delivery from IO wakeups like this:
>
> Producer ready-to-read wakeup: only goes as far as enqueueing the message.
this is already how it works.
>
> Consumer thread ready-to-write wakeup: visits queues that it consumes
> from and
> - if it finds messages, write until writing would block, then go back
> to sleep.
> - if there's not enough data to block writing, mark myself as an
> "unsatisfied consumer" on all queues of interest.
> - Any queue with unsatisfied consumers enables IO wakeups.
Yes this would be a good way to make it work (I think this the general
scheme that Gordon and I have been pushing for a while now). The IO
framework already has a place to hook this into (via a yet-as-unwritten
callback).
>
> Queue-has-messages wakeup:
> - write messages to unsatisfied consumers, writing until write would block.
> - remove unsatisfied consumer marker for all consumers that are blocked
> in write, leave it for any that are not.
What this needs to do is poke the IO framework to tell it that we are
now interested in writing on a specific connection. so that if we
weren't looking for writeability before we will now. This is already
possible.
>
> The idea is that every consumer is always in exactly one of these states:
> - blocked on write to network.
> - blocked waiting for message from a set of queues.
> - being serviced by a thread (which could be triggered by
> ready-to-write or a queue-has-messages, but not both)
>
> Does that make sense?
>
> Cheers,
> Alan.