You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Helen Kwong <he...@gmail.com> on 2015/01/06 02:12:42 UTC
Re: Single consumer across multiple queues
Hi Rob,
I finally got back to testing multiple queues on a consumer again, using
the changes you added to help with fairness. My broker jvm is running with
-Dqueue.maxAsynchronousDeliveries=1 for the number of messages delivered
per time slice by a queue. It has made things more fair than before, though
still significantly less so than before with single-queue consumers. Two
fairness tests that we ran:
A) In the test I described before, where a single session listens to 2
queues starting with 100 messages each, it is now indeed more fair.
Sometimes there are 3 messages in a row from 1 queue, but the order is
mostly alternating. When the first queue is drained, on average the other /
slower queue still has 6-7 messages left. However, this effect can
accumulate and the number at the end can vary a lot -- I've seen up to 26,
which seems quite high compared to 100. Also ran this starting with 1000
messages per queue: on average the slower queue had about 22 messages left
at the end, and I've seen up to 65 messages left.
B) We also ran tests with multiple connections / consumers listening to the
same set of queues. More specifically, we have 10 connections, each with 1
consumer listening to 100 queues, each queue starting with 500 messages. We
look at the order in which messages are processed, and track what the
largest difference in the number of remaining messages between any 2 queues
at any given point is. Before, in the analogous test where each connection
has 100 single-queue consumers for the 100 queues, the largest difference
at any point is around 20. Now with multi-queue consumers, it's much
higher, around 140-170.
Our questions / concerns:
1. One thing I'm not sure about is whether it's possible that 1 queue can
be favored over another queue. With this 1 consumer / 2 queues test, I use
the same 2 queues for every test run, one called BAREBONEQ0 and the other
BAREBONEQ1. What I saw intially when running the test was that BAREBONEQ0
was almost always the slower queue, over maybe 20 or so test runs. But
later I ran more test runs, and I don't see a pattern to who finishes last
anymore. It could be just by chance that I saw this in the beginning, but
just to be sure I'm not missing anything, can one queue end up being
favored over another at all?
2. With the second test, I'd often see a chunk of messages from the same
queue given to different consumers around the same time. E.g., might see 10
messages from one queue being processed consecutively, by the 10 different
consumers. 5 to 7 messages in a row from the same queue is common. Why does
this happen even though I have maxAsynchronousDeliveries configured to 1?
You mentioned a queue pushes messages to a consumer when it's notified that
the consumer has room for messages -- what does this mean in terms of
multiple consumers/connections, and does this mean whenever a queue gets
notified, it may push 1 message to each consumer?
3. Overall does the behavior from these tests seem expected to you? Is this
as fair as we can be with multi-queue consumers, before the new threading
model allows consumers to pull from queues?
Thanks a lot!
Helen
On Fri, Oct 31, 2014 at 11:18 AM, Helen Kwong <he...@gmail.com> wrote:
> Thanks a lot for the changes and the explanation! Will try this workaround
> for now. When the new queue threading model is available please let me know.
>
> On Fri, Oct 31, 2014 at 6:38 AM, Rob Godfrey <ro...@gmail.com>
> wrote:
>
>> Hi Helen,
>>
>> so the fundamental issue here is that currently inside the broker queues
>> "push" messages to consumers rather than consumers pulling from queues.
>> When a queue is informed that a consumer has room to accept more messages,
>> it immediately tries to start pump messages to that consumer. In this
>> case
>> there are two queues trying to pump messages to one consumer, and the
>> first
>> one to get notified will pump in up to 80 messages before yielding the
>> thread. At this point the second queue might be able to jump in and pump
>> 80 messages, or the first queue may actually get the lock again.
>>
>> Keith and I are planning on reworking the underlying queue threading model
>> soon to change this around so that the consumers/connections pull from the
>> queues, at which point real fairness will be easier to implement. In the
>> meantime I've made a couple of changes (in QPID-6204, revision
>> https://svn.apache.org/r1635768) which help by a) allowing the number of
>> messages delivered in one "time-slice" to be configured on a per queue
>> basis (i.e. removing the hardcoding of 80) and b) alternating which of the
>> queues is notified first when the consumer has available credit.
>>
>> After this change, by setting the time slice to one delivery
>> (-Dqueue.maxAsynchronousDeliveries=1) I saw reasonably fair behaviour
>> (runs
>> of no more the 3 messages for the same queue). Note that reducing the
>> timeslice probably has some negative performance impact, but you can
>> configure this value on a per queue basis (rather than setting it as a
>> system property you can set on each individual queue as a context
>> variable).
>>
>> When Keith gets done with what he's currently working on we'll try to
>> update you on our work on changing the queue threading model around.
>>
>> Cheers,
>> Rob
>>
>>
>> On 31 October 2014 01:19, Helen Kwong <he...@gmail.com> wrote:
>>
>> > Hi Rob,
>> >
>> > I got around to doing some testing on the multi-queue consumer feature
>> you
>> > added. So far things have looked good mostly, but there is one issue
>> I've
>> > run into and would like your help on.
>> >
>> > When we had single-queue consumers, we had fair allocation behavior
>> across
>> > queues, in the sense that if I have 2 queues A and B, each with 100
>> > messages, and one JMS session having a listening consumer on queue A
>> and a
>> > listener on queue B, the message processing order will be round robin --
>> > i.e., M_A_1 (representing the first message on queue A), M_B_1, M_A_2,
>> > M_B_2, M_A_3, M_B_3, and so on. But now, if I run the same test with the
>> > session having a single multi-queue consumer on A and B instead, the
>> order
>> > is, roughly, first the 100 messages on A, followed by the messages on B
>> > (only a few B messages are processed before all A messages are done). I
>> > enqueue the messages in the round robin order. I've also tried this with
>> > synchronous receives from both queues instead of asynchronous listening,
>> > and I see similar behavior.
>> >
>> > Is there any way we can mimic the "fair" behavior of single-queue
>> consumers
>> > with multi-queue consumers?
>> >
>> > Thanks,
>> > Helen
>> >
>>
>
>
Re: Single consumer across multiple queues
Posted by Helen Kwong <he...@gmail.com>.
Hi Rob, not sure if you saw my email earlier; would appreciate your help on
this. Thank you!
On Mon, Jan 5, 2015 at 5:12 PM, Helen Kwong <he...@gmail.com> wrote:
> Hi Rob,
>
> I finally got back to testing multiple queues on a consumer again, using
> the changes you added to help with fairness. My broker jvm is running with
> -Dqueue.maxAsynchronousDeliveries=1 for the number of messages delivered
> per time slice by a queue. It has made things more fair than before, though
> still significantly less so than before with single-queue consumers. Two
> fairness tests that we ran:
>
> A) In the test I described before, where a single session listens to 2
> queues starting with 100 messages each, it is now indeed more fair.
> Sometimes there are 3 messages in a row from 1 queue, but the order is
> mostly alternating. When the first queue is drained, on average the other /
> slower queue still has 6-7 messages left. However, this effect can
> accumulate and the number at the end can vary a lot -- I've seen up to 26,
> which seems quite high compared to 100. Also ran this starting with 1000
> messages per queue: on average the slower queue had about 22 messages left
> at the end, and I've seen up to 65 messages left.
>
> B) We also ran tests with multiple connections / consumers listening to
> the same set of queues. More specifically, we have 10 connections, each
> with 1 consumer listening to 100 queues, each queue starting with 500
> messages. We look at the order in which messages are processed, and track
> what the largest difference in the number of remaining messages between any
> 2 queues at any given point is. Before, in the analogous test where each
> connection has 100 single-queue consumers for the 100 queues, the largest
> difference at any point is around 20. Now with multi-queue consumers, it's
> much higher, around 140-170.
>
> Our questions / concerns:
>
> 1. One thing I'm not sure about is whether it's possible that 1 queue can
> be favored over another queue. With this 1 consumer / 2 queues test, I use
> the same 2 queues for every test run, one called BAREBONEQ0 and the other
> BAREBONEQ1. What I saw intially when running the test was that BAREBONEQ0
> was almost always the slower queue, over maybe 20 or so test runs. But
> later I ran more test runs, and I don't see a pattern to who finishes last
> anymore. It could be just by chance that I saw this in the beginning, but
> just to be sure I'm not missing anything, can one queue end up being
> favored over another at all?
>
> 2. With the second test, I'd often see a chunk of messages from the same
> queue given to different consumers around the same time. E.g., might see 10
> messages from one queue being processed consecutively, by the 10 different
> consumers. 5 to 7 messages in a row from the same queue is common. Why does
> this happen even though I have maxAsynchronousDeliveries configured to 1?
> You mentioned a queue pushes messages to a consumer when it's notified that
> the consumer has room for messages -- what does this mean in terms of
> multiple consumers/connections, and does this mean whenever a queue gets
> notified, it may push 1 message to each consumer?
>
> 3. Overall does the behavior from these tests seem expected to you? Is
> this as fair as we can be with multi-queue consumers, before the new
> threading model allows consumers to pull from queues?
>
> Thanks a lot!
> Helen
>
> On Fri, Oct 31, 2014 at 11:18 AM, Helen Kwong <he...@gmail.com>
> wrote:
>
>> Thanks a lot for the changes and the explanation! Will try this
>> workaround for now. When the new queue threading model is available please
>> let me know.
>>
>> On Fri, Oct 31, 2014 at 6:38 AM, Rob Godfrey <ro...@gmail.com>
>> wrote:
>>
>>> Hi Helen,
>>>
>>> so the fundamental issue here is that currently inside the broker queues
>>> "push" messages to consumers rather than consumers pulling from queues.
>>> When a queue is informed that a consumer has room to accept more
>>> messages,
>>> it immediately tries to start pump messages to that consumer. In this
>>> case
>>> there are two queues trying to pump messages to one consumer, and the
>>> first
>>> one to get notified will pump in up to 80 messages before yielding the
>>> thread. At this point the second queue might be able to jump in and pump
>>> 80 messages, or the first queue may actually get the lock again.
>>>
>>> Keith and I are planning on reworking the underlying queue threading
>>> model
>>> soon to change this around so that the consumers/connections pull from
>>> the
>>> queues, at which point real fairness will be easier to implement. In the
>>> meantime I've made a couple of changes (in QPID-6204, revision
>>> https://svn.apache.org/r1635768) which help by a) allowing the number of
>>> messages delivered in one "time-slice" to be configured on a per queue
>>> basis (i.e. removing the hardcoding of 80) and b) alternating which of
>>> the
>>> queues is notified first when the consumer has available credit.
>>>
>>> After this change, by setting the time slice to one delivery
>>> (-Dqueue.maxAsynchronousDeliveries=1) I saw reasonably fair behaviour
>>> (runs
>>> of no more the 3 messages for the same queue). Note that reducing the
>>> timeslice probably has some negative performance impact, but you can
>>> configure this value on a per queue basis (rather than setting it as a
>>> system property you can set on each individual queue as a context
>>> variable).
>>>
>>> When Keith gets done with what he's currently working on we'll try to
>>> update you on our work on changing the queue threading model around.
>>>
>>> Cheers,
>>> Rob
>>>
>>>
>>> On 31 October 2014 01:19, Helen Kwong <he...@gmail.com> wrote:
>>>
>>> > Hi Rob,
>>> >
>>> > I got around to doing some testing on the multi-queue consumer feature
>>> you
>>> > added. So far things have looked good mostly, but there is one issue
>>> I've
>>> > run into and would like your help on.
>>> >
>>> > When we had single-queue consumers, we had fair allocation behavior
>>> across
>>> > queues, in the sense that if I have 2 queues A and B, each with 100
>>> > messages, and one JMS session having a listening consumer on queue A
>>> and a
>>> > listener on queue B, the message processing order will be round robin
>>> --
>>> > i.e., M_A_1 (representing the first message on queue A), M_B_1, M_A_2,
>>> > M_B_2, M_A_3, M_B_3, and so on. But now, if I run the same test with
>>> the
>>> > session having a single multi-queue consumer on A and B instead, the
>>> order
>>> > is, roughly, first the 100 messages on A, followed by the messages on B
>>> > (only a few B messages are processed before all A messages are done). I
>>> > enqueue the messages in the round robin order. I've also tried this
>>> with
>>> > synchronous receives from both queues instead of asynchronous
>>> listening,
>>> > and I see similar behavior.
>>> >
>>> > Is there any way we can mimic the "fair" behavior of single-queue
>>> consumers
>>> > with multi-queue consumers?
>>> >
>>> > Thanks,
>>> > Helen
>>> >
>>>
>>
>>
>
Re: Single consumer across multiple queues
Posted by Helen Kwong <he...@gmail.com>.
Hi Rob,
Thanks for looking at this. We'll probably continue using single-queue
consumers for now then, since the fair behavior is important to us, and try
using multiple-queue consumers again when the new threading model is
available.
Thanks,
Helen
On Tue, Jan 20, 2015 at 10:32 AM, Rob Godfrey <ro...@gmail.com>
wrote:
> Hi Helen,
>
> apologies for taking so long to respond
>
> On 6 January 2015 at 02:12, Helen Kwong <he...@gmail.com> wrote:
>
> > Hi Rob,
> >
> > I finally got back to testing multiple queues on a consumer again, using
> > the changes you added to help with fairness. My broker jvm is running
> with
> > -Dqueue.maxAsynchronousDeliveries=1 for the number of messages delivered
> > per time slice by a queue. It has made things more fair than before,
> though
> > still significantly less so than before with single-queue consumers. Two
> > fairness tests that we ran:
> >
> > A) In the test I described before, where a single session listens to 2
> > queues starting with 100 messages each, it is now indeed more fair.
> > Sometimes there are 3 messages in a row from 1 queue, but the order is
> > mostly alternating. When the first queue is drained, on average the
> other /
> > slower queue still has 6-7 messages left. However, this effect can
> > accumulate and the number at the end can vary a lot -- I've seen up to
> 26,
> > which seems quite high compared to 100. Also ran this starting with 1000
> > messages per queue: on average the slower queue had about 22 messages
> left
> > at the end, and I've seen up to 65 messages left.
> >
> > B) We also ran tests with multiple connections / consumers listening to
> the
> > same set of queues. More specifically, we have 10 connections, each with
> 1
> > consumer listening to 100 queues, each queue starting with 500 messages.
> We
> > look at the order in which messages are processed, and track what the
> > largest difference in the number of remaining messages between any 2
> queues
> > at any given point is. Before, in the analogous test where each
> connection
> > has 100 single-queue consumers for the 100 queues, the largest difference
> > at any point is around 20. Now with multi-queue consumers, it's much
> > higher, around 140-170.
> >
> > Our questions / concerns:
> >
> > 1. One thing I'm not sure about is whether it's possible that 1 queue can
> > be favored over another queue. With this 1 consumer / 2 queues test, I
> use
> > the same 2 queues for every test run, one called BAREBONEQ0 and the other
> > BAREBONEQ1. What I saw intially when running the test was that BAREBONEQ0
> > was almost always the slower queue, over maybe 20 or so test runs. But
> > later I ran more test runs, and I don't see a pattern to who finishes
> last
> > anymore. It could be just by chance that I saw this in the beginning, but
> > just to be sure I'm not missing anything, can one queue end up being
> > favored over another at all?
> >
>
> There shouldn't be any favouring - it will come down to thread scheduling
> in the JVM at this point. The queues are "notified" of consumer credit in
> a round robin order, with the "first" to be notified changed each time.
> The JVM or operating system may simply favour a thread that is already
> running.
>
>
> >
> > 2. With the second test, I'd often see a chunk of messages from the same
> > queue given to different consumers around the same time. E.g., might see
> 10
> > messages from one queue being processed consecutively, by the 10
> different
> > consumers. 5 to 7 messages in a row from the same queue is common. Why
> does
> > this happen even though I have maxAsynchronousDeliveries configured to 1?
> > You mentioned a queue pushes messages to a consumer when it's notified
> that
> > the consumer has room for messages -- what does this mean in terms of
> > multiple consumers/connections, and does this mean whenever a queue gets
> > notified, it may push 1 message to each consumer?
> >
> >
> So, when I ran a similar test on my laptop I saw no more than about 3 or 4
> in a row from the same queue (before I changed how the round-robin
> notification was done it was a lot worse)... again though this comes down
> to thread scheduling... I was running everything (client / broker) on my 4
> core laptop... I imagine using a different machine would potentially see
> different results.
>
>
> > 3. Overall does the behavior from these tests seem expected to you? Is
> this
> > as fair as we can be with multi-queue consumers, before the new threading
> > model allows consumers to pull from queues?
> >
> >
> I made it as fair as I could with the current threading model... Keith and
> I are reworking the IO / threading model at the moment though I still can't
> really give an ETA for when that work will be finished - certainly not for
> the release that will be cut shortly (15.02 or whatever we decide to call
> it), possibly for the release after that though,
>
> Cheers,
> Rob
>
>
> > Thanks a lot!
> > Helen
> >
> > On Fri, Oct 31, 2014 at 11:18 AM, Helen Kwong <he...@gmail.com>
> > wrote:
> >
> > > Thanks a lot for the changes and the explanation! Will try this
> > workaround
> > > for now. When the new queue threading model is available please let me
> > know.
> > >
> > > On Fri, Oct 31, 2014 at 6:38 AM, Rob Godfrey <ro...@gmail.com>
> > > wrote:
> > >
> > >> Hi Helen,
> > >>
> > >> so the fundamental issue here is that currently inside the broker
> queues
> > >> "push" messages to consumers rather than consumers pulling from
> queues.
> > >> When a queue is informed that a consumer has room to accept more
> > messages,
> > >> it immediately tries to start pump messages to that consumer. In this
> > >> case
> > >> there are two queues trying to pump messages to one consumer, and the
> > >> first
> > >> one to get notified will pump in up to 80 messages before yielding the
> > >> thread. At this point the second queue might be able to jump in and
> > pump
> > >> 80 messages, or the first queue may actually get the lock again.
> > >>
> > >> Keith and I are planning on reworking the underlying queue threading
> > model
> > >> soon to change this around so that the consumers/connections pull from
> > the
> > >> queues, at which point real fairness will be easier to implement. In
> > the
> > >> meantime I've made a couple of changes (in QPID-6204, revision
> > >> https://svn.apache.org/r1635768) which help by a) allowing the number
> > of
> > >> messages delivered in one "time-slice" to be configured on a per queue
> > >> basis (i.e. removing the hardcoding of 80) and b) alternating which of
> > the
> > >> queues is notified first when the consumer has available credit.
> > >>
> > >> After this change, by setting the time slice to one delivery
> > >> (-Dqueue.maxAsynchronousDeliveries=1) I saw reasonably fair behaviour
> > >> (runs
> > >> of no more the 3 messages for the same queue). Note that reducing the
> > >> timeslice probably has some negative performance impact, but you can
> > >> configure this value on a per queue basis (rather than setting it as a
> > >> system property you can set on each individual queue as a context
> > >> variable).
> > >>
> > >> When Keith gets done with what he's currently working on we'll try to
> > >> update you on our work on changing the queue threading model around.
> > >>
> > >> Cheers,
> > >> Rob
> > >>
> > >>
> > >> On 31 October 2014 01:19, Helen Kwong <he...@gmail.com> wrote:
> > >>
> > >> > Hi Rob,
> > >> >
> > >> > I got around to doing some testing on the multi-queue consumer
> feature
> > >> you
> > >> > added. So far things have looked good mostly, but there is one issue
> > >> I've
> > >> > run into and would like your help on.
> > >> >
> > >> > When we had single-queue consumers, we had fair allocation behavior
> > >> across
> > >> > queues, in the sense that if I have 2 queues A and B, each with 100
> > >> > messages, and one JMS session having a listening consumer on queue A
> > >> and a
> > >> > listener on queue B, the message processing order will be round
> robin
> > --
> > >> > i.e., M_A_1 (representing the first message on queue A), M_B_1,
> M_A_2,
> > >> > M_B_2, M_A_3, M_B_3, and so on. But now, if I run the same test with
> > the
> > >> > session having a single multi-queue consumer on A and B instead, the
> > >> order
> > >> > is, roughly, first the 100 messages on A, followed by the messages
> on
> > B
> > >> > (only a few B messages are processed before all A messages are
> done).
> > I
> > >> > enqueue the messages in the round robin order. I've also tried this
> > with
> > >> > synchronous receives from both queues instead of asynchronous
> > listening,
> > >> > and I see similar behavior.
> > >> >
> > >> > Is there any way we can mimic the "fair" behavior of single-queue
> > >> consumers
> > >> > with multi-queue consumers?
> > >> >
> > >> > Thanks,
> > >> > Helen
> > >> >
> > >>
> > >
> > >
> >
>
Re: Single consumer across multiple queues
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Helen,
apologies for taking so long to respond
On 6 January 2015 at 02:12, Helen Kwong <he...@gmail.com> wrote:
> Hi Rob,
>
> I finally got back to testing multiple queues on a consumer again, using
> the changes you added to help with fairness. My broker jvm is running with
> -Dqueue.maxAsynchronousDeliveries=1 for the number of messages delivered
> per time slice by a queue. It has made things more fair than before, though
> still significantly less so than before with single-queue consumers. Two
> fairness tests that we ran:
>
> A) In the test I described before, where a single session listens to 2
> queues starting with 100 messages each, it is now indeed more fair.
> Sometimes there are 3 messages in a row from 1 queue, but the order is
> mostly alternating. When the first queue is drained, on average the other /
> slower queue still has 6-7 messages left. However, this effect can
> accumulate and the number at the end can vary a lot -- I've seen up to 26,
> which seems quite high compared to 100. Also ran this starting with 1000
> messages per queue: on average the slower queue had about 22 messages left
> at the end, and I've seen up to 65 messages left.
>
> B) We also ran tests with multiple connections / consumers listening to the
> same set of queues. More specifically, we have 10 connections, each with 1
> consumer listening to 100 queues, each queue starting with 500 messages. We
> look at the order in which messages are processed, and track what the
> largest difference in the number of remaining messages between any 2 queues
> at any given point is. Before, in the analogous test where each connection
> has 100 single-queue consumers for the 100 queues, the largest difference
> at any point is around 20. Now with multi-queue consumers, it's much
> higher, around 140-170.
>
> Our questions / concerns:
>
> 1. One thing I'm not sure about is whether it's possible that 1 queue can
> be favored over another queue. With this 1 consumer / 2 queues test, I use
> the same 2 queues for every test run, one called BAREBONEQ0 and the other
> BAREBONEQ1. What I saw intially when running the test was that BAREBONEQ0
> was almost always the slower queue, over maybe 20 or so test runs. But
> later I ran more test runs, and I don't see a pattern to who finishes last
> anymore. It could be just by chance that I saw this in the beginning, but
> just to be sure I'm not missing anything, can one queue end up being
> favored over another at all?
>
There shouldn't be any favouring - it will come down to thread scheduling
in the JVM at this point. The queues are "notified" of consumer credit in
a round robin order, with the "first" to be notified changed each time.
The JVM or operating system may simply favour a thread that is already
running.
>
> 2. With the second test, I'd often see a chunk of messages from the same
> queue given to different consumers around the same time. E.g., might see 10
> messages from one queue being processed consecutively, by the 10 different
> consumers. 5 to 7 messages in a row from the same queue is common. Why does
> this happen even though I have maxAsynchronousDeliveries configured to 1?
> You mentioned a queue pushes messages to a consumer when it's notified that
> the consumer has room for messages -- what does this mean in terms of
> multiple consumers/connections, and does this mean whenever a queue gets
> notified, it may push 1 message to each consumer?
>
>
So, when I ran a similar test on my laptop I saw no more than about 3 or 4
in a row from the same queue (before I changed how the round-robin
notification was done it was a lot worse)... again though this comes down
to thread scheduling... I was running everything (client / broker) on my 4
core laptop... I imagine using a different machine would potentially see
different results.
> 3. Overall does the behavior from these tests seem expected to you? Is this
> as fair as we can be with multi-queue consumers, before the new threading
> model allows consumers to pull from queues?
>
>
I made it as fair as I could with the current threading model... Keith and
I are reworking the IO / threading model at the moment though I still can't
really give an ETA for when that work will be finished - certainly not for
the release that will be cut shortly (15.02 or whatever we decide to call
it), possibly for the release after that though,
Cheers,
Rob
> Thanks a lot!
> Helen
>
> On Fri, Oct 31, 2014 at 11:18 AM, Helen Kwong <he...@gmail.com>
> wrote:
>
> > Thanks a lot for the changes and the explanation! Will try this
> workaround
> > for now. When the new queue threading model is available please let me
> know.
> >
> > On Fri, Oct 31, 2014 at 6:38 AM, Rob Godfrey <ro...@gmail.com>
> > wrote:
> >
> >> Hi Helen,
> >>
> >> so the fundamental issue here is that currently inside the broker queues
> >> "push" messages to consumers rather than consumers pulling from queues.
> >> When a queue is informed that a consumer has room to accept more
> messages,
> >> it immediately tries to start pump messages to that consumer. In this
> >> case
> >> there are two queues trying to pump messages to one consumer, and the
> >> first
> >> one to get notified will pump in up to 80 messages before yielding the
> >> thread. At this point the second queue might be able to jump in and
> pump
> >> 80 messages, or the first queue may actually get the lock again.
> >>
> >> Keith and I are planning on reworking the underlying queue threading
> model
> >> soon to change this around so that the consumers/connections pull from
> the
> >> queues, at which point real fairness will be easier to implement. In
> the
> >> meantime I've made a couple of changes (in QPID-6204, revision
> >> https://svn.apache.org/r1635768) which help by a) allowing the number
> of
> >> messages delivered in one "time-slice" to be configured on a per queue
> >> basis (i.e. removing the hardcoding of 80) and b) alternating which of
> the
> >> queues is notified first when the consumer has available credit.
> >>
> >> After this change, by setting the time slice to one delivery
> >> (-Dqueue.maxAsynchronousDeliveries=1) I saw reasonably fair behaviour
> >> (runs
> >> of no more the 3 messages for the same queue). Note that reducing the
> >> timeslice probably has some negative performance impact, but you can
> >> configure this value on a per queue basis (rather than setting it as a
> >> system property you can set on each individual queue as a context
> >> variable).
> >>
> >> When Keith gets done with what he's currently working on we'll try to
> >> update you on our work on changing the queue threading model around.
> >>
> >> Cheers,
> >> Rob
> >>
> >>
> >> On 31 October 2014 01:19, Helen Kwong <he...@gmail.com> wrote:
> >>
> >> > Hi Rob,
> >> >
> >> > I got around to doing some testing on the multi-queue consumer feature
> >> you
> >> > added. So far things have looked good mostly, but there is one issue
> >> I've
> >> > run into and would like your help on.
> >> >
> >> > When we had single-queue consumers, we had fair allocation behavior
> >> across
> >> > queues, in the sense that if I have 2 queues A and B, each with 100
> >> > messages, and one JMS session having a listening consumer on queue A
> >> and a
> >> > listener on queue B, the message processing order will be round robin
> --
> >> > i.e., M_A_1 (representing the first message on queue A), M_B_1, M_A_2,
> >> > M_B_2, M_A_3, M_B_3, and so on. But now, if I run the same test with
> the
> >> > session having a single multi-queue consumer on A and B instead, the
> >> order
> >> > is, roughly, first the 100 messages on A, followed by the messages on
> B
> >> > (only a few B messages are processed before all A messages are done).
> I
> >> > enqueue the messages in the round robin order. I've also tried this
> with
> >> > synchronous receives from both queues instead of asynchronous
> listening,
> >> > and I see similar behavior.
> >> >
> >> > Is there any way we can mimic the "fair" behavior of single-queue
> >> consumers
> >> > with multi-queue consumers?
> >> >
> >> > Thanks,
> >> > Helen
> >> >
> >>
> >
> >
>