Multiple consumers on a distributed queue

[Sznajder ForMailingList <bs...@gmail.com>]

Hi

I used the implementation of Curator for the distributed queue recipe of
Zookeeper.

However, if see some strange behavior:

* The throughput of the consumption of the items drastically decreases when
I enhance the number of consumers?

* When I launch several consumers on several servers ( of a same Zookeeper
chorum, of course), I see that some of the consumers are in an idle state
most of the time, instead of consuming items....

Is it a known behavior? Can we solve this? In the meanwhile, it reduces the
motivation to use a distributed queue!! ...

Many thanks for your help!

Regards
Benjamin

Re: Multiple consumers on a distributed queue

[Sznajder ForMailingList <bs...@gmail.com>]

Hi,

Sorry, I missed your message and I realize, I put a very similar question
today.

I put message to the queue, when the consumer monitors and finds that queue
reached 1/2 of its size (I monitor that every second)

I attach here the simple code, I used for testing the behavior.
Thanks for your help




On Mon, Nov 18, 2013 at 7:45 PM, kishore g <g....@gmail.com> wrote:

> What is the rate at which you are adding messages to queue?
>
>
>
> On Sun, Nov 17, 2013 at 6:48 AM, Sznajder ForMailingList <
> bs4mailinglist@gmail.com> wrote:
>
> > Hi
> >
> > I used the implementation of Curator for the distributed queue recipe of
> > Zookeeper.
> >
> > However, if see some strange behavior:
> >
> > * The throughput of the consumption of the items drastically decreases
> when
> > I enhance the number of consumers?
> >
> > * When I launch several consumers on several servers ( of a same
> Zookeeper
> > chorum, of course), I see that some of the consumers are in an idle state
> > most of the time, instead of consuming items....
> >
> > Is it a known behavior? Can we solve this? In the meanwhile, it reduces
> the
> > motivation to use a distributed queue!! ...
> >
> > Many thanks for your help!
> >
> > Regards
> > Benjamin
> >
>

Re: Multiple consumers on a distributed queue

[kishore g <g....@gmail.com>]

What is the rate at which you are adding messages to queue?



On Sun, Nov 17, 2013 at 6:48 AM, Sznajder ForMailingList <
bs4mailinglist@gmail.com> wrote:

> Hi
>
> I used the implementation of Curator for the distributed queue recipe of
> Zookeeper.
>
> However, if see some strange behavior:
>
> * The throughput of the consumption of the items drastically decreases when
> I enhance the number of consumers?
>
> * When I launch several consumers on several servers ( of a same Zookeeper
> chorum, of course), I see that some of the consumers are in an idle state
> most of the time, instead of consuming items....
>
> Is it a known behavior? Can we solve this? In the meanwhile, it reduces the
> motivation to use a distributed queue!! ...
>
> Many thanks for your help!
>
> Regards
> Benjamin
>