You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Keith Freeman <8f...@gmail.com> on 2013/10/26 23:38:50 UTC

capacity of seda/vm with multipleConsumers=true?

My understanding is that using multipleConsumers=true on a seda:/vm: 
endpoint makes it behave more-or-less like a JMS topic.  Does it then 
have a capacity at which producers will throw/block (e.g. if there are 
no consumers), or does it behave more like an LRU ring buffer, writing 
new values on top of oldest ones?  Or something else?

The component page doesn't give much detail about this case.


Re: capacity of seda/vm with multipleConsumers=true?

Posted by gquintana <ge...@gmail.com>.
The answer I is: before the first reader takes the message. And again it's
not related to multiple consumers.
Gerald



--
View this message in context: http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742323.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: capacity of seda/vm with multipleConsumers=true?

Posted by Claus Ibsen <cl...@gmail.com>.
I suggest to study the source code.


On Mon, Oct 28, 2013 at 3:32 PM, Keith Freeman <8f...@gmail.com> wrote:
> Ok, but I don't think that answers the question.  Say I have one route
> putting things in the queue, and 3 other (parallel) routes reading the
> messages.  All 3 get each message, but when does Camel remove the message
> from the queue?
>
>  * after at least 1 of the readers takes the message
>  * after all of the readers take the message
>  * something else, like a quorum of N/2 + 1?
>
> I think we need to know this to understand the semantics of seda:/vm: with
> multipleConsumers=true.  But I'm guessing that the answer is that Camel
> removes the message only after all readers have taken it, otherwise multiple
> readers wouldn't be guaranteed to get every message.
>
>
> On 10/28/2013 06:11 AM, gquintana wrote:
>>
>> As it's a LinkedBlockingQueue, the messages are read using
>> BlockingQueue#poll
>> method, then whatever the result of the processing may be, the messages
>> has
>> been picked.
>>
>> I think the multipleConsumers acts like the  multicast
>> <http://camel.apache.org/multicast.html>   processor, you could replace
>> SEDA+multipleConsumers by SEDA+multicast and have the same result, and you
>> could execute theses consumers in parallel (fork & join).
>>
>> If you have only one parallel consumer (concurrentConsumers=1) and it
>> stalls, then the queue could fill and the producer could block or not
>> depending on the blockWhenFull setting. This is not related to
>> multipleConsumers settings though.
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742296.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Re: capacity of seda/vm with multipleConsumers=true?

Posted by Keith Freeman <8f...@gmail.com>.
Ok, but I don't think that answers the question.  Say I have one route 
putting things in the queue, and 3 other (parallel) routes reading the 
messages.  All 3 get each message, but when does Camel remove the 
message from the queue?

  * after at least 1 of the readers takes the message
  * after all of the readers take the message
  * something else, like a quorum of N/2 + 1?

I think we need to know this to understand the semantics of seda:/vm: 
with multipleConsumers=true.  But I'm guessing that the answer is that 
Camel removes the message only after all readers have taken it, 
otherwise multiple readers wouldn't be guaranteed to get every message.

On 10/28/2013 06:11 AM, gquintana wrote:
> As it's a LinkedBlockingQueue, the messages are read using BlockingQueue#poll
> method, then whatever the result of the processing may be, the messages has
> been picked.
>
> I think the multipleConsumers acts like the  multicast
> <http://camel.apache.org/multicast.html>   processor, you could replace
> SEDA+multipleConsumers by SEDA+multicast and have the same result, and you
> could execute theses consumers in parallel (fork & join).
>
> If you have only one parallel consumer (concurrentConsumers=1) and it
> stalls, then the queue could fill and the producer could block or not
> depending on the blockWhenFull setting. This is not related to
> multipleConsumers settings though.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742296.html
> Sent from the Camel - Users mailing list archive at Nabble.com.


Re: capacity of seda/vm with multipleConsumers=true?

Posted by gquintana <ge...@gmail.com>.
As it's a LinkedBlockingQueue, the messages are read using BlockingQueue#poll
method, then whatever the result of the processing may be, the messages has
been picked. 

I think the multipleConsumers acts like the  multicast
<http://camel.apache.org/multicast.html>   processor, you could replace
SEDA+multipleConsumers by SEDA+multicast and have the same result, and you
could execute theses consumers in parallel (fork & join).

If you have only one parallel consumer (concurrentConsumers=1) and it
stalls, then the queue could fill and the producer could block or not
depending on the blockWhenFull setting. This is not related to
multipleConsumers settings though.



--
View this message in context: http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742296.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: capacity of seda/vm with multipleConsumers=true?

Posted by Keith Freeman <8f...@gmail.com>.
Thanks for responding, I had read that before, but I don't think it 
explains how or when a queue with multipleConsumers becomes full. 
Without reading the code, I would guess that probably a message is not 
removed from the queue until ALL consumers have read it (meaning that if 
any single consumer stalls, the queue will eventually fill and the 
producer will be blocked), but whether that's true or not is not 
indicated in the documentation.  And even knowing that it's a 
LinkedBlockingQueue doesn't help answer that question, since it depends 
entirely on how the queue is used by Camel.

So I think it would be useful to know the detailed semantics of SEDA 
queue when blockWhenFull is true and multipleConsumers is true, e.g. is 
my guess above correct?



On 10/27/2013 01:52 PM, gquintana wrote:
> Hi,
>
> Reading from  SEDA component documentation
> <http://camel.apache.org/seda.html#SEDA-Options>  , I can see:
> "blockWhenFull	2.9	false	 Whether a thread that sends messages to a full
> SEDA queue will block until the queue's capacity is no longer exhausted. By*
> default, an exception will be thrown stating that the queue is full. By
> enabling this option, the calling thread will instead block and wait until
> the message can be accepted.*"
>
> Internally, the Seda component is based on a  LinkedBlockingQueue
> <http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/LinkedBlockingQueue.html>
> , it's not a ring buffer.
>
> Gérald
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742280.html
> Sent from the Camel - Users mailing list archive at Nabble.com.


Re: capacity of seda/vm with multipleConsumers=true?

Posted by gquintana <ge...@gmail.com>.
Hi,

Reading from  SEDA component documentation
<http://camel.apache.org/seda.html#SEDA-Options>  , I can see:
"blockWhenFull	2.9	false	 Whether a thread that sends messages to a full
SEDA queue will block until the queue's capacity is no longer exhausted. By*
default, an exception will be thrown stating that the queue is full. By
enabling this option, the calling thread will instead block and wait until
the message can be accepted.*"

Internally, the Seda component is based on a  LinkedBlockingQueue
<http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/LinkedBlockingQueue.html> 
, it's not a ring buffer.

Gérald



--
View this message in context: http://camel.465427.n5.nabble.com/capacity-of-seda-vm-with-multipleConsumers-true-tp5742266p5742280.html
Sent from the Camel - Users mailing list archive at Nabble.com.