You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Zhihua Che <zh...@gmail.com> on 2012/07/12 04:36:01 UTC
what does mean 'Capacity'?
hi,
I got confused with the term of 'capacity' of the sender and receiver.
In my app, I create a queue and a sender and a receiver bound with it.
I also set both the capacity of the sender and receiver as 100.
Through qpid-tool, I find that, every time the receiver receives
one message from the queue, a field named 'acquires' of the queue is
increased by one.
When the 'acquires' number reaches the 100, my app cannot get
message from the queue any more.
So, what does the term 'capacity' and the field 'acquires' mean?
I noticed that, in qpid-tool, another field 'release' which is
actually above 'acquires' is always 0, is it right?
As for my code, I use c++ client and qpid::messaging api. I send
the message using sender.send(msg, true), and fetch message with
fetch(msg, Duration::IMMEDIATE), and acknowledge by call
session.acknowledge();
Do I miss any thing?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what does mean 'Capacity'?
Posted by Zhihua Che <zh...@gmail.com>.
2012/7/12 Gordon Sim <gs...@redhat.com>:
> On 07/12/2012 03:36 AM, Zhihua Che wrote:
>>
>> hi,
>>
>> I got confused with the term of 'capacity' of the sender and
>> receiver.
>
>
> Essentially 'capacity' restricts the number of 'buffered' messages. In the
> case of the sender, this is messages that have been sent but not yet
> confirmed by the broker. In the case of the receiver this is the number of
> messages the broker can pre-emptively send to the client - what is often
> known as 'prefetch', i.e. messages send by the broker before the application
> has actually fetched them, in anticipation of- and in order to speed up-
> subsequent fetches.
>
>
>> In my app, I create a queue and a sender and a receiver bound with
>> it.
>> I also set both the capacity of the sender and receiver as 100.
>> Through qpid-tool, I find that, every time the receiver receives
>> one message from the queue, a field named 'acquires' of the queue is
>> increased by one.
>
>
> Right, that indicates the broker allocating a message to the consumer.
>
>
>> When the 'acquires' number reaches the 100, my app cannot get
>> message from the queue any more.
>
>
> What do you mean here? Certainly that number should not go up further
> without the receiver calling fetch(), but the receiver should certainly be
> able to fetch() more messages. The capacity 'window' is moved as messages
> sent by the broker are handed off to the application in response to fetch()
> calls.
>
>
>> So, what does the term 'capacity' and the field 'acquires' mean?
>
>
> Hopefully the description of capacity above will make sense. The 'acquires'
> stat in management tracks the number of message 'allocations'. For competing
> consumers, each message is given to only one consumer - the consumer is
> considered to have 'acquired' the message.
>
>
>> I noticed that, in qpid-tool, another field 'release' which is
>> actually above 'acquires' is always 0, is it right?
>
>
> Yes, unless you either release messages explicitly (through
> Session::release()) or end a session without acknowledging messages, there
> will be no released messages.
>
> Releasing a message is a mechanism for allowing a message acquired by one
> consumer to be returned to the available state and re-acquired (potentially
> by another consumer).
>
>
>> As for my code, I use c++ client and qpid::messaging api. I send
>> the message using sender.send(msg, true), and fetch message with
>> fetch(msg, Duration::IMMEDIATE), and acknowledge by call
>> session.acknowledge();
>
>
> If sending synchronously (i.e. specifying true for the second parameter to
> send()), the senders capacity is of no real significance as you are blocking
> until that send has completed. (In effect the used capacity will never be
> greater than 1).
>
> If you are finding that you stop being able to fetch() messages even when
> there are some available on the queue, please provide a bit more detail on
> the receiving application, ideally a reproducer.
>
> Hope this helps...
>
>
Thanks for your detailed reply and it helps a lot.
But I still cannot understand the behavior of my app.
Precisely, there are two processes, a producer (sender) and a consumer
(receiver).
Before sending messing, sender.setCapacity(100) and
receiver.setCapacity(100) are called.
Then, sender.send(msg, true) is used to send message while
receiver.fetch(msg, Duration::IMMEDIATE) is used to fetch message.
The receiver also print the message content every time when it fetches
a message successfully.
The strange thing is the receiver cannot fetch any message from the
queue after it receives the 100 messages.
And the sender process still continues, that is, not blocked by the
call to send(msg, true).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what does mean 'Capacity'?
Posted by Gordon Sim <gs...@redhat.com>.
On 07/12/2012 01:41 PM, Zhihua Che wrote:
> 2012/7/12 Gordon Sim <gs...@redhat.com>:
>> On 07/12/2012 01:15 PM, Zhihua Che wrote:
>>>
>>> According what you said, I think the capacity of the receiver is used
>>> to speed up fetching action by prefetching the message.
>>> It should not limit the number the message the receiver could fetch.
>>> Right?
>>
>>
>> Right.
>>
>> I've attached a simple example of what you described. When I run these, I
>> see the receiver getting more than 100 messages, provided I start the sender
>> first and allow the queue to build up a little before starting the receiver.
>> Do you see the same?
>>
>> Note that because you are using Duration::IMMEDIATE for the receiver, it
>> will return false if at any attempt to fetch there are no more messages.
>> Since the sender is publishing synchronously, message come in to the queue
>> much more slowly that they go out, so eventually it is likely that the
>> receiver doesn't find a message.
>>
>> One solution there is to use a timeout of say 1 second, that would mean the
>> fetch() call waist for up to a second for messages if there are none on the
>> queue at the time. Another option is to simply wait and retry if fetch()
>> returns false.
>>
>>
> Yes, I didn't mention that but I did add a quick sleep after everytime
> fetching the message failed.
> I read your code, it's the same as mine.
> I ran your code and yes, the receiver fetched more than 100 messages.
> It's wired...
>
> I also watched your 'my-queue' by qpid-tool and found that your
> 'release' reaches the same number as 'acquires'
> and your 'msgTotalDequeues' is 0.
> The stats are not like mine queue.
Sorry, thats just because I forgot to acknowledge() the messages.
Correction attached. It still works as expected, but now the messages
get dequeued.
Re: what does mean 'Capacity'?
Posted by Zhihua Che <zh...@gmail.com>.
2012/7/12 Gordon Sim <gs...@redhat.com>:
> On 07/12/2012 01:15 PM, Zhihua Che wrote:
>>
>> According what you said, I think the capacity of the receiver is used
>> to speed up fetching action by prefetching the message.
>> It should not limit the number the message the receiver could fetch.
>> Right?
>
>
> Right.
>
> I've attached a simple example of what you described. When I run these, I
> see the receiver getting more than 100 messages, provided I start the sender
> first and allow the queue to build up a little before starting the receiver.
> Do you see the same?
>
> Note that because you are using Duration::IMMEDIATE for the receiver, it
> will return false if at any attempt to fetch there are no more messages.
> Since the sender is publishing synchronously, message come in to the queue
> much more slowly that they go out, so eventually it is likely that the
> receiver doesn't find a message.
>
> One solution there is to use a timeout of say 1 second, that would mean the
> fetch() call waist for up to a second for messages if there are none on the
> queue at the time. Another option is to simply wait and retry if fetch()
> returns false.
>
>
Yes, I didn't mention that but I did add a quick sleep after everytime
fetching the message failed.
I read your code, it's the same as mine.
I ran your code and yes, the receiver fetched more than 100 messages.
It's wired...
I also watched your 'my-queue' by qpid-tool and found that your
'release' reaches the same number as 'acquires'
and your 'msgTotalDequeues' is 0.
The stats are not like mine queue.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what does mean 'Capacity'?
Posted by Gordon Sim <gs...@redhat.com>.
On 07/12/2012 01:15 PM, Zhihua Che wrote:
> According what you said, I think the capacity of the receiver is used
> to speed up fetching action by prefetching the message.
> It should not limit the number the message the receiver could fetch. Right?
Right.
I've attached a simple example of what you described. When I run these,
I see the receiver getting more than 100 messages, provided I start the
sender first and allow the queue to build up a little before starting
the receiver. Do you see the same?
Note that because you are using Duration::IMMEDIATE for the receiver, it
will return false if at any attempt to fetch there are no more messages.
Since the sender is publishing synchronously, message come in to the
queue much more slowly that they go out, so eventually it is likely that
the receiver doesn't find a message.
One solution there is to use a timeout of say 1 second, that would mean
the fetch() call waist for up to a second for messages if there are none
on the queue at the time. Another option is to simply wait and retry if
fetch() returns false.
Re: what does mean 'Capacity'?
Posted by Zhihua Che <zh...@gmail.com>.
2012/7/12 Gordon Sim <gs...@redhat.com>:
> On 07/12/2012 03:36 AM, Zhihua Che wrote:
>>
>> hi,
>>
>> I got confused with the term of 'capacity' of the sender and
>> receiver.
>
>
> Essentially 'capacity' restricts the number of 'buffered' messages. In the
> case of the sender, this is messages that have been sent but not yet
> confirmed by the broker. In the case of the receiver this is the number of
> messages the broker can pre-emptively send to the client - what is often
> known as 'prefetch', i.e. messages send by the broker before the application
> has actually fetched them, in anticipation of- and in order to speed up-
> subsequent fetches.
>
>
>> In my app, I create a queue and a sender and a receiver bound with
>> it.
>> I also set both the capacity of the sender and receiver as 100.
>> Through qpid-tool, I find that, every time the receiver receives
>> one message from the queue, a field named 'acquires' of the queue is
>> increased by one.
>
>
> Right, that indicates the broker allocating a message to the consumer.
>
>
>> When the 'acquires' number reaches the 100, my app cannot get
>> message from the queue any more.
>
>
> What do you mean here? Certainly that number should not go up further
> without the receiver calling fetch(), but the receiver should certainly be
> able to fetch() more messages. The capacity 'window' is moved as messages
> sent by the broker are handed off to the application in response to fetch()
> calls.
>
>
>> So, what does the term 'capacity' and the field 'acquires' mean?
>
>
> Hopefully the description of capacity above will make sense. The 'acquires'
> stat in management tracks the number of message 'allocations'. For competing
> consumers, each message is given to only one consumer - the consumer is
> considered to have 'acquired' the message.
>
>
>> I noticed that, in qpid-tool, another field 'release' which is
>> actually above 'acquires' is always 0, is it right?
>
>
> Yes, unless you either release messages explicitly (through
> Session::release()) or end a session without acknowledging messages, there
> will be no released messages.
>
> Releasing a message is a mechanism for allowing a message acquired by one
> consumer to be returned to the available state and re-acquired (potentially
> by another consumer).
>
>
>> As for my code, I use c++ client and qpid::messaging api. I send
>> the message using sender.send(msg, true), and fetch message with
>> fetch(msg, Duration::IMMEDIATE), and acknowledge by call
>> session.acknowledge();
>
>
> If sending synchronously (i.e. specifying true for the second parameter to
> send()), the senders capacity is of no real significance as you are blocking
> until that send has completed. (In effect the used capacity will never be
> greater than 1).
>
> If you are finding that you stop being able to fetch() messages even when
> there are some available on the queue, please provide a bit more detail on
> the receiving application, ideally a reproducer.
>
> Hope this helps...
>
>
According what you said, I think the capacity of the receiver is used
to speed up fetching action by prefetching the message.
It should not limit the number the message the receiver could fetch. Right?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what does mean 'Capacity'?
Posted by Zhihua Che <zh...@gmail.com>.
2012/7/12 Gordon Sim <gs...@redhat.com>:
> On 07/12/2012 03:36 AM, Zhihua Che wrote:
>>
>> hi,
>>
>> I got confused with the term of 'capacity' of the sender and
>> receiver.
>
>
> Essentially 'capacity' restricts the number of 'buffered' messages. In the
> case of the sender, this is messages that have been sent but not yet
> confirmed by the broker. In the case of the receiver this is the number of
> messages the broker can pre-emptively send to the client - what is often
> known as 'prefetch', i.e. messages send by the broker before the application
> has actually fetched them, in anticipation of- and in order to speed up-
> subsequent fetches.
>
>
>> In my app, I create a queue and a sender and a receiver bound with
>> it.
>> I also set both the capacity of the sender and receiver as 100.
>> Through qpid-tool, I find that, every time the receiver receives
>> one message from the queue, a field named 'acquires' of the queue is
>> increased by one.
>
>
> Right, that indicates the broker allocating a message to the consumer.
>
>
>> When the 'acquires' number reaches the 100, my app cannot get
>> message from the queue any more.
>
>
> What do you mean here? Certainly that number should not go up further
> without the receiver calling fetch(), but the receiver should certainly be
> able to fetch() more messages. The capacity 'window' is moved as messages
> sent by the broker are handed off to the application in response to fetch()
> calls.
>
I checked it again.
Here are two process, a sender and a receiver.
This time, I just set the capacity of the receiver by calling
receiver.setCapacity(100)
Then, the sender started sending message and receiver started fetching message.
I watched the queue by qpid-tool and found that.
After the 'acquires' reaches the 100 and stop increasing, the
'msgTotalDequeues' also reaches 100 and stop increasing.
In the meantime, the 'msgTotalEnqueues' still increases.
>> So, what does the term 'capacity' and the field 'acquires' mean?
>
>
> Hopefully the description of capacity above will make sense. The 'acquires'
> stat in management tracks the number of message 'allocations'. For competing
> consumers, each message is given to only one consumer - the consumer is
> considered to have 'acquired' the message.
>
>
>> I noticed that, in qpid-tool, another field 'release' which is
>> actually above 'acquires' is always 0, is it right?
>
>
> Yes, unless you either release messages explicitly (through
> Session::release()) or end a session without acknowledging messages, there
> will be no released messages.
>
> Releasing a message is a mechanism for allowing a message acquired by one
> consumer to be returned to the available state and re-acquired (potentially
> by another consumer).
>
>
>> As for my code, I use c++ client and qpid::messaging api. I send
>> the message using sender.send(msg, true), and fetch message with
>> fetch(msg, Duration::IMMEDIATE), and acknowledge by call
>> session.acknowledge();
>
>
> If sending synchronously (i.e. specifying true for the second parameter to
> send()), the senders capacity is of no real significance as you are blocking
> until that send has completed. (In effect the used capacity will never be
> greater than 1).
>
> If you are finding that you stop being able to fetch() messages even when
> there are some available on the queue, please provide a bit more detail on
> the receiving application, ideally a reproducer.
>
> Hope this helps...
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what does mean 'Capacity'?
Posted by Gordon Sim <gs...@redhat.com>.
On 07/12/2012 03:36 AM, Zhihua Che wrote:
> hi,
>
> I got confused with the term of 'capacity' of the sender and receiver.
Essentially 'capacity' restricts the number of 'buffered' messages. In
the case of the sender, this is messages that have been sent but not yet
confirmed by the broker. In the case of the receiver this is the number
of messages the broker can pre-emptively send to the client - what is
often known as 'prefetch', i.e. messages send by the broker before the
application has actually fetched them, in anticipation of- and in order
to speed up- subsequent fetches.
> In my app, I create a queue and a sender and a receiver bound with it.
> I also set both the capacity of the sender and receiver as 100.
> Through qpid-tool, I find that, every time the receiver receives
> one message from the queue, a field named 'acquires' of the queue is
> increased by one.
Right, that indicates the broker allocating a message to the consumer.
> When the 'acquires' number reaches the 100, my app cannot get
> message from the queue any more.
What do you mean here? Certainly that number should not go up further
without the receiver calling fetch(), but the receiver should certainly
be able to fetch() more messages. The capacity 'window' is moved as
messages sent by the broker are handed off to the application in
response to fetch() calls.
> So, what does the term 'capacity' and the field 'acquires' mean?
Hopefully the description of capacity above will make sense. The
'acquires' stat in management tracks the number of message
'allocations'. For competing consumers, each message is given to only
one consumer - the consumer is considered to have 'acquired' the message.
> I noticed that, in qpid-tool, another field 'release' which is
> actually above 'acquires' is always 0, is it right?
Yes, unless you either release messages explicitly (through
Session::release()) or end a session without acknowledging messages,
there will be no released messages.
Releasing a message is a mechanism for allowing a message acquired by
one consumer to be returned to the available state and re-acquired
(potentially by another consumer).
> As for my code, I use c++ client and qpid::messaging api. I send
> the message using sender.send(msg, true), and fetch message with
> fetch(msg, Duration::IMMEDIATE), and acknowledge by call
> session.acknowledge();
If sending synchronously (i.e. specifying true for the second parameter
to send()), the senders capacity is of no real significance as you are
blocking until that send has completed. (In effect the used capacity
will never be greater than 1).
If you are finding that you stop being able to fetch() messages even
when there are some available on the queue, please provide a bit more
detail on the receiving application, ideally a reproducer.
Hope this helps...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org