You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Liviu I." <il...@gmail.com> on 2013/07/30 15:55:57 UTC

ActiveMQ ResourceAllocationException is delayed?

Hi guys,

I have a problem related to throttling in ActiveMQ. We are using the
ResourceAllocationException from ActiveMQ for throttling error handling.
Because we have encountered some problems related to throttling I am doing
some tests. 
The test scenario is relatively simple:
 - my application sends two messages in a queue
 - I have disabled the consumer for this test because I want the queue to be
filled up as I need it
 - in activemq.xml I have configured the queue like this:

<policyEntry queue="TEST_QUEUE" producerFlowControl="true" memoryLimit="1b"
useCache="false"/>

memoryLimit="1b" - because of this, one message will be allowed in the queue
and all other attempts to put messages will receive a
ResourceAllocationException (is this correct?)

When executing the test with two messages I expected to receive a
ResourceAllocationException when I tried to put the second message inside
the queue. But no exception came. At this point there were two messages
hanging in the queue. When executing the test again, both of the two new
messages received the exception. I have then purged the queue (0 messages in
the queue), retried the test and again no exception.

What I did after this is set up a loop that sent a message every second for
30 seconds (purged the queue before executing the test ofc). After 10
messages my application started receiving the ResourceAllocationException
that I have expected. (At this point there were 10 messages stuck in the
queue) 
It seemed that AMQ started sending the exception 10 seconds after receiving
the first one. I retried my first test (with the two messages and with the
queue empty) but with a delay. The test sent 1 message, waited for 11
seconds and then sent the second one. 
This time the test was successful. The second message received the expected
exception. After further testing it seems that there is a delay with which
AMQ sends the ResourceAllocationException, 10 seconds after the memoryLimit
for the queue is reached.

Why is this happening? Have I done something wrong or am I missing
something? Why is this apparent 10 seconds delay and how can I turn this
off? 

Thanks
L.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by "Liviu I." <il...@gmail.com>.
Thank you for the quick answer.

We are using ActiveMQ 5.8.0. Our application uses Spring(3.1.0) and Camel
(2.10.3).

I will look into this test and see if I can reproduce what I am seeing. I
believe that this is more like an integration problem, how our application
and Camel work with AMQ or how the systems are configured.

I will dig deeper and post my findings.

Thanks.
L.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871p4669905.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by Christian Posta <ch...@gmail.com>.
Post your test case and instructions to re-create.



On Wed, Sep 11, 2013 at 1:38 AM, Liviu I. <il...@gmail.com> wrote:

> Hello again and sorry for answering so late :)
>
> I was refreshing jconsole manually and it did not show any updates until
> around 10 seconds later (this seemed to vary a little bit).
>
> In the tests that we have done there were no consumers for the queue that
> we've tested. After activating a consumer on the queue, everything worked
> as
> expected. ActiveMQ was refreshing the MemoryUsageByteCount with every
> message that was dequeued (no more delays) and we had received the
> ResourceAllocationException.
>
> I don't know exactly why this happens. Could this be a bug?
>
> Thanks
> L.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871p4671400.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by "Liviu I." <il...@gmail.com>.
Hello again and sorry for answering so late :)

I was refreshing jconsole manually and it did not show any updates until
around 10 seconds later (this seemed to vary a little bit). 

In the tests that we have done there were no consumers for the queue that
we've tested. After activating a consumer on the queue, everything worked as
expected. ActiveMQ was refreshing the MemoryUsageByteCount with every
message that was dequeued (no more delays) and we had received the
ResourceAllocationException.

I don't know exactly why this happens. Could this be a bug? 

Thanks
L.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871p4671400.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by Christian Posta <ch...@gmail.com>.
Could it be that jconsole wasn't refreshing as often as you had expected?
ie, if you manually refresh, does it show the updated values? The memory
counters should not have a lag as they are updated as messages are coming
in and going out in real time.


On Wed, Jul 31, 2013 at 5:06 AM, Liviu I. <il...@gmail.com> wrote:

> I have tested this scenario again and I have noticed something strange.
>
> Every time I run an integration test I check the queue with jconsole. Among
> the attributes of the queue that I am able to see in jconsole are four that
> caught my attention: CursorMemoryUsage, CursorPercentUsage,
> MemoryPercentUsage and MemoryUsageByteCount.
>
> What I have noticed is that when the queue is purged the values of these
> attributes are set to 0. As long as the values are 0 AMQ does not send any
> ResourceAllocationException. These attributes are not updated with every
> message that enters the queue but after a certain period of time. After the
> attributes are updated AMQ starts sending the exception (as expected).
>
> It seems that AMQ compares the memoryLimit that I have setup
> (memoryLimit="1b") with one of these attributes (I think
> MemoryUsageByteCount) to see if it needs to throw the exception. This may
> very well explain the "delay" that I saw in my first tests.
>
> Does AMQ throw the ResourceAllocationException only when comparing one of
> these attributes (MemoryUsageByteCount?) with the memoryLimit configured
> for
> the queue or is this just a coincidence? And if so why are they not updated
> with every incoming message or at a faster pace?
>
> L.
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871p4669912.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by "Liviu I." <il...@gmail.com>.
I have tested this scenario again and I have noticed something strange.

Every time I run an integration test I check the queue with jconsole. Among
the attributes of the queue that I am able to see in jconsole are four that
caught my attention: CursorMemoryUsage, CursorPercentUsage,
MemoryPercentUsage and MemoryUsageByteCount.

What I have noticed is that when the queue is purged the values of these
attributes are set to 0. As long as the values are 0 AMQ does not send any
ResourceAllocationException. These attributes are not updated with every
message that enters the queue but after a certain period of time. After the
attributes are updated AMQ starts sending the exception (as expected). 

It seems that AMQ compares the memoryLimit that I have setup
(memoryLimit="1b") with one of these attributes (I think
MemoryUsageByteCount) to see if it needs to throw the exception. This may
very well explain the "delay" that I saw in my first tests.

Does AMQ throw the ResourceAllocationException only when comparing one of
these attributes (MemoryUsageByteCount?) with the memoryLimit configured for
the queue or is this just a coincidence? And if so why are they not updated
with every incoming message or at a faster pace?

L.





--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871p4669912.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ ResourceAllocationException is delayed?

Posted by Christian Posta <ch...@gmail.com>.
What version are you using?

Your best bet is to write a test that demonstrates this. Then we can look
at it closer.
There is an existing test in the activemq code base that you could use to
start with. I've tried recreating your scenario, but I couldn't reproduce
what you're seeing. When the cache is full (or off in your case), the
messages will be sent directly to the store, and no memory will be used.

Start with this test to recreate your scenario and i can help further:

https://github.com/apache/activemq/blob/trunk/activemq-unit-tests/src/test/java/org/apache/activemq/ProducerFlowControlSendFailTest.java#L119




On Tue, Jul 30, 2013 at 6:55 AM, Liviu I. <il...@gmail.com> wrote:

> Hi guys,
>
> I have a problem related to throttling in ActiveMQ. We are using the
> ResourceAllocationException from ActiveMQ for throttling error handling.
> Because we have encountered some problems related to throttling I am doing
> some tests.
> The test scenario is relatively simple:
>  - my application sends two messages in a queue
>  - I have disabled the consumer for this test because I want the queue to
> be
> filled up as I need it
>  - in activemq.xml I have configured the queue like this:
>
> <policyEntry queue="TEST_QUEUE" producerFlowControl="true" memoryLimit="1b"
> useCache="false"/>
>
> memoryLimit="1b" - because of this, one message will be allowed in the
> queue
> and all other attempts to put messages will receive a
> ResourceAllocationException (is this correct?)
>
> When executing the test with two messages I expected to receive a
> ResourceAllocationException when I tried to put the second message inside
> the queue. But no exception came. At this point there were two messages
> hanging in the queue. When executing the test again, both of the two new
> messages received the exception. I have then purged the queue (0 messages
> in
> the queue), retried the test and again no exception.
>
> What I did after this is set up a loop that sent a message every second for
> 30 seconds (purged the queue before executing the test ofc). After 10
> messages my application started receiving the ResourceAllocationException
> that I have expected. (At this point there were 10 messages stuck in the
> queue)
> It seemed that AMQ started sending the exception 10 seconds after receiving
> the first one. I retried my first test (with the two messages and with the
> queue empty) but with a delay. The test sent 1 message, waited for 11
> seconds and then sent the second one.
> This time the test was successful. The second message received the expected
> exception. After further testing it seems that there is a delay with which
> AMQ sends the ResourceAllocationException, 10 seconds after the memoryLimit
> for the queue is reached.
>
> Why is this happening? Have I done something wrong or am I missing
> something? Why is this apparent 10 seconds delay and how can I turn this
> off?
>
> Thanks
> L.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-ResourceAllocationException-is-delayed-tp4669871.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta