You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by lurga <az...@netease.com> on 2008/11/10 09:30:16 UTC
memoryUsage configuration
I have a question about activemq:
I tried setting the destination limit low and the system limit high, with
the following config
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" memoryLimit="1kb"/>
<policyEntry topic=">" memoryLimit="1kb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#derby-ds"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="10kb"/>
</memoryUsage>
</systemUsage>
</systemUsage>
When I'm trying to send 10 non-persistent messages with 1k for each message
to a queue named "TEST.FOO", my producer blocks after sending 2 messages.
If I set the system limit equal to destination limit, my producer won't
block and send 10 messages.
I was browsing the source code, it seems BaseDestination uses two
memoryUsage. When BaseDestination.memoryUsage reaches its limit, the queue
does not send an ack back to producer and producer blocks. When
BaseDestination.systemUsage.memoryUsage reaches its limit, FilePendingCursor
would flush the messages to disk and reduce memory usage. Is it right?
--
View this message in context: http://www.nabble.com/memoryUsage-configuration-tp20416088p20416088.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: memoryUsage configuration
Posted by Gary Tully <ga...@gmail.com>.
sorry, my reply was not that clear.
I agree that the use of two memory usage metrics seems a little odd.
In other words, it appears to be an anomaly indicative of a bug!
My suggestion is that you create a junit test case that shows the
'odd' behavior and attach it to a new jira issue. In addition, submit
your changed code as a patch to that jira. (Hence the link to the
contributing page)
With a valid use case and some further investigation it should be
possible to determine the intent and correctness of the code in
BaseDestination. If your patch fixes the problem and does not have any
undesirable side effects as determined by the junit test suite, we can
commit your changes.
2008/11/12 lurga <az...@netease.com>:
>
> Hi Gary,
> Think you for your reply,but I still have some doubt about the 'valid
> anomaly' you mentioned .
> I was puzzled with this question for some days, and I also attempted to
> resolve it by writing some temperory code .
> What the most I want to know is why the BaseDestination uses two memory
> usage.
> I am looking forword your advice. thanks.
>
>
> Gary Tully wrote:
>>
>> Hi Lurga,
>> this seems to be a valid anomaly.
>> Please check out the contributing[1] details to see the best way to
>> make your changes permanent.
>> [1] http://activemq.apache.org/contributing.html
>>
>> 2008/11/11 lurga <az...@netease.com>:
>>>
>>> Queue invokes memoryUsage.isFull() method to decide when to block
>>> producer
>>> sending message, and FilePendingCursor invokes hasSpace() method (uses
>>> systemUsage of the Queue) to decide when to flush messages to disk. I
>>> made a
>>> patch to avoid diverse results from these two different ways of
>>> judgement.
>>> Now it seems to work correct.
>>>
>>> Index:
>>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>>> ===================================================================
>>> ---
>>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>>> (revision 712969)
>>> +++
>>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>>> (working copy)
>>> @@ -141,6 +141,12 @@
>>> memoryUsage.setParent(systemUsage.getMemoryUsage());
>>> }
>>>
>>> + long memoryUsageHighWaterMark =
>>> + Math.min(memoryUsage.getLimit(),
>>> systemUsage.getMemoryUsage().getLimit())
>>> + * messages.getMemoryUsageHighWaterMark()
>>> + / systemUsage.getMemoryUsage().getLimit();
>>> + messages.setMemoryUsageHighWaterMark((int)
>>> memoryUsageHighWaterMark);
>>> +
>>> if (isOptimizedDispatch()) {
>>> this.taskRunner = taskFactory.createTaskRunner(this,
>>> "TempQueue: " + destination.getPhysicalName());
>>> }else {
>>> --
>>> View this message in context:
>>> http://www.nabble.com/memoryUsage-configuration-tp20416088p20435735.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/memoryUsage-configuration-tp20416088p20453099.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: memoryUsage configuration
Posted by lurga <az...@netease.com>.
Hi Gary,
Think you for your reply,but I still have some doubt about the 'valid
anomaly' you mentioned .
I was puzzled with this question for some days, and I also attempted to
resolve it by writing some temperory code .
What the most I want to know is why the BaseDestination uses two memory
usage.
I am looking forword your advice. thanks.
Gary Tully wrote:
>
> Hi Lurga,
> this seems to be a valid anomaly.
> Please check out the contributing[1] details to see the best way to
> make your changes permanent.
> [1] http://activemq.apache.org/contributing.html
>
> 2008/11/11 lurga <az...@netease.com>:
>>
>> Queue invokes memoryUsage.isFull() method to decide when to block
>> producer
>> sending message, and FilePendingCursor invokes hasSpace() method (uses
>> systemUsage of the Queue) to decide when to flush messages to disk. I
>> made a
>> patch to avoid diverse results from these two different ways of
>> judgement.
>> Now it seems to work correct.
>>
>> Index:
>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>> ===================================================================
>> ---
>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>> (revision 712969)
>> +++
>> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
>> (working copy)
>> @@ -141,6 +141,12 @@
>> memoryUsage.setParent(systemUsage.getMemoryUsage());
>> }
>>
>> + long memoryUsageHighWaterMark =
>> + Math.min(memoryUsage.getLimit(),
>> systemUsage.getMemoryUsage().getLimit())
>> + * messages.getMemoryUsageHighWaterMark()
>> + / systemUsage.getMemoryUsage().getLimit();
>> + messages.setMemoryUsageHighWaterMark((int)
>> memoryUsageHighWaterMark);
>> +
>> if (isOptimizedDispatch()) {
>> this.taskRunner = taskFactory.createTaskRunner(this,
>> "TempQueue: " + destination.getPhysicalName());
>> }else {
>> --
>> View this message in context:
>> http://www.nabble.com/memoryUsage-configuration-tp20416088p20435735.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://www.nabble.com/memoryUsage-configuration-tp20416088p20453099.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: memoryUsage configuration
Posted by Gary Tully <ga...@gmail.com>.
Hi Lurga,
this seems to be a valid anomaly.
Please check out the contributing[1] details to see the best way to
make your changes permanent.
[1] http://activemq.apache.org/contributing.html
2008/11/11 lurga <az...@netease.com>:
>
> Queue invokes memoryUsage.isFull() method to decide when to block producer
> sending message, and FilePendingCursor invokes hasSpace() method (uses
> systemUsage of the Queue) to decide when to flush messages to disk. I made a
> patch to avoid diverse results from these two different ways of judgement.
> Now it seems to work correct.
>
> Index:
> activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
> ===================================================================
> --- activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
> (revision 712969)
> +++ activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
> (working copy)
> @@ -141,6 +141,12 @@
> memoryUsage.setParent(systemUsage.getMemoryUsage());
> }
>
> + long memoryUsageHighWaterMark =
> + Math.min(memoryUsage.getLimit(),
> systemUsage.getMemoryUsage().getLimit())
> + * messages.getMemoryUsageHighWaterMark()
> + / systemUsage.getMemoryUsage().getLimit();
> + messages.setMemoryUsageHighWaterMark((int)
> memoryUsageHighWaterMark);
> +
> if (isOptimizedDispatch()) {
> this.taskRunner = taskFactory.createTaskRunner(this,
> "TempQueue: " + destination.getPhysicalName());
> }else {
> --
> View this message in context: http://www.nabble.com/memoryUsage-configuration-tp20416088p20435735.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: memoryUsage configuration
Posted by lurga <az...@netease.com>.
Queue invokes memoryUsage.isFull() method to decide when to block producer
sending message, and FilePendingCursor invokes hasSpace() method (uses
systemUsage of the Queue) to decide when to flush messages to disk. I made a
patch to avoid diverse results from these two different ways of judgement.
Now it seems to work correct.
Index:
activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
===================================================================
--- activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
(revision 712969)
+++ activemq-core/src/main/java/org/apache/activemq/broker/region/Queue.java
(working copy)
@@ -141,6 +141,12 @@
memoryUsage.setParent(systemUsage.getMemoryUsage());
}
+ long memoryUsageHighWaterMark =
+ Math.min(memoryUsage.getLimit(),
systemUsage.getMemoryUsage().getLimit())
+ * messages.getMemoryUsageHighWaterMark()
+ / systemUsage.getMemoryUsage().getLimit();
+ messages.setMemoryUsageHighWaterMark((int)
memoryUsageHighWaterMark);
+
if (isOptimizedDispatch()) {
this.taskRunner = taskFactory.createTaskRunner(this,
"TempQueue: " + destination.getPhysicalName());
}else {
--
View this message in context: http://www.nabble.com/memoryUsage-configuration-tp20416088p20435735.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.