You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Bruce Snyder <br...@gmail.com> on 2010/11/09 19:40:44 UTC
Re: storeUsage with kahaDB which files
On Tue, Nov 9, 2010 at 10:24 AM, Josh Carlson <jc...@e-dialog.com> wrote:
> We running the activemq broker version 5.3.0 and using the STOMP clients for producers and consumers. We ran into this morning where the store usage had been exceeded and producers were blocked on the sends. I noticed that we had the following configured for systemUsage:
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="20 mb"/>
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="1 gb" name="foo"/>
> </storeUsage>
> <tempUsage>
> <tempUsage limit="100 mb"/>
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> I solved the problem simply by increasing the storeUsage and restarting. But what I'm wondering is what constitutes the store and thus what file(s) are being counted against this threshold. I noted that the root of the data directory is close to 1 GB. Is it counting the whole directory?
Yes, correct.
> In this directory I see the files db.data, db.redo, a lock file, and several db-N.log files. What are the .log files? From: http://activemq.apache.org/kahadb.html it sounds like they might be the 'message data logs'. Are these the actual persisted messages?
Yes, the .log files are the actual persisted messages.
> Does the fact that I have some old .log files (from Augest) mean I have some abandoned messages?
I don't know if you have abandoned messages, but I can tell you that
there was a bug that prevented the .log files from being cleaned up
appropriately. This bug was just fixed recently. Currently we are
discussing a release very soon (5.4.2) that will include this fix.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Re: storeUsage with kahaDB which files
Posted by Bruce Snyder <br...@gmail.com>.
On Tue, Nov 9, 2010 at 2:41 PM, Josh Carlson <jc...@e-dialog.com> wrote:
> Thanks for the quick response Bruce we must be hitting the 5.3 bug where the persisted messages are not being cleaned up because those messages don't exist in the queues.
>
> While on the topic of the systemUsage thresholds. Can you or someone else describe what the memoryUsage limit refers to. I have read two descriptions, one here: http://activemq.apache.org/javalangoutofmemory.html that says it is used to control how much memory is used before non-persistent messages are spooled to disk. However, we are persisting all messages but yet in the web console that "Memory percent used" is non-zero. In another document which I purchased from Total Transaction Management it says that this limits the amount of memory the broker can use. However, this confuses me as well because we had it set to 20 MB which doesn't seem like it would be nearly enough and I have seen the JVM well more than a GB.
Here is another description that includes more detail about the
systemUsage element and how it relates to producer flow control:
http://activemq.apache.org/producer-flow-control.html
At a high level, here is a quick description of the systemUsage element:
* memoryUsage - Specifies how much heap memory ActiveMQ is allowed to use
* storeUsage - Specifies the amount of disk space that can be used to
store persistent messages
* tempUsage - Specifies the amount of disk space that can be used to
store non-persistent messages
Hope that helps.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
RE: storeUsage with kahaDB which files
Posted by Josh Carlson <jc...@e-dialog.com>.
Thanks for the quick response Bruce we must be hitting the 5.3 bug where the persisted messages are not being cleaned up because those messages don't exist in the queues.
While on the topic of the systemUsage thresholds. Can you or someone else describe what the memoryUsage limit refers to. I have read two descriptions, one here: http://activemq.apache.org/javalangoutofmemory.html that says it is used to control how much memory is used before non-persistent messages are spooled to disk. However, we are persisting all messages but yet in the web console that "Memory percent used" is non-zero. In another document which I purchased from Total Transaction Management it says that this limits the amount of memory the broker can use. However, this confuses me as well because we had it set to 20 MB which doesn't seem like it would be nearly enough and I have seen the JVM well more than a GB.
Thanks for the help
> -----Original Message-----
> From: Bruce Snyder [mailto:bruce.snyder@gmail.com]
> Sent: Tuesday, November 09, 2010 1:41 PM
> To: users@activemq.apache.org
> Subject: Re: storeUsage with kahaDB which files
>
> On Tue, Nov 9, 2010 at 10:24 AM, Josh Carlson <jc...@e-dialog.com>
> wrote:
> > We running the activemq broker version 5.3.0 and using the STOMP clients
> for producers and consumers. We ran into this morning where the store usage
> had been exceeded and producers were blocked on the sends. I noticed that
> we had the following configured for systemUsage:
> >
> > <systemUsage>
> > <systemUsage>
> > <memoryUsage>
> > <memoryUsage limit="20 mb"/>
> > </memoryUsage>
> > <storeUsage>
> > <storeUsage limit="1 gb" name="foo"/>
> > </storeUsage>
> > <tempUsage>
> > <tempUsage limit="100 mb"/>
> > </tempUsage>
> > </systemUsage>
> > </systemUsage>
> >
> > I solved the problem simply by increasing the storeUsage and restarting.
> But what I'm wondering is what constitutes the store and thus what file(s)
> are being counted against this threshold. I noted that the root of the data
> directory is close to 1 GB. Is it counting the whole directory?
>
> Yes, correct.
>
> > In this directory I see the files db.data, db.redo, a lock file, and
> several db-N.log files. What are the .log files? From:
> http://activemq.apache.org/kahadb.html it sounds like they might be the
> 'message data logs'. Are these the actual persisted messages?
>
> Yes, the .log files are the actual persisted messages.
>
> > Does the fact that I have some old .log files (from Augest) mean I have
> some abandoned messages?
>
> I don't know if you have abandoned messages, but I can tell you that
> there was a bug that prevented the .log files from being cleaned up
> appropriately. This bug was just fixed recently. Currently we are
> discussing a release very soon (5.4.2) that will include this fix.
>
> Bruce
> --
> perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-
> N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bruceblog.org/
> Twitter: http://twitter.com/brucesnyder