You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by fenbers <Ma...@noaa.gov> on 2012/08/07 14:47:26 UTC
db-N.log files accumulating
I've configured my ActiveMQ client not to acknowledge messages. This is
because I want the messages redelivered if the user stops and restarts the
client app. The messages expire in 6 minutes anyway. So any older messages
*shouldn't* be delivered.
I notice in my $ACTIVEMQ_HOME/data/kahadb directory there are files named
db-N.log where N is a sequential number starting at 1, and these are all 33
Megabytes in size. When N gets to 300+, we see a huge performance hit in
the apps that use the broker – even though we still have plenty of free disk
space. The only way I've found to rectify the performance problem is to
delete the contents of the kahadb directory and restart ActiveMQ. This
fixes the problem temporarily until the log files increase to over 300
again.
When N > 300, the modification times of lower-numbered files are
significantly older. If the messages expire after 6 minutes, what purpose
do these older db-N.log files serve? What can I do to prevent the
performance degradation as these db-N.log files build up? What can I do to
prevent these files from accumulating in the first place?
This article:
http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
suggests the problem occurs because there are un-acknowledged messages
causing the files to still be “in-use”. So I could probably eliminate the
*.log file accumulation by acknowledging the messages, but I cause another
problem in doing so, in which messages will not be not redelivered if the
client is restarted. For the purposes described in my first paragraph, the
messages are deliberately not acknowledged. Can I acknowledge messages and
yet still have the unexpired messages redelivered if my client app is
restarted? Or can I safely delete the log files that are significantly
older than 6 minutes?
Mark
--
View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: db-N.log files accumulating
Posted by fenbers <Ma...@noaa.gov>.
Ah, this sounds like the solution I've been seeking!! I'll have to
try this out! If it works, you must be an angel from the Almighty!
(It's a Topic, BTW.)
Mark
On 8/10/2012 4:52 AM, Raul Kripalani
[via ActiveMQ] wrote:
Is the destination a topic or a queue? If it's a
topic, have you considered
using retroactive consumers with a timed subscription recovery
policy [2]
set to 6 minutes?
[1] http://activemq.apache.org/retroactive-consumer.html
[2] http://activemq.apache.org/subscription-recovery-policy.html
Regards,
*Raúl Kripalani*
*Principal Consultant | FuseSource Corp.
[hidden email]
| fusesource.com < http://www.fusesource.com/ >
skype: raul.fuse | twitter: @raulvk < http://twitter.com/raulvk >,
@fusenews< http://twitter.com/fusenews >
*
blog: F3 - Flashes From the
Field< http://blog.raulkr.net/?utm_source=fusesourceemail&utm_medium=email&utm_campaign=fusesourcemail >
| aboutme: http://about.me/raulkripalani
< http://twitter.com/fusenews >
On 7 August 2012 13:47, fenbers < [hidden email] >
wrote:
> I've configured my ActiveMQ client not to acknowledge
messages. This is
> because I want the messages redelivered if the user stops
and restarts the
> client app. The messages expire in 6 minutes anyway. So
any older
> messages
> *shouldn't* be delivered.
>
> I notice in my $ACTIVEMQ_HOME/data/kahadb directory there
are files named
> db-N.log where N is a sequential number starting at 1, and
these are all 33
> Megabytes in size. When N gets to 300+, we see a huge
performance hit in
> the apps that use the broker – even though we still have
plenty of free
> disk
> space. The only way I've found to rectify the performance
problem is to
> delete the contents of the kahadb directory and restart
ActiveMQ. This
> fixes the problem temporarily until the log files increase
to over 300
> again.
>
> When N > 300, the modification times of lower-numbered
files are
> significantly older. If the messages expire after 6
minutes, what purpose
> do these older db-N.log files serve? What can I do to
prevent the
> performance degradation as these db-N.log files build up?
What can I do to
> prevent these files from accumulating in the first place?
>
> This article:
>
> http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
> suggests the problem occurs because there are
un-acknowledged messages
> causing the files to still be “in-use”. So I could
probably eliminate the
> *.log file accumulation by acknowledging the messages, but
I cause another
> problem in doing so, in which messages will not be not
redelivered if the
> client is restarted. For the purposes described in my
first paragraph, the
> messages are deliberately not acknowledged. Can I
acknowledge messages and
> yet still have the unexpired messages redelivered if my
client app is
> restarted? Or can I safely delete the log files that are
significantly
> older than 6 minutes?
>
> Mark
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843.html
> Sent from the ActiveMQ - User mailing list archive at
Nabble.com.
>
If you reply to this email, your
message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4655002.html
To unsubscribe from db-N.log files accumulating, click
here .
NAML
--
View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4655026.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: db-N.log files accumulating
Posted by Raul Kripalani <ra...@fusesource.com>.
I'm glad to hear this solution works for you!
You can configure the subscription recovery policy on any granularity
level you wish, by simply indicating the appropriate destination name
or wildcards in the destinationPolicy element inside which the
recovery policy is nested.
I'm on a mobile device now and can't search for the links to the
relevant doc sections, but you want to search for "destination
policies" inside the AMQ site.
Hope that helps,
Raúl.
Sent from my iPhone
On 10 Aug 2012, at 23:34, fenbers <Ma...@noaa.gov> wrote:
> Yes, your suggestion works, and you *are* an angel!! I have just one simple
> follow-up question. I have one ActiveMQ instance brokering two separate
> topics. I want to set one topic (with many large messages) with a
> timedSubscriptionRecoveryPolicy of 6 minutes, while the other topic has much
> fewer and smaller messages and is to have a timedSubscriptionRecoveryPolicy
> of 12 hours, but it appears that because this activemq.xml setting is not
> topic-specific, I can only set it broker-wide, i.e., it can only be one or
> the other (either 6 minutes or 12 hours). Is this a correct assumption?
>
> Mark
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4655027.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: db-N.log files accumulating
Posted by fenbers <Ma...@noaa.gov>.
Yes, your suggestion works, and you *are* an angel!! I have just one simple
follow-up question. I have one ActiveMQ instance brokering two separate
topics. I want to set one topic (with many large messages) with a
timedSubscriptionRecoveryPolicy of 6 minutes, while the other topic has much
fewer and smaller messages and is to have a timedSubscriptionRecoveryPolicy
of 12 hours, but it appears that because this activemq.xml setting is not
topic-specific, I can only set it broker-wide, i.e., it can only be one or
the other (either 6 minutes or 12 hours). Is this a correct assumption?
Mark
--
View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4655027.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: db-N.log files accumulating
Posted by Raul Kripalani <ra...@fusesource.com>.
Is the destination a topic or a queue? If it's a topic, have you considered
using retroactive consumers with a timed subscription recovery policy [2]
set to 6 minutes?
[1] http://activemq.apache.org/retroactive-consumer.html
[2] http://activemq.apache.org/subscription-recovery-policy.html
Regards,
*Raúl Kripalani*
*Principal Consultant | FuseSource Corp.
raul@fusesource.com | fusesource.com <http://www.fusesource.com/>
skype: raul.fuse | twitter: @raulvk <http://twitter.com/raulvk>,
@fusenews<http://twitter.com/fusenews>
*
blog: F3 - Flashes From the
Field<http://blog.raulkr.net/?utm_source=fusesourceemail&utm_medium=email&utm_campaign=fusesourcemail>
| aboutme: http://about.me/raulkripalani
<http://twitter.com/fusenews>
On 7 August 2012 13:47, fenbers <Ma...@noaa.gov> wrote:
> I've configured my ActiveMQ client not to acknowledge messages. This is
> because I want the messages redelivered if the user stops and restarts the
> client app. The messages expire in 6 minutes anyway. So any older
> messages
> *shouldn't* be delivered.
>
> I notice in my $ACTIVEMQ_HOME/data/kahadb directory there are files named
> db-N.log where N is a sequential number starting at 1, and these are all 33
> Megabytes in size. When N gets to 300+, we see a huge performance hit in
> the apps that use the broker – even though we still have plenty of free
> disk
> space. The only way I've found to rectify the performance problem is to
> delete the contents of the kahadb directory and restart ActiveMQ. This
> fixes the problem temporarily until the log files increase to over 300
> again.
>
> When N > 300, the modification times of lower-numbered files are
> significantly older. If the messages expire after 6 minutes, what purpose
> do these older db-N.log files serve? What can I do to prevent the
> performance degradation as these db-N.log files build up? What can I do to
> prevent these files from accumulating in the first place?
>
> This article:
>
> http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
> suggests the problem occurs because there are un-acknowledged messages
> causing the files to still be “in-use”. So I could probably eliminate the
> *.log file accumulation by acknowledging the messages, but I cause another
> problem in doing so, in which messages will not be not redelivered if the
> client is restarted. For the purposes described in my first paragraph, the
> messages are deliberately not acknowledged. Can I acknowledge messages and
> yet still have the unexpired messages redelivered if my client app is
> restarted? Or can I safely delete the log files that are significantly
> older than 6 minutes?
>
> Mark
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: db-N.log files accumulating
Posted by Tim Bain <tb...@alumni.duke.edu>.
And when you used the Log4J configuration described in that link, which
destination or destinations were the ones that prevented the files from
being deleted?
On Jul 27, 2017 11:49 AM, "srramu45" <sb...@gmail.com> wrote:
> HI,
>
> I do have same situation and db-log1-89 with 33 mb size seems to be
> increasing. Temporary we are stopping both the amq servers and cleaning
> kahaDB and again after few weeks same issue.
>
> I have tried below link , no luck Can you help me to resolve this
> permanently ?
>
>
> http://activemq.apache.org/why-do-kahadb-log-files-
> remain-after-cleanup.html
>
> Thanks,
> Ramu
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/db-N-log-files-accumulating-tp4654843p4728974.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: db-N.log files accumulating
Posted by fenbers <Ma...@noaa.gov>.
These accumulate (are not auto-purged) when they still contain undelivered
messages to at least one registered subscriber. Check that all subscribers
are acknowledging receipt of all messages. Also check that your message
expiration times are set to a reasonably short period of time.
Mark
On Thu, Jul 27, 2017 at 12:54 PM, srramu45 [via ActiveMQ] <
ml+s2283324n4728974h12@n4.nabble.com> wrote:
> HI,
>
> I do have same situation and db-log1-89 with 33 mb size seems to be
> increasing. Temporary we are stopping both the amq servers and cleaning
> kahaDB and again after few weeks same issue.
>
> I have tried below link , no luck Can you help me to resolve this
> permanently ?
>
> http://activemq.apache.org/why-do-kahadb-log-files-
> remain-after-cleanup.html
>
> Thanks,
> Ramu
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-
> tp4654843p4728974.html
> To unsubscribe from db-N.log files accumulating, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4654843&code=TWFyay5GZW5iZXJzQG5vYWEuZ292fDQ2NTQ4NDN8LTE3NTIwMDk3MTU=>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4728976.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: db-N.log files accumulating
Posted by srramu45 <sb...@gmail.com>.
HI,
I do have same situation and db-log1-89 with 33 mb size seems to be
increasing. Temporary we are stopping both the amq servers and cleaning
kahaDB and again after few weeks same issue.
I have tried below link , no luck Can you help me to resolve this
permanently ?
http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
Thanks,
Ramu
--
View this message in context: http://activemq.2283324.n4.nabble.com/db-N-log-files-accumulating-tp4654843p4728974.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.