You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Andy Boothe [WCG]" <ab...@wcgworld.com> on 2013/12/13 17:27:33 UTC
KahaDB log files not getting cleaned up -- Why?
Hello,
I have an ActiveMQ instance that has been running nicely for a couple of months, but doesn't seem to clean up its KahaDB logs like I expect it to. (More specifically, I expect to have like 1 log file, but in reality I have ~1500 log files with 49G of data.)
ActiveMQ has processed about 12M messages over the past couple of months, and has roughly 106K messages currently pending (in DLQs):
[cid:8627D172-A47C-4687-BDE1-89CFA12B5F38]
Given the number of messages currently pending, I would not expect the KahaDB logs to be very big. In reality, they're about 49G:
$ cd apache-activemq-5.8.0/data/kahadb/
$ ls -lht | head
total 49G
-rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
-rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
-rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
Based on the maximum log ID, almost none of the log files appear to have been cleaned up:
$ ls db-*.log | wc
1540 1540 17421
I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla configuration is Java heap size, which shouldn't affect whether or not DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with Camel. Obviously from the number of queues, I have several different workflows in flight, but here's a representative example of how I'm using ActiveMQ in my Java code:
from("activemq:statuslinks")
.bean(new LinkExtract())
.split(body(String.class).tokenize("\n"))
.filter(new LinkHandled(jdbcConnectionString))
.to("activemq:linkunwind");
The only problems I'm seeing in my activemq.log file appear to be "normal." I get rashes of transport exceptions every few weeks, which I have associated (rightly or wrongly) with hitting the web admin interface. Given the infrequency and nature of the errors, I don't think they explain why the logs wouldn't be deleted. In case it's useful, though, here are a couple of representative lines from the log file:
2013-12-13 09:46:52,269 | WARN | Transport Connection to: tcp://127.0.0.1:39604 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://127.0.0.1:39604 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker
2013-12-13 14:43:41,215 | WARN | Transport Connection to: tcp://127.0.0.1:35711 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:35711@61616
Given the use case — very simple message processing using Camel — and ActiveMQ's vanilla configuration, I would expect the log files to be tiny. Clearly, though they're not. Does anyone have any idea why those files aren't getting deleted? I'm sure I'm doing (or not doing) something very silly, but after pouring over the logs and documentation, I just can't figure out what I'm doing wrong. Does anyone have an idea why my logs wouldn't be getting cleaned up?
Thanks everyone in advance for your attention and help! And thank you ActiveMQ devs for such a wonderful product! :)
Andy Boothe
Re: KahaDB log files not getting cleaned up -- Why?
Posted by Timothy Bish <ta...@gmail.com>.
On 12/13/2013 01:02 PM, Andy Boothe [WCG] wrote:
> Timothy,
>
> Thank you for the quick response!
>
> I had read that, but didn't see anything useful in it. Having re-read it,
> I realize that my DLQs (which contain 100k messages) were a likely suspect
> in keeping the log files "live." I cleared the DLQs, and ActiveMQ
> diligently deleted most of the log files a couple minutes later:
>
> $ cd apache-activemq-5.8.0/data/kahadb/
> $ ls -lht | head
> $ ls -lht | head
> total 4.9G
> -rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 18:00 db-1564.log
> -rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 18:00 db.data
> -rw-rw-r-- 1 ubuntu ubuntu 2.6M Dec 13 18:00 db.redo
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 17:44 db-1563.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 16:14 db-1562.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
>
> Thank you for the hint, Timothy!
>
>
> Andy Boothe
> Director of Data Engineering, Analytics
>
> WCG
> 60 Francisco St, San Francisco, 94133
> direct 512-961-3993 cell 979-574-1089
> twitter @sigpwned <http://twitter.com/sigpwned> linkedin aboothe
> <http://www.linkedin.com/in/aboothe>
> github sigpwned <https://github.com/sigpwned> stack overflow sigpwned
> <http://stackoverflow.com/users/2103602/sigpwned>
>
> € € € € € € € Go. Ahead.
> ŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠ
>
>
>
>
>
> On 12/13/13 10:59 AM, "Timothy Bish" <ta...@gmail.com> wrote:
>
>> On 12/13/2013 11:27 AM, Andy Boothe [WCG] wrote:
>>> Hello,
>>>
>>> I have an ActiveMQ instance that has been running nicely for a couple
>>> of months, but doesn't seem to clean up its KahaDB logs like I expect
>>> it to. (More specifically, I expect to have like 1 log file, but in
>>> reality I have ~1500 log files with 49G of data.)
>>>
>>> ActiveMQ has processed about 12M messages over the past couple of
>>> months, and has roughly 106K messages currently pending (in DLQs):
>>>
>>>
>>> Given the number of messages currently pending, I would not expect the
>>> KahaDB logs to be very big. In reality, they're about 49G:
>>>
>>> $ cd apache-activemq-5.8.0/data/kahadb/
>>> $ ls -lht | head
>>> total 49G
>>> -rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
>>> -rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
>>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
>>>
>>> Based on the maximum log ID, almost none of the log files appear to
>>> have been cleaned up:
>>>
>>> $ ls db-*.log | wc
>>> 1540 1540 17421
>>>
>>> I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla
>>> configuration is Java heap size, which shouldn't affect whether or not
>>> DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with
>>> Camel. Obviously from the number of queues, I have several different
>>> workflows in flight, but here's a representative example of how I'm
>>> using ActiveMQ in my Java code:
>>>
>>> from("activemq:statuslinks")
>>>
>>> .bean(new LinkExtract())
>>>
>>> .split(body(String.class).tokenize("\n"))
>>>
>>> .filter(new LinkHandled(jdbcConnectionString))
>>>
>>> .to("activemq:linkunwind");
>>>
>>>
>>> The only problems I'm seeing in my activemq.log file appear to be
>>> "normal." I get rashes of transport exceptions every few weeks, which
>>> I have associated (rightly or wrongly) with hitting the web admin
>>> interface. Given the infrequency and nature of the errors, I don't
>>> think they explain why the logs wouldn't be deleted. In case it's
>>> useful, though, here are a couple of representative lines from the log
>>> file:
>>>
>>> 2013-12-13 09:46:52,269 | WARN | Transport Connection to:
>>> tcp://127.0.0.1:39604 failed:
>>> org.apache.activemq.transport.InactivityIOException: Channel was
>>> inactive for too (>30000) long: tcp://127.0.0.1:39604 |
>>> org.apache.activemq.broker.TransportConnection.Transport |
>>> ActiveMQ InactivityMonitor Worker
>>> 2013-12-13 14:43:41,215 | WARN | Transport Connection to:
>>> tcp://127.0.0.1:35711 failed: java.io.EOFException |
>>> org.apache.activemq.broker.TransportConnection.Transport |
>>> ActiveMQ Transport: tcp:///127.0.0.1:35711@61616
>>>
>>>
>>> Given the use case ‹ very simple message processing using Camel ‹ and
>>> ActiveMQ's vanilla configuration, I would expect the log files to be
>>> tiny. Clearly, though they're not. Does anyone have any idea why those
>>> files aren't getting deleted? I'm sure I'm doing (or not doing)
>>> something very silly, but after pouring over the logs and
>>> documentation, I just can't figure out what I'm doing wrong. Does
>>> anyone have an idea why my logs wouldn't be getting cleaned up?
>>>
>>> Thanks everyone in advance for your attention and help! And thank you
>>> ActiveMQ devs for such a wonderful product! :)
>>>
>>> *Andy Boothe*
>>>
>>> ****
>> Have you referenced this documentation:
>> http://cp.mcafee.com/d/FZsS93gs839J5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5
>> AsYeKCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7
>> fc9L9CzBXBHFShhlLtzBgY-F6lK1FJ4SMrLOtXTLuZXTdTdw0ZJehDRhrJmSNf-00Uo9ByW5iV
>> niuE42VZi9tSh_QbNZnpgbWXfUBX7_I6RK1NJKDNEnzYSUMbIzCnXBZ8lHcDVNyN4Nm9W7bwHr
>> jApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPDa14Qgmf_ouwq86Rzh
>> 0nbCy01ipoArKruq0ctEi81xx
>>
>>
>> --
>> Tim Bish
>> Sr Software Engineer | RedHat Inc.
>> tim.bish@redhat.com |
>> http://cp.mcafee.com/d/k-Kr6xEp3xESyOYUqenSbFECXCQQmhPMWWrMUSCyOeuuK-PsSCy
>> Oeu7njuudFEIzDDArzC3uAaRCjY56b4j5oDEsKrcm8CaNfgVsSMCwOgRd_HYCOYYOVRXBQrICz
>> DC4TAPhOZORQX8EGTKNOEuvkzaT0QSCrodTVeZXTLuZXCXCODeJQ-BmUS_BPrtfzgL7VJNwnp7
>> cLTbWgHmpfPz5y9yIjQen1mSD8PWEJSHroD_00joVZZ4S2_id405qjQe6jh0vkyq87p7_gQggH
>> cQg4vDek29EwIv-MZ0QgdH6y0Knd402AON8TsS5aMcvwY |
>> http://cp.mcafee.com/d/2DRPoAd6QmnD3hO-Ntd4TsSCyOeu7nju76QQmhPPRTSrCQQmhPM
>> WWrPNJd5AsYYzssMrQxmIOvwENoyoH4Z3BPpyN4Nm9W7bCS4Q6i6FLZvASnDCneLsKztAQsYMC
>> YCqenKmKDp55mZSel3PWApmU6CSjr1K_9TLuZXTLsTsSkWfE4ZDOVJKDNEnzYSUMbIzCnXBZ8l
>> HcDVNyN4Nm9W7bwHrjApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPD
>> a14Qgmf_ouwq86Rzh0nbCy01ipoArKrZzCB4t6l
>> skype: tabish121 | twitter: @tabish121
>> blog:
>> http://cp.mcafee.com/d/FZsS73hJ5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5AsYe
>> KCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7fc9L
>> 9CzBXBHFShhlLtzBgY-F6lK1FJwSMrLOtXTLuZXTdTdw0VOl8iEjUwvkzhqI-nMSTjUQbN-rso
>> 5ShPbZO-AaRCjYUNoyoH4Z3BMlJFOc-GbtGSS9_M04SevvhdwLQzh01mAZ3xAQg7R8Cy1Sh_Qd
>> 44aPd417VPB0yq8b7_Ifgd43qNEwbBPh00FcIidTdKLPH3zP8
>>
>
Without seeing the log messages from that article all we can say is, you
still have some messages in there.
--
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.fusesource.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: KahaDB log files not getting cleaned up -- Why?
Posted by "Andy Boothe [WCG]" <ab...@wcgworld.com>.
Timothy,
Thank you for the quick response!
I had read that, but didn't see anything useful in it. Having re-read it,
I realize that my DLQs (which contain 100k messages) were a likely suspect
in keeping the log files "live." I cleared the DLQs, and ActiveMQ
diligently deleted most of the log files a couple minutes later:
$ cd apache-activemq-5.8.0/data/kahadb/
$ ls -lht | head
$ ls -lht | head
total 4.9G
-rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 18:00 db-1564.log
-rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 18:00 db.data
-rw-rw-r-- 1 ubuntu ubuntu 2.6M Dec 13 18:00 db.redo
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 17:44 db-1563.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 16:14 db-1562.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
-rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
Thank you for the hint, Timothy!
Andy Boothe
Director of Data Engineering, Analytics
WCG
60 Francisco St, San Francisco, 94133
direct 512-961-3993 cell 979-574-1089
twitter @sigpwned <http://twitter.com/sigpwned> linkedin aboothe
<http://www.linkedin.com/in/aboothe>
github sigpwned <https://github.com/sigpwned> stack overflow sigpwned
<http://stackoverflow.com/users/2103602/sigpwned>
€ € € € € € € Go. Ahead.
ŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠŠ
On 12/13/13 10:59 AM, "Timothy Bish" <ta...@gmail.com> wrote:
>On 12/13/2013 11:27 AM, Andy Boothe [WCG] wrote:
>> Hello,
>>
>> I have an ActiveMQ instance that has been running nicely for a couple
>> of months, but doesn't seem to clean up its KahaDB logs like I expect
>> it to. (More specifically, I expect to have like 1 log file, but in
>> reality I have ~1500 log files with 49G of data.)
>>
>> ActiveMQ has processed about 12M messages over the past couple of
>> months, and has roughly 106K messages currently pending (in DLQs):
>>
>>
>> Given the number of messages currently pending, I would not expect the
>> KahaDB logs to be very big. In reality, they're about 49G:
>>
>> $ cd apache-activemq-5.8.0/data/kahadb/
>> $ ls -lht | head
>> total 49G
>> -rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
>> -rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
>> -rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
>> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
>>
>> Based on the maximum log ID, almost none of the log files appear to
>> have been cleaned up:
>>
>> $ ls db-*.log | wc
>> 1540 1540 17421
>>
>> I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla
>> configuration is Java heap size, which shouldn't affect whether or not
>> DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with
>> Camel. Obviously from the number of queues, I have several different
>> workflows in flight, but here's a representative example of how I'm
>> using ActiveMQ in my Java code:
>>
>> from("activemq:statuslinks")
>>
>> .bean(new LinkExtract())
>>
>> .split(body(String.class).tokenize("\n"))
>>
>> .filter(new LinkHandled(jdbcConnectionString))
>>
>> .to("activemq:linkunwind");
>>
>>
>> The only problems I'm seeing in my activemq.log file appear to be
>> "normal." I get rashes of transport exceptions every few weeks, which
>> I have associated (rightly or wrongly) with hitting the web admin
>> interface. Given the infrequency and nature of the errors, I don't
>> think they explain why the logs wouldn't be deleted. In case it's
>> useful, though, here are a couple of representative lines from the log
>> file:
>>
>> 2013-12-13 09:46:52,269 | WARN | Transport Connection to:
>> tcp://127.0.0.1:39604 failed:
>> org.apache.activemq.transport.InactivityIOException: Channel was
>> inactive for too (>30000) long: tcp://127.0.0.1:39604 |
>> org.apache.activemq.broker.TransportConnection.Transport |
>> ActiveMQ InactivityMonitor Worker
>> 2013-12-13 14:43:41,215 | WARN | Transport Connection to:
>> tcp://127.0.0.1:35711 failed: java.io.EOFException |
>> org.apache.activemq.broker.TransportConnection.Transport |
>> ActiveMQ Transport: tcp:///127.0.0.1:35711@61616
>>
>>
>> Given the use case ‹ very simple message processing using Camel ‹ and
>> ActiveMQ's vanilla configuration, I would expect the log files to be
>> tiny. Clearly, though they're not. Does anyone have any idea why those
>> files aren't getting deleted? I'm sure I'm doing (or not doing)
>> something very silly, but after pouring over the logs and
>> documentation, I just can't figure out what I'm doing wrong. Does
>> anyone have an idea why my logs wouldn't be getting cleaned up?
>>
>> Thanks everyone in advance for your attention and help! And thank you
>> ActiveMQ devs for such a wonderful product! :)
>>
>> *Andy Boothe*
>>
>> ****
>Have you referenced this documentation:
>http://cp.mcafee.com/d/FZsS93gs839J5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5
>AsYeKCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7
>fc9L9CzBXBHFShhlLtzBgY-F6lK1FJ4SMrLOtXTLuZXTdTdw0ZJehDRhrJmSNf-00Uo9ByW5iV
>niuE42VZi9tSh_QbNZnpgbWXfUBX7_I6RK1NJKDNEnzYSUMbIzCnXBZ8lHcDVNyN4Nm9W7bwHr
>jApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPDa14Qgmf_ouwq86Rzh
>0nbCy01ipoArKruq0ctEi81xx
>
>
>--
>Tim Bish
>Sr Software Engineer | RedHat Inc.
>tim.bish@redhat.com |
>http://cp.mcafee.com/d/k-Kr6xEp3xESyOYUqenSbFECXCQQmhPMWWrMUSCyOeuuK-PsSCy
>Oeu7njuudFEIzDDArzC3uAaRCjY56b4j5oDEsKrcm8CaNfgVsSMCwOgRd_HYCOYYOVRXBQrICz
>DC4TAPhOZORQX8EGTKNOEuvkzaT0QSCrodTVeZXTLuZXCXCODeJQ-BmUS_BPrtfzgL7VJNwnp7
>cLTbWgHmpfPz5y9yIjQen1mSD8PWEJSHroD_00joVZZ4S2_id405qjQe6jh0vkyq87p7_gQggH
>cQg4vDek29EwIv-MZ0QgdH6y0Knd402AON8TsS5aMcvwY |
>http://cp.mcafee.com/d/2DRPoAd6QmnD3hO-Ntd4TsSCyOeu7nju76QQmhPPRTSrCQQmhPM
>WWrPNJd5AsYYzssMrQxmIOvwENoyoH4Z3BPpyN4Nm9W7bCS4Q6i6FLZvASnDCneLsKztAQsYMC
>YCqenKmKDp55mZSel3PWApmU6CSjr1K_9TLuZXTLsTsSkWfE4ZDOVJKDNEnzYSUMbIzCnXBZ8l
>HcDVNyN4Nm9W7bwHrjApZkmXlJIj_w09Is--yr1vF6y02J9W739EwfGhd43Iz_Eq88lCq82fPD
>a14Qgmf_ouwq86Rzh0nbCy01ipoArKrZzCB4t6l
>skype: tabish121 | twitter: @tabish121
>blog:
>http://cp.mcafee.com/d/FZsS73hJ5BVMQsLInjhdTdFEIzDxRQTxNJd5AsYZtZCVJd5AsYe
>KCYYrjhp7ff8T7c6Z8lHcDUacm8CaNfgVsSoIhclyuxOVJxd1AxGr_nVdBVVBPHTbETpd7fc9L
>9CzBXBHFShhlLtzBgY-F6lK1FJwSMrLOtXTLuZXTdTdw0VOl8iEjUwvkzhqI-nMSTjUQbN-rso
>5ShPbZO-AaRCjYUNoyoH4Z3BMlJFOc-GbtGSS9_M04SevvhdwLQzh01mAZ3xAQg7R8Cy1Sh_Qd
>44aPd417VPB0yq8b7_Ifgd43qNEwbBPh00FcIidTdKLPH3zP8
>
Re: KahaDB log files not getting cleaned up -- Why?
Posted by Timothy Bish <ta...@gmail.com>.
On 12/13/2013 11:27 AM, Andy Boothe [WCG] wrote:
> Hello,
>
> I have an ActiveMQ instance that has been running nicely for a couple
> of months, but doesn't seem to clean up its KahaDB logs like I expect
> it to. (More specifically, I expect to have like 1 log file, but in
> reality I have ~1500 log files with 49G of data.)
>
> ActiveMQ has processed about 12M messages over the past couple of
> months, and has roughly 106K messages currently pending (in DLQs):
>
>
> Given the number of messages currently pending, I would not expect the
> KahaDB logs to be very big. In reality, they're about 49G:
>
> $ cd apache-activemq-5.8.0/data/kahadb/
> $ ls -lht | head
> total 49G
> -rw-rw-r-- 1 ubuntu ubuntu 32M Dec 13 16:00 db-1562.log
> -rw-rw-r-- 1 ubuntu ubuntu 36M Dec 13 16:00 db.data
> -rw-rw-r-- 1 ubuntu ubuntu 173K Dec 13 16:00 db.redo
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 14:47 db-1561.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 13:28 db-1560.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 12:09 db-1559.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 10:55 db-1558.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 09:48 db-1557.log
> -rw-rw-r-- 1 ubuntu ubuntu 33M Dec 13 08:40 db-1556.log
>
> Based on the maximum log ID, almost none of the log files appear to
> have been cleaned up:
>
> $ ls db-*.log | wc
> 1540 1540 17421
>
> I'm using ActiveMQ 5.8.0. The only change I've made to the vanilla
> configuration is Java heap size, which shouldn't affect whether or not
> DB log files get cleaned up (AFAIK). I'm "connecting" to ActiveMQ with
> Camel. Obviously from the number of queues, I have several different
> workflows in flight, but here's a representative example of how I'm
> using ActiveMQ in my Java code:
>
> from("activemq:statuslinks")
>
> .bean(new LinkExtract())
>
> .split(body(String.class).tokenize("\n"))
>
> .filter(new LinkHandled(jdbcConnectionString))
>
> .to("activemq:linkunwind");
>
>
> The only problems I'm seeing in my activemq.log file appear to be
> "normal." I get rashes of transport exceptions every few weeks, which
> I have associated (rightly or wrongly) with hitting the web admin
> interface. Given the infrequency and nature of the errors, I don't
> think they explain why the logs wouldn't be deleted. In case it's
> useful, though, here are a couple of representative lines from the log
> file:
>
> 2013-12-13 09:46:52,269 | WARN | Transport Connection to:
> tcp://127.0.0.1:39604 failed:
> org.apache.activemq.transport.InactivityIOException: Channel was
> inactive for too (>30000) long: tcp://127.0.0.1:39604 |
> org.apache.activemq.broker.TransportConnection.Transport |
> ActiveMQ InactivityMonitor Worker
> 2013-12-13 14:43:41,215 | WARN | Transport Connection to:
> tcp://127.0.0.1:35711 failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport |
> ActiveMQ Transport: tcp:///127.0.0.1:35711@61616
>
>
> Given the use case — very simple message processing using Camel — and
> ActiveMQ's vanilla configuration, I would expect the log files to be
> tiny. Clearly, though they're not. Does anyone have any idea why those
> files aren't getting deleted? I'm sure I'm doing (or not doing)
> something very silly, but after pouring over the logs and
> documentation, I just can't figure out what I'm doing wrong. Does
> anyone have an idea why my logs wouldn't be getting cleaned up?
>
> Thanks everyone in advance for your attention and help! And thank you
> ActiveMQ devs for such a wonderful product! :)
>
> *Andy Boothe*
>
> ****
Have you referenced this documentation:
http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
--
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.fusesource.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/