You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Hariharan (JIRA)" <ji...@apache.org> on 2011/08/16 14:47:27 UTC
[jira] [Created] (AMQ-3453) Tool for reading messages in journal
files
Tool for reading messages in journal files
------------------------------------------
Key: AMQ-3453
URL: https://issues.apache.org/jira/browse/AMQ-3453
Project: ActiveMQ
Issue Type: Wish
Components: Broker
Reporter: Hariharan
Hi,
Am facing some serious issues with AMQ. Some inputs from my side on what I see:
1. Am using AMQ 5.5
2. kahaDB persistence mechanism is being used.
2. The max journal file size set is 64 mb.
3. All messages posted on queues and topics are being consumed successfully by the clients.
4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
Total size of this folder is 1.6G and growing.
{code:xml}
-rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
-rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
-rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
-rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
-rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
-rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
-rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
-rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
-rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
-rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
-rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
-rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
-rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
-rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
-rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
-rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
-rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
-rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
-rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
-rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
-rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
-rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
-rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
-rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
-rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
-rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
-rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
{code}
If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
Am sure it would be really useful for a lot of us.
Thanks in advance.
Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Hariharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085685#comment-13085685 ]
Hariharan commented on AMQ-3453:
--------------------------------
Gary, thanks for the quick reply.
I remember doing this some time back but could not infer much from it.
Can you please let me know what kind of patterns I can search for in the log file to find out such destinations.
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "James Green (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085781#comment-13085781 ]
James Green commented on AMQ-3453:
----------------------------------
http://activemq.2283324.n4.nabble.com/KahaDB-storage-size-grows-despite-almost-no-pending-messages-td3573894.html for debugging ideas.
Gary - perhaps extract the import bits from the thread and add to the documentation with a FAQ entry "KahaDB log files keeps growing"?
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Hariharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086174#comment-13086174 ]
Hariharan commented on AMQ-3453:
--------------------------------
Gary, logging really helped me in identifying the issue.
We had not configured broker to discard the dead letters and so DLQ was holding reference to all of them.
Thank you so much.
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Hariharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086108#comment-13086108 ]
Hariharan commented on AMQ-3453:
--------------------------------
@Gary the cwiki page you have created is pretty clear as to how this works. I'll give it a try and update here soon.
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085677#comment-13085677 ]
Gary Tully commented on AMQ-3453:
---------------------------------
A journal reader does not exist for kahaDB at the moment, but trace level logging of MessageDatabase can help diagnose what destinations are referencing journal data files.
see some detain in the thread: http://activemq.2283324.n4.nabble.com/KahaDB-storage-size-grows-despite-almost-no-pending-messages-tp3573894p3581979.html
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085833#comment-13085833 ]
Gary Tully commented on AMQ-3453:
---------------------------------
added a description to the FAQ: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27824400
@Hariharan it is selective TRACE logging, just for the MessageDatabase class.
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3453) Tool for reading messages in journal
files
Posted by "Hariharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085806#comment-13085806 ]
Hariharan commented on AMQ-3453:
--------------------------------
James / Gary,
Can you let me know what kind of pattern (any key words) I should be searching for from the log file. I believe it would be logging a lot of things once I start the broker in DEBUG or TRACE mode.
I have spent a lot of time trying to resolve this, so looking for some help in directly identifying the issue.
> Tool for reading messages in journal files
> ------------------------------------------
>
> Key: AMQ-3453
> URL: https://issues.apache.org/jira/browse/AMQ-3453
> Project: ActiveMQ
> Issue Type: Wish
> Components: Broker
> Reporter: Hariharan
> Labels: tools
>
> Hi,
> Am facing some serious issues with AMQ. Some inputs from my side on what I see:
> 1. Am using AMQ 5.5
> 2. kahaDB persistence mechanism is being used.
> 2. The max journal file size set is 64 mb.
> 3. All messages posted on queues and topics are being consumed successfully by the clients.
> 4. I checked the pending messages by clicking on browseMessages button in Operations tab under each queue and topic and a lot of them were empty while some had messages that were variable over time. So am sure the messages are being acknowledged properly.
> But what I see is that the journal files are not getting cleaned up regularly. Some of them do get cleaned up but a lot of them stay back. A snap shot is given below.
> Total size of this folder is 1.6G and growing.
> {code:xml}
> -rw-rw-r-- 1 manih manih 0 Aug 6 09:15 lock
> -rw-rw-r-- 1 manih manih 67110696 Aug 8 10:37 db-8.log
> -rw-rw-r-- 1 manih manih 67115667 Aug 8 13:34 db-9.log
> -rw-rw-r-- 1 manih manih 67109195 Aug 9 03:43 db-13.log
> -rw-rw-r-- 1 manih manih 67108941 Aug 9 07:42 db-14.log
> -rw-rw-r-- 1 manih manih 67110336 Aug 9 15:56 db-16.log
> -rw-rw-r-- 1 manih manih 67108973 Aug 9 18:41 db-17.log
> -rw-rw-r-- 1 manih manih 67109661 Aug 10 06:06 db-20.log
> -rw-rw-r-- 1 manih manih 67112421 Aug 10 14:43 db-22.log
> -rw-rw-r-- 1 manih manih 67108882 Aug 10 20:30 db-24.log
> -rw-rw-r-- 1 manih manih 67109313 Aug 11 09:19 db-27.log
> -rw-rw-r-- 1 manih manih 67109241 Aug 11 16:30 db-29.log
> -rw-rw-r-- 1 manih manih 67108976 Aug 12 06:02 db-33.log
> -rw-rw-r-- 1 manih manih 67116308 Aug 12 11:11 db-34.log
> -rw-rw-r-- 1 manih manih 67116690 Aug 12 15:54 db-35.log
> -rw-rw-r-- 1 manih manih 67109627 Aug 12 18:57 db-36.log
> -rw-rw-r-- 1 manih manih 67111521 Aug 13 02:54 db-37.log
> -rw-rw-r-- 1 manih manih 67114239 Aug 13 12:13 db-38.log
> -rw-rw-r-- 1 manih manih 67117361 Aug 13 21:26 db-39.log
> -rw-rw-r-- 1 manih manih 67117848 Aug 14 06:46 db-40.log
> -rw-rw-r-- 1 manih manih 67113289 Aug 15 13:09 db-46.log
> -rw-rw-r-- 1 manih manih 67109565 Aug 15 17:05 db-47.log
> -rw-rw-r-- 1 manih manih 67115347 Aug 15 21:32 db-48.log
> -rw-rw-r-- 1 manih manih 67109370 Aug 16 05:10 db-50.log
> -rw-rw-r-- 1 manih manih 3291400 Aug 16 08:36 db.redo
> -rw-rw-r-- 1 manih manih 53694464 Aug 16 08:36 db.data
> -rw-rw-r-- 1 manih manih 66584576 Aug 16 08:36 db-51.log
> {code}
> If all messages are being consumed regularly, then am not sure why the old files are staying back while some of them created after them got cleaned up.
> Am not able to read the files as they are in binary format. Am also not sure if I can go ahead and delete the old files.
> Is there any tool that can help me in finding out what messages these files are actually holding? If there are no tools like that can AMQ developers develop one?
> Am sure it would be really useful for a lot of us.
> Thanks in advance.
> Hari
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira