You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Lior Okman (JIRA)" <ji...@apache.org> on 2011/03/08 15:36:59 UTC
[jira] Created: (AMQ-3210) OutOfMemory error on ActiveMQ startup
OutOfMemory error on ActiveMQ startup
-------------------------------------
Key: AMQ-3210
URL: https://issues.apache.org/jira/browse/AMQ-3210
Project: ActiveMQ
Issue Type: Bug
Components: Message Store
Affects Versions: 5.4.2
Environment: # java -version
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
# cat /etc/debian_version
5.0.8
Reporter: Lior Okman
Priority: Critical
Attachments: exception.log
Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089387#comment-13089387 ]
Sree Panchajanyam D commented on AMQ-3210:
------------------------------------------
In this particular case kahaDB metadata got corrupted which is preventing the server to come up. I don't see any other way to resolve this issue as we do not have causes of the crash. In any case failure to update the metadata( *.redo file) before the server goes down is causing the issue.
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (AMQ-3210) OutOfMemory error on
ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089426#comment-13089426 ]
Sree Panchajanyam D edited comment on AMQ-3210 at 8/23/11 12:46 PM:
--------------------------------------------------------------------
Corrupt journal files can be identified but not corrupt metadata.
You can ensure that the metadata is synced up regularly by setting the parameters "indexWriteBatchSize" and "checkpointInterval" to practically low values. Take a look at the documentation for these parameters at below links:
http://activemq.apache.org/kahadb.html
http://fusesource.com/docs/broker/5.5/persistence/index.html ( Optimizing the Metadata Cache)
Metadata is not synced with the cache during server crashes.
Hence, the best thing to do is to prevent ActiveMQ from crashing.
I see that in your xml you have used producer flow control I would advocate against it if you are not sure why you need it.
If you are using persistent messages use them with a time to live. Allocate store space with following caluculation "store space = no. of messages/second * avg message size * time to live * 2".
For non-persistent messages the above calc. will not hold good.
PS: in activemq.xml
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb" checkForCorruptJournalFiles="true" checksumJournalFiles="true" indexWriteBatchSize="1000" checkpointInterval="1000"/>
</persistenceAdapter>
was (Author: sreepanchajanyam):
Corrupt journal files can be identified but not corrupt metadata.
You can ensure that the metadata is synced up regularly by setting the parameters "indexWriteBatchSize" and "checkpointInterval" to practically low values. Take a look at the documentation for these parameters at below links:
http://activemq.apache.org/kahadb.html
http://fusesource.com/docs/broker/5.5/persistence/index.html ( Optimizing the Metadata Cache)
Metadata is not synced with the cache during server crashes.
Hence, the best thing to do is to prevent ActiveMQ from crashing.
I see that in your xml you have used producer flow control I would advocate against it if you are not sure why you need it.
If you are using persistent messages use them with a time to live. Allocate store space with following caluculation "store space = no. of messages/second * avg message size * time to live * 2".
For non-persistent messages the above calc. will not hold good.
PS: in activemq.xml
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb"/>
<kahaDB checkForCorruptJournalFiles="true"/>
<kahaDB checksumJournalFiles="true"/>
<kahaDB indexWriteBatchSize="1000"/>
<kahaDB checkpointInterval="1000"/>
</persistenceAdapter>
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sree Panchajanyam D updated AMQ-3210:
-------------------------------------
Comment: was deleted
(was: Sorry! please use the below configuration not the one in the PS of above comment
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb" checkForCorruptJournalFiles="true" checksumJournalFiles="true" indexWriteBatchSize="1000" checkpointInterval="1000"/>
</persistenceAdapter>)
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108476#comment-13108476 ]
Sree Panchajanyam D commented on AMQ-3210:
------------------------------------------
Did the suggested changes solve the problem ?
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090020#comment-13090020 ]
Lior Okman commented on AMQ-3210:
---------------------------------
Thanks, I'll try this configuration change.
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lior Okman updated AMQ-3210:
----------------------------
Attachment: kahadb.tar.bz2
The corrupt KahaDB directory.
In order to reproduce this issue, just start ActiveMQ with this KahaDB in /var/local/apache-activemq
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089426#comment-13089426 ]
Sree Panchajanyam D commented on AMQ-3210:
------------------------------------------
Corrupt journal files can be identified but not corrupt metadata.
You can ensure that the metadata is synced up regularly by setting the parameters "indexWriteBatchSize" and "checkpointInterval" to practically low values. Take a look at the documentation for these parameters at below links:
http://activemq.apache.org/kahadb.html
http://fusesource.com/docs/broker/5.5/persistence/index.html ( Optimizing the Metadata Cache)
Metadata is not synced with the cache during server crashes.
Hence, the best thing to do is to prevent ActiveMQ from crashing.
I see that in your xml you have used producer flow control I would advocate against it if you are not sure why you need it.
If you are using persistent messages use them with a time to live. Allocate store space with following caluculation "store space = no. of messages/second * avg message size * time to live * 2".
For non-persistent messages the above calc. will not hold good.
PS: in activemq.xml
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb"/>
<kahaDB checkForCorruptJournalFiles="true"/>
<kahaDB checksumJournalFiles="true"/>
<kahaDB indexWriteBatchSize="1000"/>
<kahaDB checkpointInterval="1000"/>
</persistenceAdapter>
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089393#comment-13089393 ]
Lior Okman commented on AMQ-3210:
---------------------------------
Is there any way to identify this corruption on startup and purge the database if it is identified, so that the ActiveMQ process can start without an operator having to manually delete the KahaDB files?
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lior Okman updated AMQ-3210:
----------------------------
Attachment: exception.log
The full exception that is shown on startup.
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: exception.log
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089428#comment-13089428 ]
Sree Panchajanyam D commented on AMQ-3210:
------------------------------------------
Sorry! please use the below configuration not the one in the PS of above comment
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb" checkForCorruptJournalFiles="true" checksumJournalFiles="true" indexWriteBatchSize="1000" checkpointInterval="1000"/>
</persistenceAdapter>
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lior Okman updated AMQ-3210:
----------------------------
Attachment: activemq.xml
The activemq.xml file being used
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089356#comment-13089356 ]
Lior Okman commented on AMQ-3210:
---------------------------------
> Were you trying to restart the activeMQ server after a crash ?
ActiveMQ is being run with the Tanuki Software wrapper as a daemon, with the wrapper.on_exit.default set to RESTART, so after the server crashes it is automatically being restarted.
> If so what were the conditions it crashed under?
The application can send around the order of a few hundred messages on 6 topics on startup. All the messages being sent are persistent, and there are 3 applications communicating via ActiveMQ, one written in Java and two written in C++ with the CMS driver downloaded from the project homepage. Everything is being run on the same host.
> What was the heap space allocated to activeMQ ?
ActiveMQ is started with -Xmx512m .
> Were there any persistent messages that were delivered when you tried to start activeMQ? If so please give the number of pending messages.
When these crashes occurred then most of the messages sent via ActiveMQ were persistent. Unfortunately, I don't know how to check how many messages are pending, since ActiveMQ doesn't restart and that prevents me from checking via the admin console.
The application was modified to not send any persistent messages, and KahaDB was disabled via the ActiveMQ configuration. After these changes the crash hasn't reoccurred.
> As I see it huge number of persistent messages not delivered could be the issue.
Is there anything I can do to help resolve this issue?
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Gary Tully (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully resolved AMQ-3210.
-----------------------------
Resolution: Incomplete
Fix Version/s: 5.6.0
Marking this as incomplete as there is no test case and the suggested comments should help.
If the index is out of sync w.r.t the journal, just deleting the index, db.data is sufficient to have the index rebuild automatically on restart, by replaying the journal.
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Fix For: 5.6.0
>
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Sree Panchajanyam D (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089350#comment-13089350 ]
Sree Panchajanyam D commented on AMQ-3210:
------------------------------------------
Please answer the above queries:
Were you trying to restart the activeMQ server after a crash ? If so what were the conditions it crashed under?
What was the heap space allocated to activeMQ ? Were there any persistent messages that were delivered when you tried to start activeMQ? If so please give the number of pending messages. As I see it huge number of persistent messages not delivered could be the issue. \
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-3210) OutOfMemory error on ActiveMQ startup
Posted by "Lior Okman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108482#comment-13108482 ]
Lior Okman commented on AMQ-3210:
---------------------------------
Still checking.
Since I don't have a scenario to recreate the issue and the issue doesn't always happen, I can't say for sure if the configuration change fixed it.
> OutOfMemory error on ActiveMQ startup
> -------------------------------------
>
> Key: AMQ-3210
> URL: https://issues.apache.org/jira/browse/AMQ-3210
> Project: ActiveMQ
> Issue Type: Bug
> Components: Message Store
> Affects Versions: 5.4.2
> Environment: # java -version
> java version "1.6.0_18"
> OpenJDK Runtime Environment (IcedTea6 1.8.3) (6b18-1.8.3-2~lenny1)
> OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)
> # cat /etc/debian_version
> 5.0.8
> Reporter: Lior Okman
> Priority: Critical
> Attachments: activemq.xml, exception.log, kahadb.tar.bz2
>
>
> Probably due to some kind of message store corruption, when trying to start ActiveMQ, I get OutOfMemory errors and the startup simply fails.
> This can be solved by deleting /var/local/apache-activemq/kahadb, after which ActiveMQ starts with no issue.
> This issue doesn't always happen, and I'm not sure of a scenario that can reproduce this. I do have a corrupted kahadb directory that reproduces the problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira