You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "masc (JIRA)" <ji...@apache.org> on 2017/10/09 16:43:00 UTC
[jira] [Comment Edited] (AMQ-6831) Broker fails to start
complaining about missing/corrupt journal files after upgrading to 5.15.1
[ https://issues.apache.org/jira/browse/AMQ-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197268#comment-16197268 ]
masc edited comment on AMQ-6831 at 10/9/17 4:42 PM:
----------------------------------------------------
yes, that would be possible, as it's my local dev db, it shouldn't contain anything sensitive.
I also found that this issue is related to the preallocation kahadb settings:
{code}
pa.preallocationScope = Journal.PreallocationScope.ENTIRE_JOURNAL_ASYNC.name
pa.preallocationStrategy = Journal.PreallocationStrategy.ZEROS.name
{code}
which I enabled for performance reasons.
Disabling preallocation will yield both
{code}
java.io.IOException: Invalid location size: 1:28, size: 0
at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:88) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:936) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1151) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recoverProducerAudit(MessageDatabase.java:784) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:674) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:473) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:493) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:219) [activemq-kahadb-store-5.15.1.jar:5.15.1]
{code}
and subsequently
{code}
2017-10-09 18:34:57.513 WARN 10617 --- [pool-3-thread-1] o.a.a.store.kahadb.MessageDatabase : Cannot recover ackMessageFileMap
java.io.IOException: Invalid location size: 1:431, size: 0
at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:88) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:936) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1151) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recoverAckMessageFileMap(MessageDatabase.java:805) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:675) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:473) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:493) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:219) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) [activemq-client-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:232) [activemq-kahadb-store-5.15.1.jar:5.15.1]
{code}
but the broker will at least start.
was (Author: masc3d):
yes, that would be possible, as it's my local dev db, it shouldn't contain anything sensitive.
I also found that this issue is related to the preallocation kahadb settings:
{code}
pa.preallocationScope = Journal.PreallocationScope.ENTIRE_JOURNAL_ASYNC.name
pa.preallocationStrategy = Journal.PreallocationStrategy.ZEROS.name
{code}
which I enabled for performance reasons.
Disabling preallocation will both
{code}
java.io.IOException: Invalid location size: 1:28, size: 0
at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:88) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:936) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1151) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recoverProducerAudit(MessageDatabase.java:784) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:674) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:473) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:493) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:219) [activemq-kahadb-store-5.15.1.jar:5.15.1]
{code}
and subsequently
{code}
2017-10-09 18:34:57.513 WARN 10617 --- [pool-3-thread-1] o.a.a.store.kahadb.MessageDatabase : Cannot recover ackMessageFileMap
java.io.IOException: Invalid location size: 1:431, size: 0
at org.apache.activemq.store.kahadb.disk.journal.DataFileAccessor.readRecord(DataFileAccessor.java:88) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.disk.journal.Journal.read(Journal.java:936) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:1151) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recoverAckMessageFileMap(MessageDatabase.java:805) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:675) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:473) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:493) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:219) [activemq-kahadb-store-5.15.1.jar:5.15.1]
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) [activemq-client-5.15.1.jar:5.15.1]
at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:232) [activemq-kahadb-store-5.15.1.jar:5.15.1]
{code}
but the broker will at least start.
> Broker fails to start complaining about missing/corrupt journal files after upgrading to 5.15.1
> -----------------------------------------------------------------------------------------------
>
> Key: AMQ-6831
> URL: https://issues.apache.org/jira/browse/AMQ-6831
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.15.1
> Reporter: masc
>
> {code}
> 2017-10-09 12:29:36.802 ERROR 4197 --- [pool-3-thread-1] org.deku.leoz.node.Application : java.io.IOException: Detected missing/corrupt journal files referenced by:[0:ActiveMQ.DLQ] 3 messages affected.
> java.lang.Error: java.io.IOException: Detected missing/corrupt journal files referenced by:[0:ActiveMQ.DLQ] 3 messages affected.
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1148) ~[na:1.8.0_131]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]
> Caused by: java.io.IOException: Detected missing/corrupt journal files referenced by:[0:ActiveMQ.DLQ] 3 messages affected.
> at org.apache.activemq.store.kahadb.MessageDatabase.recoverIndex(MessageDatabase.java:965) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:717) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:714) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:473) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:493) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:219) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) ~[activemq-client-5.15.1.jar:5.15.1]
> at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:232) ~[activemq-kahadb-store-5.15.1.jar:5.15.1]
> {code}
> Downgrading to 5.15.0 resolves the problem.
> This is an embedded broker setup, using
> {code:none}
> val pa = brokerService.persistenceAdapter as KahaDBPersistenceAdapter
> pa.isCheckForCorruptJournalFiles = true
> pa.isIgnoreMissingJournalfiles = false
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)