You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ripienaar <ri...@devco.net> on 2010/08/06 21:13:54 UTC
Kaha corruption in 5.4-SNAPSHOT
Leaving my persistance settings to default I wanted to do some tests about
the behavior of the broker with lots of messages incoming when the consumer
is dead.
I am using a slow virtual machine and Tanukiwrapper, when Tanuki detects the
ActiveMQ VM is not responding it kills and restarts it - in effect it's a
test what would happen should a server just loose power and I'm quite
interested in knowing my persistent queues will be able to survive such a
power event.
It's my understanding that the Kaha store is specifically resilient to this
kind of crash/server death/VM crash but what I am seeing are exceptions
like:
2010-08-06 20:02:49,948 [erSimpleAppMain] ERROR BrokerService
- Failed to start ActiveMQ JMS Message Broker. Reason: java.io.EOFException:
Chunk stream does not exist at page: 0
java.io.EOFException: Chunk stream does not exist at page: 0
at
org.apache.kahadb.page.Transaction$2.readPage(Transaction.java:454)
at org.apache.kahadb.page.Transaction$2.<init>(Transaction.java:431)
at
org.apache.kahadb.page.Transaction.openInputStream(Transaction.java:428)
at org.apache.kahadb.page.Transaction.load(Transaction.java:404)
at org.apache.kahadb.page.Transaction.load(Transaction.java:361)
at
org.apache.activemq.store.kahadb.MessageDatabase$1.execute(MessageDatabase.java:233)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
at
org.apache.activemq.store.kahadb.MessageDatabase.loadPageFile(MessageDatabase.java:220)
2010-08-06 20:02:49,992 [erSimpleAppMain] ERROR KahaDBStore
- Could not stop service:
org.apache.activemq.store.kahadb.KahaDBStore@2a869113. Reason:
java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.kahadb.page.Transaction.openOutputStream(Transaction.java:262)
at org.apache.kahadb.page.Transaction.store(Transaction.java:247)
at
org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:382)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
at
org.apache.activemq.store.kahadb.MessageDatabase.unload(MessageDatabase.java:380)
If I just delete all the data it comes back - but looses the data obviously.
Is Kaha the best storage system to use for persistence I was under the
impression that it's default enableJournalDiskSyncs should always keep the
journal updated and that a recovery will then be able to recover things.
I'd even be happy if the in-flight transaction gets aborted but would be
really nice to be able to recover the bulk of a queue post power event.
--
View this message in context: http://old.nabble.com/Kaha-corruption-in-5.4-SNAPSHOT-tp29356482p29356482.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha corruption in 5.4-SNAPSHOT
Posted by ripienaar <ri...@devco.net>.
Gary Tully wrote:
>
> your understanding is correct, If you delete the index, db.data, it
> should be able to recover the index by replaying the journal and
> recreating the index from scratch.
> If that does not work, can you please open a jira issue and attach
> your test case or configuration setup and the relevant logs. Also if
> you can share the data directory that cannot be recovered it would
> help reproduce.
>
Thanks, I'll give it a go.
Is there a reason why recovery from journal isn't the default behavior when
a broker starts and cant load its data? This is in line with what most
databases do so that unattended startups work.
--
View this message in context: http://old.nabble.com/Kaha-corruption-in-5.4-SNAPSHOT-tp29356482p29387461.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha corruption in 5.4-SNAPSHOT
Posted by hodgesz <ho...@gmail.com>.
I am also getting this issue on 5.4.1. Is it resolved in a later release?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Kaha-corruption-in-5-4-SNAPSHOT-tp2366036p3450835.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha corruption in 5.4-SNAPSHOT
Posted by rodos77 <eu...@nexj.com>.
I'm getting the same issue with the release version of 5.4.0. Deleting the
db.data avoids the problem, however, I would like to have the recovery
happen automatically without having to manually delete this file. Is there
a way to achieve this?
ripienaar wrote:
>
>
>
> Gary Tully wrote:
>>
>> your understanding is correct, If you delete the index, db.data, it
>> should be able to recover the index by replaying the journal and
>> recreating the index from scratch.
>> If that does not work, can you please open a jira issue and attach
>> your test case or configuration setup and the relevant logs. Also if
>> you can share the data directory that cannot be recovered it would
>> help reproduce.
>>
>
> Thanks, I'll give it a go.
>
> Is there a reason why recovery from journal isn't the default behavior
> when a broker starts and cant load its data? This is in line with what
> most databases do so that unattended startups work.
>
--
View this message in context: http://activemq.2283324.n4.nabble.com/Kaha-corruption-in-5-4-SNAPSHOT-tp2366036p2530590.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha corruption in 5.4-SNAPSHOT
Posted by Gary Tully <ga...@gmail.com>.
your understanding is correct, If you delete the index, db.data, it
should be able to recover the index by replaying the journal and
recreating the index from scratch.
If that does not work, can you please open a jira issue and attach
your test case or configuration setup and the relevant logs. Also if
you can share the data directory that cannot be recovered it would
help reproduce.
On 6 August 2010 20:13, ripienaar <ri...@devco.net> wrote:
>
> Leaving my persistance settings to default I wanted to do some tests about
> the behavior of the broker with lots of messages incoming when the consumer
> is dead.
>
> I am using a slow virtual machine and Tanukiwrapper, when Tanuki detects the
> ActiveMQ VM is not responding it kills and restarts it - in effect it's a
> test what would happen should a server just loose power and I'm quite
> interested in knowing my persistent queues will be able to survive such a
> power event.
>
> It's my understanding that the Kaha store is specifically resilient to this
> kind of crash/server death/VM crash but what I am seeing are exceptions
> like:
>
> 2010-08-06 20:02:49,948 [erSimpleAppMain] ERROR BrokerService
> - Failed to start ActiveMQ JMS Message Broker. Reason: java.io.EOFException:
> Chunk stream does not exist at page: 0
> java.io.EOFException: Chunk stream does not exist at page: 0
> at
> org.apache.kahadb.page.Transaction$2.readPage(Transaction.java:454)
> at org.apache.kahadb.page.Transaction$2.<init>(Transaction.java:431)
> at
> org.apache.kahadb.page.Transaction.openInputStream(Transaction.java:428)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:404)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:361)
> at
> org.apache.activemq.store.kahadb.MessageDatabase$1.execute(MessageDatabase.java:233)
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
> at
> org.apache.activemq.store.kahadb.MessageDatabase.loadPageFile(MessageDatabase.java:220)
>
> 2010-08-06 20:02:49,992 [erSimpleAppMain] ERROR KahaDBStore
> - Could not stop service:
> org.apache.activemq.store.kahadb.KahaDBStore@2a869113. Reason:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.kahadb.page.Transaction.openOutputStream(Transaction.java:262)
> at org.apache.kahadb.page.Transaction.store(Transaction.java:247)
> at
> org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:382)
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
> at
> org.apache.activemq.store.kahadb.MessageDatabase.unload(MessageDatabase.java:380)
>
>
>
> If I just delete all the data it comes back - but looses the data obviously.
>
> Is Kaha the best storage system to use for persistence I was under the
> impression that it's default enableJournalDiskSyncs should always keep the
> journal updated and that a recovery will then be able to recover things.
> I'd even be happy if the in-flight transaction gets aborted but would be
> really nice to be able to recover the bulk of a queue post power event.
>
>
> --
> View this message in context: http://old.nabble.com/Kaha-corruption-in-5.4-SNAPSHOT-tp29356482p29356482.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com