You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by nfx <se...@gmail.com> on 2014/09/01 16:09:36 UTC
LevelDBClient operation failed. NullPointerException after entering
recovery mode
Once in a while we're getting following exception in AMQ logs and there is no
other way than purging the queue. How can we overcome this issue? Is it okay
to use LevelDB store now? Any answers?
2014-09-01 13:25:52,065 [erSimpleAppMain] DEBUG AbstractRegion - localhost
adding destination: queue://files/dead
2014-09-01 13:25:52,081 [erSimpleAppMain] DEBUG TaskRunnerFactory -
Initialized TaskRunnerFactory[ActiveMQ BrokerService[localhost] Task] using
ExecutorService: null
2014-09-01 13:25:52,098 [erSimpleAppMain] WARN LevelDBClient - DB operation
failed. (entering recovery mode)
2014-09-01 13:25:52,099 [erSimpleAppMain] DEBUG LevelDBClient -
java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:966)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:962)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$9.apply(LevelDBClient.scala:1038)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$9.apply(LevelDBClient.scala:1037)
at
org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:309)
at
org.apache.activemq.leveldb.LevelDBClient$RichDB.cursorRange(LevelDBClient.scala:311)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply$mcV$sp(LevelDBClient.scala:1037)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1037)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1037)
at
org.apache.activemq.leveldb.LevelDBClient.usingIndex(LevelDBClient.scala:760)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$retryUsingIndex$1.apply(LevelDBClient.scala:766)
at
org.apache.activemq.leveldb.util.RetrySupport$.retry(RetrySupport.scala:38)
at org.apache.activemq.leveldb.LevelDBClient.retry(LevelDBClient.scala:457)
at
org.apache.activemq.leveldb.LevelDBClient.retryUsingIndex(LevelDBClient.scala:766)
at
org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1036)
at
org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:962)
at org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:633)
at
org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:643)
at org.apache.activemq.broker.region.Queue.initialize(Queue.java:381)
at
org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:87)
--
View this message in context: http://activemq.2283324.n4.nabble.com/LevelDBClient-operation-failed-NullPointerException-after-entering-recovery-mode-tp4685208.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: LevelDBClient operation failed. NullPointerException after
entering recovery mode
Posted by nfx <se...@gmail.com>.
The problem is that it's not possible to reproduce this issue. It happens
once a week and log messages don't hold any errors.
Some more insights to problem:
- XXX.index and XXX.log files exist
- XXX.index probably has some dirty changes inside
- XXX.log which is supposed to be used for recovery, is filled with 100Mb
of NULL bytes.
Tried multiple ways of hacking around it to recover data, but something is
really wrong there.
-----
Best regards,
*Serge*
--
View this message in context: http://activemq.2283324.n4.nabble.com/LevelDBClient-operation-failed-NullPointerException-after-entering-recovery-mode-tp4685208p4685266.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: LevelDBClient operation failed. NullPointerException after entering
recovery mode
Posted by Timothy Bish <ta...@gmail.com>.
On 09/01/2014 10:09 AM, nfx wrote:
> Once in a while we're getting following exception in AMQ logs and there is no
> other way than purging the queue. How can we overcome this issue? Is it okay
> to use LevelDB store now? Any answers?
>
> 2014-09-01 13:25:52,065 [erSimpleAppMain] DEBUG AbstractRegion - localhost
> adding destination: queue://files/dead
> 2014-09-01 13:25:52,081 [erSimpleAppMain] DEBUG TaskRunnerFactory -
> Initialized TaskRunnerFactory[ActiveMQ BrokerService[localhost] Task] using
> ExecutorService: null
> 2014-09-01 13:25:52,098 [erSimpleAppMain] WARN LevelDBClient - DB operation
> failed. (entering recovery mode)
> 2014-09-01 13:25:52,099 [erSimpleAppMain] DEBUG LevelDBClient -
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:966)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:962)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$9.apply(LevelDBClient.scala:1038)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$9.apply(LevelDBClient.scala:1037)
> at
> org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:309)
> at
> org.apache.activemq.leveldb.LevelDBClient$RichDB.cursorRange(LevelDBClient.scala:311)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply$mcV$sp(LevelDBClient.scala:1037)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1037)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1.apply(LevelDBClient.scala:1037)
> at
> org.apache.activemq.leveldb.LevelDBClient.usingIndex(LevelDBClient.scala:760)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$retryUsingIndex$1.apply(LevelDBClient.scala:766)
> at
> org.apache.activemq.leveldb.util.RetrySupport$.retry(RetrySupport.scala:38)
> at org.apache.activemq.leveldb.LevelDBClient.retry(LevelDBClient.scala:457)
> at
> org.apache.activemq.leveldb.LevelDBClient.retryUsingIndex(LevelDBClient.scala:766)
> at
> org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1036)
> at
> org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:962)
> at org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:633)
> at
> org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:643)
> at org.apache.activemq.broker.region.Queue.initialize(Queue.java:381)
> at
> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:87)
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/LevelDBClient-operation-failed-NullPointerException-after-entering-recovery-mode-tp4685208.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
LevelDB is the newest store implemented in ActiveMQ and as such it's
evolving with each release. I know that some issues of this type have
been fixed in recent releases so it would be good to test against the
latest release if you aren't already using that. If it happens in the
latest release then it'd be good to try and create a test that can
reproduce it if possible as that will provide the groundwork for someone
to investigate and fix it.
--
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.redhat.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/