You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by DV <vi...@gmail.com> on 2012/07/16 03:15:06 UTC

ArrayIndexOutOfBoundsException with mKahaDB in 5.6.0

We've recently started testing ActiveMQ 5.6.0 with mKahaDB adapter,
configured as suggested on the site:

            <mKahaDB directory="/data/activemq">
                <filteredPersistenceAdapters>
                    <filteredKahaDB perDestination="true">
                        <persistenceAdapter>
                            <kahaDB journalMaxFileLength="256mb" />
                        </persistenceAdapter>
                    </filteredKahaDB>
                </filteredPersistenceAdapters>
            </mKahaDB>

We're using it for publishing and there's a variety of producers and
consumers connecting to the broker from multitude of environments. At
any single point the size of the datastore could be from under 1GB to
about 100GB, with 200GB a hard maximum.

After the change we've started seeing the following error message that
effectively takes the whole broker down. The broker would work fine
for a few days or weeks, and then break:

2012-07-12 13:22:22,982 |  INFO | Using Persistence Adapter:
MultiKahaDBPersistenceAdapter[/data/activemq][]
2012-07-12 13:22:23,019 |  INFO | JMX consoles can connect to
service:jmx:rmi://localhost:1100/jndi/rmi://localhost:1101/jmxrmi
2012-07-12 13:22:26,090 |  INFO | KahaDB is version 4
2012-07-12 13:22:26,114 |  INFO | Recovering from the journal ...
2012-07-12 13:22:26,115 |  INFO | Recovery replayed 4 operations from
the journal in 0.011 seconds.
2012-07-12 13:22:26,128 | ERROR | Failed to start ActiveMQ JMS Message
Broker (broker-publishing, null). Reason:
java.lang.ArrayIndexOutOfBoundsException: 0
java.lang.ArrayIndexOutOfBoundsException: 0
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.registerExistingAdapter(MultiKahaDBPersistenceAdapter.java:283)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.findAndRegisterExistingAdapters(MultiKahaDBPersistenceAdapter.java:275)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.start(MultiKahaDBPersistenceAdapter.java:257)
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:525)
        at org.apache.activemq.broker.BrokerService.autoStart(BrokerService.java:494)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:340)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:293)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1413)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:108)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:108)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:72)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:115)
        at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:74)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:148)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:90)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:257)
        at org.apache.activemq.console.Main.main(Main.java:111)
2012-07-12 13:22:26,131 |  INFO | ActiveMQ Message Broker
(broker-publishing, null) is shutting down
2012-07-12 13:22:26,131 |  INFO | Connector nio Stopped
2012-07-12 13:22:26,140 |  INFO | ActiveMQ JMS Message Broker
(broker-publishing, null) stopped

We have been using the same broker with standard KahaDB configuration
for a long time without any issues. We've rolled back to 5.6.0 without
mKahaDB enabled and so far aren't seeing this issue.

I've found [#AMQ-3528] that describes the same error, but it's from a
while back and for a different type of datastore.

Should I open an issue? Has anyone successfully switched their
broker(s) to mKahaDB?

-- 
Best regards, Dmitriy V.

Re: ArrayIndexOutOfBoundsException with mKahaDB in 5.6.0

Posted by Timothy Bish <ta...@gmail.com>.
On Sun, 2012-07-15 at 18:15 -0700, DV wrote: 
> We've recently started testing ActiveMQ 5.6.0 with mKahaDB adapter,
> configured as suggested on the site:
> 
>             <mKahaDB directory="/data/activemq">
>                 <filteredPersistenceAdapters>
>                     <filteredKahaDB perDestination="true">
>                         <persistenceAdapter>
>                             <kahaDB journalMaxFileLength="256mb" />
>                         </persistenceAdapter>
>                     </filteredKahaDB>
>                 </filteredPersistenceAdapters>
>             </mKahaDB>
> 
> We're using it for publishing and there's a variety of producers and
> consumers connecting to the broker from multitude of environments. At
> any single point the size of the datastore could be from under 1GB to
> about 100GB, with 200GB a hard maximum.
> 
> After the change we've started seeing the following error message that
> effectively takes the whole broker down. The broker would work fine
> for a few days or weeks, and then break:
> 
> 2012-07-12 13:22:22,982 |  INFO | Using Persistence Adapter:
> MultiKahaDBPersistenceAdapter[/data/activemq][]
> 2012-07-12 13:22:23,019 |  INFO | JMX consoles can connect to
> service:jmx:rmi://localhost:1100/jndi/rmi://localhost:1101/jmxrmi
> 2012-07-12 13:22:26,090 |  INFO | KahaDB is version 4
> 2012-07-12 13:22:26,114 |  INFO | Recovering from the journal ...
> 2012-07-12 13:22:26,115 |  INFO | Recovery replayed 4 operations from
> the journal in 0.011 seconds.
> 2012-07-12 13:22:26,128 | ERROR | Failed to start ActiveMQ JMS Message
> Broker (broker-publishing, null). Reason:
> java.lang.ArrayIndexOutOfBoundsException: 0
> java.lang.ArrayIndexOutOfBoundsException: 0
>         at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.registerExistingAdapter(MultiKahaDBPersistenceAdapter.java:283)
>         at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.findAndRegisterExistingAdapters(MultiKahaDBPersistenceAdapter.java:275)
>         at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.start(MultiKahaDBPersistenceAdapter.java:257)
>         at org.apache.activemq.broker.BrokerService.start(BrokerService.java:525)
>         at org.apache.activemq.broker.BrokerService.autoStart(BrokerService.java:494)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:340)
>         at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:293)
>         at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)
>         at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
>         at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1413)
>         at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
>         at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
>         at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
>         at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
>         at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
>         at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
>         at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
>         at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
>         at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
>         at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
>         at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
>         at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:108)
>         at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:108)
>         at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:72)
>         at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
>         at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
>         at org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:115)
>         at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:74)
>         at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:148)
>         at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:90)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.activemq.console.Main.runTaskClass(Main.java:257)
>         at org.apache.activemq.console.Main.main(Main.java:111)
> 2012-07-12 13:22:26,131 |  INFO | ActiveMQ Message Broker
> (broker-publishing, null) is shutting down
> 2012-07-12 13:22:26,131 |  INFO | Connector nio Stopped
> 2012-07-12 13:22:26,140 |  INFO | ActiveMQ JMS Message Broker
> (broker-publishing, null) stopped
> 
> We have been using the same broker with standard KahaDB configuration
> for a long time without any issues. We've rolled back to 5.6.0 without
> mKahaDB enabled and so far aren't seeing this issue.
> 
> I've found [#AMQ-3528] that describes the same error, but it's from a
> while back and for a different type of datastore.
> 
> Should I open an issue? Has anyone successfully switched their
> broker(s) to mKahaDB?
> 

This is fixed by https://issues.apache.org/jira/browse/AMQ-3841 you'd
need to pick up a SNAPHOT build of 5.7.0 for this fix. 

-- 
Tim Bish
Sr Software Engineer | FuseSource Corp
tim.bish@fusesource.com | www.fusesource.com
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/