You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Ehsan Zaery Moghaddam <za...@gmail.com> on 2019/11/22 12:39:16 UTC
Cannot initialize queue:Resource temporarily unavailable
Hi
We're using WildFly 18 which is using ActiveMQ 2.10.1. Since a few days
ago, we're getting the following exception whenever we try to start the
WildFly on some of our servers. It seems to be on some random servers, but
those that have this issue, continuously throw the same exception. It's
from the JNI part of the artemis component.
We tried running a fresh installation of WildFly and even tried to use the
older version that we had before (WildFly 13) without deploying any
application but nothing changed. So it shouldn't be related to the ActiveMQ
or WildFly but it's also not clear for us what that part of the code is
trying to do with file system. So I thought maybe you guys can help us in
this regard.
The artemis component is using libaio linux library (as mentioned in their
GitHub page). So we checked if we had any update on that library on given
servers and we find out that it didn't get any update since few months ago.
Any help would be really appreciated.
WildFly 18 startup exception:
2019-11-21 12:15:47,076 INFO [ServerService Thread
Pool -- 72] [server] AMQ221000: live Message Broker is starting with
configuration Broker Configuration
(clustered=true,journalDirectory=/opt/wildfly/standalone/data/activemq/journal,bindingsDirectory=/opt/wildfly/standalone/data/activemq/bindings,largeMessagesDirectory=/opt/wildfly/standalone/data/active
mq/largemessages,pagingDirectory=/opt/wildfly/standalone/data/activemq/paging)
2019-11-21 12:15:47,145 INFO [ServerService Thread
Pool -- 72] [server] AMQ221012: Using AIO Journal
2019-11-21 12:15:47,209 INFO [MSC service thread
1-3] [GlobalComponentRegistry] ISPN000128: Infinispan version: Infinispan
'Infinity Minus ONE +2' 9.4.16.Final
2019-11-21 12:15:47,283 INFO [ServerService Thread
Pool -- 72] [server] AMQ221043: Protocol module found: [artemis-server].
Adding protocol support for: CORE
2019-11-21 12:15:47,284 INFO [ServerService Thread
Pool -- 72] [server] AMQ221043: Protocol module found:
[artemis-amqp-protocol]. Adding protocol support for: AMQP
2019-11-21 12:15:47,284 INFO [ServerService Thread
Pool -- 72] [server] AMQ221043: Protocol module found:
[artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2019-11-21 12:15:47,285 INFO [ServerService Thread
Pool -- 72] [server] AMQ221043: Protocol module found:
[artemis-stomp-protocol]. Adding protocol support for: STOMP
2019-11-21 12:15:47,361 ERROR [ServerService Thread
Pool -- 72] [server] AMQ224000: Failure in initialisation:
java.lang.RuntimeException: Cannot initialize queue:Resource temporarily
unavailable
at
org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
Method)
at
org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
at
org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
at
org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
at
org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
at
org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
at
org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
at
org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
at
org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
at
org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at
org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
2019-11-21 12:15:47,365 ERROR [ServerService Thread
Pool -- 72] [fail] MSC000001: Failed to start service
jboss.messaging-activemq.default.jms.manager:
org.jboss.msc.service.StartException in service
jboss.messaging-activemq.default.jms.manager: java.lang.RuntimeException:
Cannot initialize queue:Resource temporarily unavailable
at
org.wildfly.extension.messaging.activemq.jms.JMSService.lambda$doStart$0(JMSService.java:147)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.callActivationFailureListeners(ActiveMQServerImpl.java:2224)
at
org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:83)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
at
org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
at
org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
at
org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
at
org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at
org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.RuntimeException: Cannot initialize queue:Resource
temporarily unavailable
at
org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
Method)
at
org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
at
org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
at
org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
at
org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
at
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
at
org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
... 14 more
Regards
Ehsan
Re: Cannot initialize queue:Resource temporarily unavailable
Posted by vinay <ka...@gmail.com>.
Hi Jbertram,
Same error mentioned by Justin is occuring in jboss 7.2 and above version,
is there any solution for this as the broker.xml is packed inside the jar
file.
Thanks,
Vinay
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: Cannot initialize queue:Resource temporarily unavailable
Posted by Justin Bertram <jb...@apache.org>.
One more thing you can try...
The default value for <journal-max-io> is 4096 so you could specify
something smaller than that, e.g. <journal-max-io>2048</journal-max-io>.
This value is used in the call to io_queue_init.
Justin
On Fri, Nov 22, 2019 at 11:06 AM Justin Bertram <jb...@apache.org> wrote:
> You could also work-around this issue by specifying
> <journal-type>NIO</journal-type> in your broker.xml. The <journal-type>
> defaults to ASYNCIO which will use the native libaio for best performance.
>
>
> Justin
>
> On Fri, Nov 22, 2019 at 11:04 AM Justin Bertram <jb...@apache.org>
> wrote:
>
>> The last bit of Java code before entering the JNI code is here [1]. The
>> JNI code [2] attempts to invoke io_queue_init [3] from libaio which results
>> in the "Resource temporarily unavailable" error. So it appears there's
>> something environmental which is preventing libaio from doing its work. My
>> searching indicates you can run "cat /proc/sys/fs/aio-max-nr" to inspect
>> the maximum number of allowable concurrent requests and potentially
>> increase this value to resolve the issue.
>>
>>
>> Justin
>>
>> [1]
>> https://github.com/apache/activemq-artemis-native/blob/master/src/main/java/org/apache/activemq/artemis/nativo/jlibaio/LibaioContext.java#L161
>> [2]
>> https://github.com/apache/activemq-artemis-native/blob/master/src/main/c/org_apache_activemq_artemis_nativo_jlibaio_LibaioContext.c#L528
>> [3]
>> https://manpages.debian.org/testing/libaio-dev/io_queue_init.2.en.html
>>
>> On Fri, Nov 22, 2019 at 8:12 AM Ehsan Zaery Moghaddam <
>> zaerymoghaddam@gmail.com> wrote:
>>
>>> Hi
>>>
>>> We're using WildFly 18 which is using ActiveMQ 2.10.1. Since a few days
>>> ago, we're getting the following exception whenever we try to start the
>>> WildFly on some of our servers. It seems to be on some random servers,
>>> but
>>> those that have this issue, continuously throw the same exception. It's
>>> from the JNI part of the artemis component.
>>>
>>> We tried running a fresh installation of WildFly and even tried to use
>>> the
>>> older version that we had before (WildFly 13) without deploying any
>>> application but nothing changed. So it shouldn't be related to the
>>> ActiveMQ
>>> or WildFly but it's also not clear for us what that part of the code is
>>> trying to do with file system. So I thought maybe you guys can help us in
>>> this regard.
>>>
>>> The artemis component is using libaio linux library (as mentioned in
>>> their
>>> GitHub page). So we checked if we had any update on that library on given
>>> servers and we find out that it didn't get any update since few months
>>> ago.
>>>
>>> Any help would be really appreciated.
>>>
>>> WildFly 18 startup exception:
>>>
>>> 2019-11-21 12:15:47,076 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221000: live Message Broker is starting with
>>> configuration Broker Configuration
>>>
>>> (clustered=true,journalDirectory=/opt/wildfly/standalone/data/activemq/journal,bindingsDirectory=/opt/wildfly/standalone/data/activemq/bindings,largeMessagesDirectory=/opt/wildfly/standalone/data/active
>>>
>>> mq/largemessages,pagingDirectory=/opt/wildfly/standalone/data/activemq/paging)
>>> 2019-11-21 12:15:47,145 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221012: Using AIO Journal
>>> 2019-11-21 12:15:47,209 INFO [MSC service thread
>>> 1-3] [GlobalComponentRegistry] ISPN000128: Infinispan version: Infinispan
>>> 'Infinity Minus ONE +2' 9.4.16.Final
>>> 2019-11-21 12:15:47,283 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221043: Protocol module found: [artemis-server].
>>> Adding protocol support for: CORE
>>> 2019-11-21 12:15:47,284 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221043: Protocol module found:
>>> [artemis-amqp-protocol]. Adding protocol support for: AMQP
>>> 2019-11-21 12:15:47,284 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221043: Protocol module found:
>>> [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
>>> 2019-11-21 12:15:47,285 INFO [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ221043: Protocol module found:
>>> [artemis-stomp-protocol]. Adding protocol support for: STOMP
>>> 2019-11-21 12:15:47,361 ERROR [ServerService
>>> Thread
>>> Pool -- 72] [server] AMQ224000: Failure in initialisation:
>>> java.lang.RuntimeException: Cannot initialize queue:Resource temporarily
>>> unavailable
>>> at
>>>
>>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
>>> Method)
>>> at
>>>
>>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
>>> at
>>>
>>> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
>>> at
>>>
>>> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
>>> at
>>>
>>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
>>> at
>>>
>>> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
>>> at java.lang.Thread.run(Thread.java:748)
>>> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
>>>
>>> 2019-11-21 12:15:47,365 ERROR [ServerService
>>> Thread
>>> Pool -- 72] [fail] MSC000001: Failed to start service
>>> jboss.messaging-activemq.default.jms.manager:
>>> org.jboss.msc.service.StartException in service
>>> jboss.messaging-activemq.default.jms.manager: java.lang.RuntimeException:
>>> Cannot initialize queue:Resource temporarily unavailable
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService.lambda$doStart$0(JMSService.java:147)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.callActivationFailureListeners(ActiveMQServerImpl.java:2224)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:83)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
>>> at
>>>
>>> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
>>> at
>>>
>>> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at
>>>
>>> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
>>> at
>>>
>>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
>>> at java.lang.Thread.run(Thread.java:748)
>>> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
>>> Caused by: java.lang.RuntimeException: Cannot initialize queue:Resource
>>> temporarily unavailable
>>> at
>>>
>>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
>>> Method)
>>> at
>>>
>>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
>>> at
>>>
>>> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
>>> at
>>>
>>> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
>>> at
>>>
>>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
>>> at
>>>
>>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
>>> ... 14 more
>>>
>>> Regards
>>> Ehsan
>>>
>>
Re: Cannot initialize queue:Resource temporarily unavailable
Posted by Justin Bertram <jb...@apache.org>.
You could also work-around this issue by specifying
<journal-type>NIO</journal-type> in your broker.xml. The <journal-type>
defaults to ASYNCIO which will use the native libaio for best performance.
Justin
On Fri, Nov 22, 2019 at 11:04 AM Justin Bertram <jb...@apache.org> wrote:
> The last bit of Java code before entering the JNI code is here [1]. The
> JNI code [2] attempts to invoke io_queue_init [3] from libaio which results
> in the "Resource temporarily unavailable" error. So it appears there's
> something environmental which is preventing libaio from doing its work. My
> searching indicates you can run "cat /proc/sys/fs/aio-max-nr" to inspect
> the maximum number of allowable concurrent requests and potentially
> increase this value to resolve the issue.
>
>
> Justin
>
> [1]
> https://github.com/apache/activemq-artemis-native/blob/master/src/main/java/org/apache/activemq/artemis/nativo/jlibaio/LibaioContext.java#L161
> [2]
> https://github.com/apache/activemq-artemis-native/blob/master/src/main/c/org_apache_activemq_artemis_nativo_jlibaio_LibaioContext.c#L528
> [3] https://manpages.debian.org/testing/libaio-dev/io_queue_init.2.en.html
>
> On Fri, Nov 22, 2019 at 8:12 AM Ehsan Zaery Moghaddam <
> zaerymoghaddam@gmail.com> wrote:
>
>> Hi
>>
>> We're using WildFly 18 which is using ActiveMQ 2.10.1. Since a few days
>> ago, we're getting the following exception whenever we try to start the
>> WildFly on some of our servers. It seems to be on some random servers, but
>> those that have this issue, continuously throw the same exception. It's
>> from the JNI part of the artemis component.
>>
>> We tried running a fresh installation of WildFly and even tried to use the
>> older version that we had before (WildFly 13) without deploying any
>> application but nothing changed. So it shouldn't be related to the
>> ActiveMQ
>> or WildFly but it's also not clear for us what that part of the code is
>> trying to do with file system. So I thought maybe you guys can help us in
>> this regard.
>>
>> The artemis component is using libaio linux library (as mentioned in their
>> GitHub page). So we checked if we had any update on that library on given
>> servers and we find out that it didn't get any update since few months
>> ago.
>>
>> Any help would be really appreciated.
>>
>> WildFly 18 startup exception:
>>
>> 2019-11-21 12:15:47,076 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221000: live Message Broker is starting with
>> configuration Broker Configuration
>>
>> (clustered=true,journalDirectory=/opt/wildfly/standalone/data/activemq/journal,bindingsDirectory=/opt/wildfly/standalone/data/activemq/bindings,largeMessagesDirectory=/opt/wildfly/standalone/data/active
>>
>> mq/largemessages,pagingDirectory=/opt/wildfly/standalone/data/activemq/paging)
>> 2019-11-21 12:15:47,145 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221012: Using AIO Journal
>> 2019-11-21 12:15:47,209 INFO [MSC service thread
>> 1-3] [GlobalComponentRegistry] ISPN000128: Infinispan version: Infinispan
>> 'Infinity Minus ONE +2' 9.4.16.Final
>> 2019-11-21 12:15:47,283 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221043: Protocol module found: [artemis-server].
>> Adding protocol support for: CORE
>> 2019-11-21 12:15:47,284 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221043: Protocol module found:
>> [artemis-amqp-protocol]. Adding protocol support for: AMQP
>> 2019-11-21 12:15:47,284 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221043: Protocol module found:
>> [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
>> 2019-11-21 12:15:47,285 INFO [ServerService
>> Thread
>> Pool -- 72] [server] AMQ221043: Protocol module found:
>> [artemis-stomp-protocol]. Adding protocol support for: STOMP
>> 2019-11-21 12:15:47,361 ERROR [ServerService
>> Thread
>> Pool -- 72] [server] AMQ224000: Failure in initialisation:
>> java.lang.RuntimeException: Cannot initialize queue:Resource temporarily
>> unavailable
>> at
>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
>> Method)
>> at
>>
>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
>> at
>>
>> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
>> at
>>
>> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
>> at
>>
>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
>> at
>>
>> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
>> at java.lang.Thread.run(Thread.java:748)
>> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
>>
>> 2019-11-21 12:15:47,365 ERROR [ServerService
>> Thread
>> Pool -- 72] [fail] MSC000001: Failed to start service
>> jboss.messaging-activemq.default.jms.manager:
>> org.jboss.msc.service.StartException in service
>> jboss.messaging-activemq.default.jms.manager: java.lang.RuntimeException:
>> Cannot initialize queue:Resource temporarily unavailable
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService.lambda$doStart$0(JMSService.java:147)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.callActivationFailureListeners(ActiveMQServerImpl.java:2224)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:83)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
>> at
>>
>> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
>> at
>>
>> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>>
>> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
>> at
>>
>> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
>> at java.lang.Thread.run(Thread.java:748)
>> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
>> Caused by: java.lang.RuntimeException: Cannot initialize queue:Resource
>> temporarily unavailable
>> at
>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
>> Method)
>> at
>>
>> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
>> at
>>
>> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
>> at
>>
>> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
>> at
>>
>> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
>> at
>>
>> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
>> ... 14 more
>>
>> Regards
>> Ehsan
>>
>
Re: Cannot initialize queue:Resource temporarily unavailable
Posted by Justin Bertram <jb...@apache.org>.
The last bit of Java code before entering the JNI code is here [1]. The
JNI code [2] attempts to invoke io_queue_init [3] from libaio which results
in the "Resource temporarily unavailable" error. So it appears there's
something environmental which is preventing libaio from doing its work. My
searching indicates you can run "cat /proc/sys/fs/aio-max-nr" to inspect
the maximum number of allowable concurrent requests and potentially
increase this value to resolve the issue.
Justin
[1]
https://github.com/apache/activemq-artemis-native/blob/master/src/main/java/org/apache/activemq/artemis/nativo/jlibaio/LibaioContext.java#L161
[2]
https://github.com/apache/activemq-artemis-native/blob/master/src/main/c/org_apache_activemq_artemis_nativo_jlibaio_LibaioContext.c#L528
[3] https://manpages.debian.org/testing/libaio-dev/io_queue_init.2.en.html
On Fri, Nov 22, 2019 at 8:12 AM Ehsan Zaery Moghaddam <
zaerymoghaddam@gmail.com> wrote:
> Hi
>
> We're using WildFly 18 which is using ActiveMQ 2.10.1. Since a few days
> ago, we're getting the following exception whenever we try to start the
> WildFly on some of our servers. It seems to be on some random servers, but
> those that have this issue, continuously throw the same exception. It's
> from the JNI part of the artemis component.
>
> We tried running a fresh installation of WildFly and even tried to use the
> older version that we had before (WildFly 13) without deploying any
> application but nothing changed. So it shouldn't be related to the ActiveMQ
> or WildFly but it's also not clear for us what that part of the code is
> trying to do with file system. So I thought maybe you guys can help us in
> this regard.
>
> The artemis component is using libaio linux library (as mentioned in their
> GitHub page). So we checked if we had any update on that library on given
> servers and we find out that it didn't get any update since few months ago.
>
> Any help would be really appreciated.
>
> WildFly 18 startup exception:
>
> 2019-11-21 12:15:47,076 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221000: live Message Broker is starting with
> configuration Broker Configuration
>
> (clustered=true,journalDirectory=/opt/wildfly/standalone/data/activemq/journal,bindingsDirectory=/opt/wildfly/standalone/data/activemq/bindings,largeMessagesDirectory=/opt/wildfly/standalone/data/active
>
> mq/largemessages,pagingDirectory=/opt/wildfly/standalone/data/activemq/paging)
> 2019-11-21 12:15:47,145 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221012: Using AIO Journal
> 2019-11-21 12:15:47,209 INFO [MSC service thread
> 1-3] [GlobalComponentRegistry] ISPN000128: Infinispan version: Infinispan
> 'Infinity Minus ONE +2' 9.4.16.Final
> 2019-11-21 12:15:47,283 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221043: Protocol module found: [artemis-server].
> Adding protocol support for: CORE
> 2019-11-21 12:15:47,284 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221043: Protocol module found:
> [artemis-amqp-protocol]. Adding protocol support for: AMQP
> 2019-11-21 12:15:47,284 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221043: Protocol module found:
> [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
> 2019-11-21 12:15:47,285 INFO [ServerService Thread
> Pool -- 72] [server] AMQ221043: Protocol module found:
> [artemis-stomp-protocol]. Adding protocol support for: STOMP
> 2019-11-21 12:15:47,361 ERROR [ServerService Thread
> Pool -- 72] [server] AMQ224000: Failure in initialisation:
> java.lang.RuntimeException: Cannot initialize queue:Resource temporarily
> unavailable
> at
> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
> Method)
> at
>
> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
> at
>
> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
> at
>
> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
> at
>
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
> at
>
> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
> at
>
> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at
>
> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
> at
>
> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
> at
>
> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
>
> 2019-11-21 12:15:47,365 ERROR [ServerService Thread
> Pool -- 72] [fail] MSC000001: Failed to start service
> jboss.messaging-activemq.default.jms.manager:
> org.jboss.msc.service.StartException in service
> jboss.messaging-activemq.default.jms.manager: java.lang.RuntimeException:
> Cannot initialize queue:Resource temporarily unavailable
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService.lambda$doStart$0(JMSService.java:147)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.callActivationFailureListeners(ActiveMQServerImpl.java:2224)
> at
>
> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:83)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:600)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:527)
> at
>
> org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:373)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:211)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:65)
> at
>
> org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:100)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at
>
> org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
> at
>
> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
> at
>
> org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> Caused by: java.lang.RuntimeException: Cannot initialize queue:Resource
> temporarily unavailable
> at
> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.newContext(Native
> Method)
> at
>
> org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.<init>(LibaioContext.java:161)
> at
>
> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.start(AIOSequentialFileFactory.java:260)
> at
>
> org.apache.activemq.artemis.core.journal.impl.JournalImpl.start(JournalImpl.java:2456)
> at
>
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.start(AbstractJournalStorageManager.java:1586)
> at
>
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2824)
> at
>
> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:64)
> ... 14 more
>
> Regards
> Ehsan
>