[jira] [Created] (AMQ-6568) Consumers stop consuming

["Andreas Gies (JIRA)" <ji...@apache.org>]

Andreas Gies created AMQ-6568:
---------------------------------

             Summary: Consumers stop consuming 
                 Key: AMQ-6568
                 URL: https://issues.apache.org/jira/browse/AMQ-6568
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker, JMS client
    Affects Versions: 5.14.2
         Environment: CentOS 6.5, Java 7, 64-bit
            Reporter: Andreas Gies


We are working on a fairly large distributed project using ActiveMQ as our JMS backend. 
Currently we are using Version 5.14.2. 

The application consists of many containers communicating over AMQ.
All JMS communication endpoints are implemented with Camel 2.17.3. 

The container itself is an OSGi container on top of Apache Felix, all implemented in Scala.
The production version is based on Scala 2.10 / JDK 7. The next version will be Scala 2.11 / JDK 8. 
We are not using Karaf, but have our own bundles to implement container configuration 
and distribution.

The overall install base is roughly 2500 containers in as many local site connected to the 
central data center. The data center does not run AMQ, but a different JMS provider. All local 
containers use AMQ und bridge into the data center. 


Our problem is that we observe in ~1% of our containers that messages are not being delivered 
from AMQ. The Queue Size is > 0 and the consuming Camel Route is started, but no message 
is dispatched. Message producers can still produce messages, but none are dispatched. 

In such a scenario no messages are dispatched to any consumers. Restating the route bundles 
does not have an effect, only restarting the containers rectifies the problem. 

Our investigation so far has not given any hints what might be special regarding the circumstances 
when we encounter the problem. We have taken thread dumps from containers in that state, but 
have not been able to spot the problem. 


The initial question is whether someone in the community could provide us a hint how we could 
narrow down the problem. 

For today I am attaching our AMQ configuration file and also one of latest thread dumps. 






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)