You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by gkh <go...@pointclickcare.com> on 2014/06/30 22:17:45 UTC
AMQ 5.6 IllegalStateException: Cannot add producer
After upgrading from v5.5.1 to 5.6, I'm now seeing the following for each
message being sent; none of these messages seem to reach the consumer. Our
app doesn't emit these messages when running with AMQ 5.5.1.
INFO | jvm 1 | 2014/06/21 06:58:34 | java.lang.IllegalStateException:
Cannot add a producer to a connection that had not been registered:
ID:QA01-APPB-61010-1403294484270-1792:0
I've already searched Google and AMQ-User archives for anything similar and
I did find similar questions, but no clear answers/resolutions. (Most of
the hits were code commits where the error text was added.)
It feels like there is something 'simple' missing in the config, but I don't
see what it is. Would someone please help me resolve this?
There are multiple message producers (Spring-based tomcat web apps), a
single AMQ broker, and a single consumer (also a Spring-based tomcat web
app); each of which are on separate machines.
Since there is only one broker, I don't think the 'Network of Brokers'
problem applies. (Please correct me if I'm wrong.)
A little history: In production, we are having issues where ActiveMQ 5.5.1
stops accepting messages and hangs, requiring a (Windows) service restart in
order to attempt to get messages to start flowing again. Sometimes, an O/S
restart is required. We are looking at moving up to 5.6 now, as it seems to
address a number of performance and memory leak issues that appear to be
plaguing our current install. We will be upgrading to more current version
(5.9.1 or later) at the end of our next major release cycle.
The AMQ .JAR files included in the web apps (both the Consumer and Producer)
are v5.3.0.fuse (in looking at the META-INF/MANIFEST.MF). We will upgrade
these .JAR files to the current version when we roll out the major upgrade.
Thanks, in advance for any insight.
The full stacktrace from one of the related events:
INFO | jvm 1 | 2014/06/21 06:58:34 | WARN | Failed to add Connection
ID:QA01-APPB-61010-1403294484270-1792:0, reason:
javax.jms.InvalidClientIDException: Broker: QAAPP01A - Client:
NC_default_outbound already connected from tcp://10.0.x.x:49439
INFO | jvm 1 | 2014/06/21 06:58:34 | WARN | Async error occurred:
javax.jms.InvalidClientIDException: Broker: QAAPP01A - Client:
NC_default_outbound already connected from tcp://10.0.x.x:49439
INFO | jvm 1 | 2014/06/21 06:58:34 |
javax.jms.InvalidClientIDException: Broker: QAAPP01A - Client:
NC_default_outbound already connected from tcp://10.0.x.x:49439
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.region.RegionBroker.addConnection(RegionBroker.java:223)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:85)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:65)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:85)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:85)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:91)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:715)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:79)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
java.lang.Thread.run(Thread.java:745)
INFO | jvm 1 | 2014/06/21 06:58:34 | WARN | Async error occurred:
java.lang.IllegalStateException: Cannot add a producer to a connection that
had not been registered: ID:QA01-APPB-61010-1403294484270-1792:0
INFO | jvm 1 | 2014/06/21 06:58:34 | java.lang.IllegalStateException:
Cannot add a producer to a connection that had not been registered:
ID:QA01-APPB-61010-1403294484270-1792:0
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.processAddProducer(TransportConnection.java:498)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.command.ProducerInfo.visit(ProducerInfo.java:105)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
java.lang.Thread.run(Thread.java:745)
INFO | jvm 1 | 2014/06/21 06:58:34 | WARN | Async error occurred:
java.lang.IllegalStateException: Cannot add a consumer to a connection that
had not been registered: ID:QA01-APPB-61010-1403294484270-1792:0
INFO | jvm 1 | 2014/06/21 06:58:34 | java.lang.IllegalStateException:
Cannot add a consumer to a connection that had not been registered:
ID:QA01-APPB-61010-1403294484270-1792:0
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.processAddConsumer(TransportConnection.java:548)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.command.ConsumerInfo.visit(ConsumerInfo.java:349)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204)
INFO | jvm 1 | 2014/06/21 06:58:34 | at
java.lang.Thread.run(Thread.java:745)
The AMQ 5.6 activemq.xml file:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>file:${activemq.base}/conf/messagebroker.properties</value>
</list>
</property>
</bean>
<broker useJmx="true" xmlns="http://activemq.apache.org/schema/core"
brokerName="${broker.name}" dataDirectory="${activemq.base}/data"
useVirtualTopics="false">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" producerFlowControl="false"
memoryLimit="10m" maxPageSize="5000" maxBrowsePageSize="5000">
<pendingQueuePolicy>
<fileQueueCursor />
</pendingQueuePolicy>
</policyEntry>
<policyEntry topic=">" producerFlowControl="false"
memoryLimit="10m" >
<pendingSubscriberPolicy>
<fileCursor />
</pendingSubscriberPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="true"
connectorPort="${broker.jmx.port}" />
</managementContext>
<networkConnectors>
</networkConnectors>
<amq:persistenceFactory>
<amq:journalPersistenceAdapterFactory journalLogFiles="5"
dataDirectory="${activemq.home}/journal" dataSource="#mssql-ds"
adapter="#adapter" useDatabaseLock="false"/>
</amq:persistenceFactory>
<sslContext>
<sslContext
keyStore="file:${activemq.base}/${broker.keystore.file}"
keyStorePassword="${broker.keystore.password}"
trustStore="file:${activemq.base}/${broker.truststore.file}"
trustStorePassword="${broker.truststore.password}"/>
</sslContext>
<systemUsage>
<systemUsage sendFailIfNoSpace="true">
<memoryUsage>
<memoryUsage limit="800mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="10gb" name="store"/>
</storeUsage>
<tempUsage>
<tempUsage limit="10gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire"
uri="nio://0.0.0.0:${broker.tcp.port}" />
</transportConnectors>
</broker>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent" >
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL"
value="vm://${broker.name}?create=false&waitForStart=10000" />
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
</property>
</bean>
<commandAgent xmlns="http://activemq.apache.org/schema/core"
brokerUrl="vm://${broker.name}" username="${activemq.username}"
password="${activemq.password}"/>
<import resource="jetty.xml"/>
<bean id="adapter"
class="org.apache.activemq.store.jdbc.adapter.ImageBasedJDBCAdaptor" />
<bean id="mssql-ds" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.inet.tds.TdsDriver"/>
<property name="url"
value="jdbc:inetdae:${mssql.db.server_name}:${mssql.db.server_port}?database=${mssql.db.database_name}"/>
<property name="username" value="${mssql.db.user}"/>
<property name="password" value="${mssql.db.password}"/>
<property name="testOnBorrow" value="true"/>
<property name="validationQuery" value="SELECT 1"/>
</bean>
</beans>
--
View this message in context: http://activemq.2283324.n4.nabble.com/AMQ-5-6-IllegalStateException-Cannot-add-producer-tp4682674.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: AMQ 5.6 IllegalStateException: Cannot add producer
Posted by gkh <go...@pointclickcare.com>.
Sorry for the mis-information and noise previously on this thread.
I've dug into this more and found out a few more details: our producer app
(Spring-base web app) spins up an embedded broker (v5.3-fuse libraries) that
'receives' messages from the app and forwards those messages on to the 'hub'
broker (that we are looking to upgrade to 5.6, from 5.5.1).
So I was wrong in my initial post that the 'network of brokers' doesn't
apply.
The embedded brokers are configured to accept messages from the app either
via a VM://<broker name> or openwire tcp://0.0.0.0:<local messaging port>
transport connectors. This embedded broker has only one configured
networkConnector:
<amq:networkConnector uri="static(tcp://<master_broker>)"
name="NotificationsBridge" dynamicOnly="false" conduitSubscriptions="true"
decreaseNetworkConsumerPriority="false" networkTTL="12">
<amq:dynamicallyIncludedDestinations>
<amq:queue physicalName="Notifications"/>
</amq:dynamicallyIncludedDestinations>
</networkConnector>
When I look at the warning messages in the logs, I see that the client name
(NC_default_outbound) is associated with the first embedded broker to
connect, not the name indicated on the networkConnector config.
So a few more questions:
1) Does ActiveMQ 5.6 support a network of brokers where the spokes are v5.3
brokers?
2) We have a pool of application servers (up to 100 in one environment)
using the above posted config. I'm guessing that there will be issues if
all the embedded brokers connect to the hub with the same name, correct?
3) How does one re-configure the embedded brokers to send a unique client
name, if the name attribute on the networkConnector element doesn't work?
Thanks in advance for any help or advice.
--
View this message in context: http://activemq.2283324.n4.nabble.com/AMQ-5-6-IllegalStateException-Cannot-add-producer-tp4682674p4683288.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: AMQ 5.6 IllegalStateException: Cannot add producer
Posted by gkh <go...@pointclickcare.com>.
Could anyone help with this problem? Is there a configuration setting within
ActiveMQ to eliminate this warning?
One more 'interesting' fact I learned last night is that most (NOT all)
messages coming from the QA01-APPB server are delayed processing by up to 5
hours: the are submitted/sent to ActiveMQ without error, but are never
picked up by the single consumer.
Thanks.
--
View this message in context: http://activemq.2283324.n4.nabble.com/AMQ-5-6-IllegalStateException-Cannot-add-producer-tp4682674p4683022.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.