You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by TBuckel <th...@jdv.com> on 2008/02/14 01:41:23 UTC
Problems with Pure Master/Slave in AMQ 5.0.0
Hi all,
I am having trouble setting up a *stable* ActiveMQ Pure Master/Slave
topology.
Initially I have tried v4.1.1 which failed with an exception. I found an AMQ
JIRA ticket which said that Pure/Master slave didn't work in v4.1.1.
Ok, so I switched to AMQ 5.0.0, created 2 configs (master/slave, see end of
message) and ran two AMQ instances (on the same box) and most of the times
my test (see below) worked, but more often I get various error messages
like:
- On the slave:
ERROR Service - Async error occurred:
javax.jms.JMSException: Slave broker out of sync with master: Dispatched
message (ID:tbuckel-desktop-41814-1202886136210-0:0:565:1:1) was not in the
pending list
javax.jms.JMSException: Slave broker out of sync with master: Dispatched
message (ID:tbuckel-desktop-41814-1202886136210-0:0:565:1:1) was not in the
pending list
at
org.apache.activemq.broker.region.PrefetchSubscription.processMessageDispatchNotification(PrefetchSubscription.java:160)
at
org.apache.activemq.broker.region.AbstractRegion.processDispatchNotification(AbstractRegion.java:381)
at
org.apache.activemq.broker.region.RegionBroker.processDispatchNotification(RegionBroker.java:550)
at
org.apache.activemq.broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:201)
at
org.apache.activemq.broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:201)
at
org.apache.activemq.broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:201)
at
org.apache.activemq.broker.MutableBrokerFilter.processDispatchNotification(MutableBrokerFilter.java:211)
at
org.apache.activemq.broker.TransportConnection.processMessageDispatchNotification(TransportConnection.java:450)
at
org.apache.activemq.command.MessageDispatchNotification.visit(MessageDispatchNotification.java:77)
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:281)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:100)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:67)
at
org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:202)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
- After having killed the master, stopped the slave, copied the slave's data
into the master's data directory various error message came up (as described
in the Master/Slave recovery section),
e.g. (internal) ActiveMQ topics were not available, the admin webApp showed
exceptions and errors on the client.
The test I've created uses Spring 2.0.x and pumps 1000 MapMessages in a
queue through Spring's JmsTempate, each message is created within its own
transaction, using JmsTransactionManager and TransactionTemplate.
The created messages are consumed by an initially instantiated transactional
DefaultMessageListenerContainer. The AMQ JARs in the test's classpath are
activemq-core-5.0.0.jar, geronimo-jms_1.1_spec-1.0.jar,
geronimo-jta_1.0.1B_spec-1.0.jar as I've noticed a really bad performance
when only using the activemq-all-5.0.0.jar (maybe this is the problem?).
The test code work's without problems with OpenMQ, but I'd prefer using the
nice Pure Master/Active ActiveMQ if I can get it running in a *stable*
config ;)
I would highly appreciate any help or suggestions. Maybe my config is wrong
or I miss something essential. I've also tried a recent AMQ 5.1 SNAPSHOT
which wasn't better...
Thanks in advance,
Thomas
<!-- MASTER config -->
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.org/config/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.org/config/1.0
http://activemq.apache.org/schema/activemq-core.xsd
http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
<broker xmlns="http://activemq.org/config/1.0" brokerName="master"
dataDirectory="${activemq.base}/data">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic="FOO.>" producerFlowControl="false"
memoryLimit="1mb">
<dispatchPolicy>
<strictOrderDispatchPolicy/>
</dispatchPolicy>
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy/>
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://tbuckel-desktop:7778" />
</transportConnectors>
<networkConnectors/>
<managementContext>
<managementContext connectorPort="1100"
jmxDomainName="org.apache.activemq"/>
</managementContext>
</broker>
<commandAgent xmlns="http://activemq.org/config/1.0"/>
<jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
<connectors>
<nioConnector port="8161" />
</connectors>
<handlers>
<webAppContext contextPath="/admin"
resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true" />
</handlers>
</jetty>
</beans>
<!-- SLAVE config -->
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.org/config/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.org/config/1.0
http://activemq.apache.org/schema/activemq-core.xsd
http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
<broker xmlns="http://activemq.org/config/1.0" brokerName="slave"
dataDirectory="${activemq.base}/data-slave"
masterConnectorURI="tcp://tbuckel-desktop:7778">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic="FOO.>" producerFlowControl="false"
memoryLimit="1mb">
<dispatchPolicy>
<strictOrderDispatchPolicy/>
</dispatchPolicy>
<subscriptionRecoveryPolicy>
<lastImageSubscriptionRecoveryPolicy/>
</subscriptionRecoveryPolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://localhost:7779"/>
</transportConnectors>
<networkConnectors/>
<managementContext>
<managementContext connectorPort="1101"
jmxDomainName="org.apache.activemq"/>
</managementContext>
</broker>
<commandAgent xmlns="http://activemq.org/config/1.0"/>
<jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
<connectors>
<nioConnector port="8162" />
</connectors>
<handlers>
<webAppContext contextPath="/admin"
resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true" />
</handlers>
</jetty>
</beans>
--
View this message in context: http://www.nabble.com/Problems-with-Pure-Master-Slave-in-AMQ-5.0.0-tp15471491s2354p15471491.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Problems with Pure Master/Slave in AMQ 5.0.0
Posted by Ramit Arora <ra...@gmail.com>.
Hi,
I am facing the same issue with 5.0.0. Has something been done about it? The
Master/Slave topology seeems to work fine for a small number of clients
(around 20-30) but fails after that. This might turn out to be a showstopper
for us.
Thanks,
Ramit
TBuckel wrote:
>
> Raised a JIRE ticket. Attached my test program but haven't written a real
> unit test for it.
>
> https://issues.apache.org/activemq/browse/AMQ-1585
>
>
> rajdavies wrote:
>>
>> Can you raise a jira - with a junit test case - so we won't loose
>> track of this?
>> Also - can you verify that you get the same behaviour on the latest
>> 5.1-snapshot
>>
>> cheers,
>>
>> Rob
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Problems-with-Pure-Master-Slave-in-AMQ-5.0.0-tp15471491s2354p16255627.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Problems with Pure Master/Slave in AMQ 5.0.0
Posted by TBuckel <th...@jdv.com>.
Raised a JIRE ticket. Attached my test program but haven't written a real
unit test for it.
https://issues.apache.org/activemq/browse/AMQ-1585
rajdavies wrote:
>
> Can you raise a jira - with a junit test case - so we won't loose
> track of this?
> Also - can you verify that you get the same behaviour on the latest
> 5.1-snapshot
>
> cheers,
>
> Rob
>
>
--
View this message in context: http://www.nabble.com/Problems-with-Pure-Master-Slave-in-AMQ-5.0.0-tp15471491s2354p15536007.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Problems with Pure Master/Slave in AMQ 5.0.0
Posted by Rob Davies <ra...@gmail.com>.
On Feb 14, 2008, at 12:41 AM, TBuckel wrote:
>
> Hi all,
>
>
> I am having trouble setting up a *stable* ActiveMQ Pure Master/Slave
> topology.
>
>
> Initially I have tried v4.1.1 which failed with an exception. I
> found an AMQ
> JIRA ticket which said that Pure/Master slave didn't work in v4.1.1.
>
> Ok, so I switched to AMQ 5.0.0, created 2 configs (master/slave, see
> end of
> message) and ran two AMQ instances (on the same box) and most of the
> times
> my test (see below) worked, but more often I get various error
> messages
> like:
>
>
> - On the slave:
>
>
> ERROR Service - Async error occurred:
> javax.jms.JMSException: Slave broker out of sync with master:
> Dispatched
> message (ID:tbuckel-desktop-41814-1202886136210-0:0:565:1:1) was not
> in the
> pending list
> javax.jms.JMSException: Slave broker out of sync with master:
> Dispatched
> message (ID:tbuckel-desktop-41814-1202886136210-0:0:565:1:1) was not
> in the
> pending list
> at
> org
> .apache
> .activemq
> .broker
> .region
> .PrefetchSubscription
> .processMessageDispatchNotification(PrefetchSubscription.java:160)
> at
> org
> .apache
> .activemq
> .broker
> .region
> .AbstractRegion.processDispatchNotification(AbstractRegion.java:381)
> at
> org
> .apache
> .activemq
> .broker
> .region.RegionBroker.processDispatchNotification(RegionBroker.java:
> 550)
> at
> org
> .apache
> .activemq
> .broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:
> 201)
> at
> org
> .apache
> .activemq
> .broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:
> 201)
> at
> org
> .apache
> .activemq
> .broker.BrokerFilter.processDispatchNotification(BrokerFilter.java:
> 201)
> at
> org
> .apache
> .activemq
> .broker
> .MutableBrokerFilter
> .processDispatchNotification(MutableBrokerFilter.java:211)
> at
> org
> .apache
> .activemq
> .broker
> .TransportConnection
> .processMessageDispatchNotification(TransportConnection.java:450)
> at
> org
> .apache
> .activemq
> .command
> .MessageDispatchNotification.visit(MessageDispatchNotification.java:
> 77)
> at
> org
> .apache
> .activemq
> .broker.TransportConnection.service(TransportConnection.java:281)
> at
> org.apache.activemq.broker.TransportConnection
> $1.onCommand(TransportConnection.java:178)
> at
> org
> .apache
> .activemq
> .transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:100)
> at
> org
> .apache
> .activemq.transport.TransportFilter.onCommand(TransportFilter.java:67)
> at
> org
> .apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:
> 202)
> at
> org
> .apache
> .activemq
> .thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
> at
> org.apache.activemq.thread.DedicatedTaskRunner
> $1.run(DedicatedTaskRunner.java:36)
>
>
> - After having killed the master, stopped the slave, copied the
> slave's data
> into the master's data directory various error message came up (as
> described
> in the Master/Slave recovery section),
> e.g. (internal) ActiveMQ topics were not available, the admin
> webApp showed
> exceptions and errors on the client.
>
>
> The test I've created uses Spring 2.0.x and pumps 1000 MapMessages
> in a
> queue through Spring's JmsTempate, each message is created within
> its own
> transaction, using JmsTransactionManager and TransactionTemplate.
>
> The created messages are consumed by an initially instantiated
> transactional
> DefaultMessageListenerContainer. The AMQ JARs in the test's
> classpath are
> activemq-core-5.0.0.jar, geronimo-jms_1.1_spec-1.0.jar,
> geronimo-jta_1.0.1B_spec-1.0.jar as I've noticed a really bad
> performance
> when only using the activemq-all-5.0.0.jar (maybe this is the
> problem?).
>
> The test code work's without problems with OpenMQ, but I'd prefer
> using the
> nice Pure Master/Active ActiveMQ if I can get it running in a *stable*
> config ;)
>
>
> I would highly appreciate any help or suggestions. Maybe my config
> is wrong
> or I miss something essential. I've also tried a recent AMQ 5.1
> SNAPSHOT
> which wasn't better...
>
>
> Thanks in advance,
>
> Thomas
>
>
> <!-- MASTER config -->
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:amq="http://activemq.org/config/1.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> http://activemq.org/config/1.0
> http://activemq.apache.org/schema/activemq-core.xsd
> http://activemq.apache.org/camel/schema/spring
> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>
> <bean
> class
> =
> "org
> .springframework.beans.factory.config.PropertyPlaceholderConfigurer"/
> >
> <broker xmlns="http://activemq.org/config/1.0" brokerName="master"
> dataDirectory="${activemq.base}/data">
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic="FOO.>" producerFlowControl="false"
> memoryLimit="1mb">
> <dispatchPolicy>
> <strictOrderDispatchPolicy/>
> </dispatchPolicy>
> <subscriptionRecoveryPolicy>
> <lastImageSubscriptionRecoveryPolicy/>
> </subscriptionRecoveryPolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <transportConnectors>
> <transportConnector name="openwire"
> uri="tcp://tbuckel-desktop:7778" />
> </transportConnectors>
>
> <networkConnectors/>
>
> <managementContext>
> <managementContext connectorPort="1100"
> jmxDomainName="org.apache.activemq"/>
> </managementContext>
>
> </broker>
>
> <commandAgent xmlns="http://activemq.org/config/1.0"/>
>
> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
> <connectors>
> <nioConnector port="8161" />
> </connectors>
>
> <handlers>
> <webAppContext contextPath="/admin"
> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true" /
> >
> </handlers>
> </jetty>
> </beans>
>
>
> <!-- SLAVE config -->
> <beans
> xmlns="http://www.springframework.org/schema/beans"
> xmlns:amq="http://activemq.org/config/1.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
> http://activemq.org/config/1.0
> http://activemq.apache.org/schema/activemq-core.xsd
> http://activemq.apache.org/camel/schema/spring
> http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
>
> <bean
> class
> =
> "org
> .springframework.beans.factory.config.PropertyPlaceholderConfigurer"/
> >
>
> <broker xmlns="http://activemq.org/config/1.0" brokerName="slave"
> dataDirectory="${activemq.base}/data-slave"
> masterConnectorURI="tcp://tbuckel-desktop:7778">
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic="FOO.>" producerFlowControl="false"
> memoryLimit="1mb">
> <dispatchPolicy>
> <strictOrderDispatchPolicy/>
> </dispatchPolicy>
> <subscriptionRecoveryPolicy>
> <lastImageSubscriptionRecoveryPolicy/>
> </subscriptionRecoveryPolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <transportConnectors>
> <transportConnector name="openwire"
> uri="tcp://localhost:7779"/>
> </transportConnectors>
>
> <networkConnectors/>
>
> <managementContext>
> <managementContext connectorPort="1101"
> jmxDomainName="org.apache.activemq"/>
> </managementContext>
>
> </broker>
>
> <commandAgent xmlns="http://activemq.org/config/1.0"/>
>
> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
> <connectors>
> <nioConnector port="8162" />
> </connectors>
>
> <handlers>
> <webAppContext contextPath="/admin"
> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true" /
> >
> </handlers>
> </jetty>
>
> </beans>
>
> --
> View this message in context: http://www.nabble.com/Problems-with-Pure-Master-Slave-in-AMQ-5.0.0-tp15471491s2354p15471491.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Can you raise a jira - with a junit test case - so we won't loose
track of this?
Also - can you verify that you get the same behaviour on the latest
5.1-snapshot
cheers,
Rob