You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Aaron Mulder <am...@alumni.princeton.edu> on 2008/05/17 03:05:49 UTC
Master/Slave Problem with v5.1.0
I've set up a master/slave cluster, and I'm running a test that
launches threads for 20 producers and 20 consumers and they read/write
persistent messages of 10kB to a queue. The consumers are using sync
receive() calls at the moment. When I start the test, I immediately
get lots of exceptions from both brokers, like below.
Any suggestions? If I do 1 producer, 1 consumer, and 1 message, no
problems. But even bumping that to 10 messages results in a couple
instance of the error. If I put in a delay of 100ms after every send
with 1 producer and consumer, no problem, but then if I go up to 10
producers and 10 consumers I get the problem again.
Thanks,
Aaron
ERROR MasterBroker - Slave Failed
javax.jms.JMSException: Slave broker out of sync with master:
Acknowledgment (MessageAck {commandId = 190, responseRequired = true,
ackType = 2, consumerId = ID:server-50670-1210985573156-0:14:1:1,
firstMessageId = null, lastMessageId =
ID:server-50670-1210985573156-0:8:1:1:1, destination =
queue://PerfTest, transactionId = null, messageCount = 1}) was not in
the dispatch list: []
at org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:365)
at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:364)
at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:470)
at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:84)
at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:443)
at org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:180)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:205)
at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
Re: Master/Slave Problem with v5.1.0
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
On Fri, May 16, 2008 at 9:29 PM, Bruce Snyder <br...@gmail.com> wrote:
> What type of master/slave are you using? Also, have you tried clearing
> the data directory from each broker and starting fresh to see if the
> problem manifests itself again?
"Pure" master/slave, and yes I whacked the data directories before
every run, so the problems are when starting from empty data dirs.
It's a pretty vanilla config based on the default activemq.xml -- I
just disabled some stuff (jetty/camel/etc.), changed ports/data
dir/broker name, bumped memory to 50MB/queue and 200MB overall, and
added the masterConnector clause to the slave config file.
I notice the error said something about async, and I wonder if I
can/should set the master/slave connection to be sync somehow? But I
didn't really see much by way of configuration options for the
master/slave connection.
Thanks,
Aaron
> On Fri, May 16, 2008 at 7:05 PM, Aaron Mulder
> <am...@alumni.princeton.edu> wrote:
>> I've set up a master/slave cluster, and I'm running a test that
>> launches threads for 20 producers and 20 consumers and they read/write
>> persistent messages of 10kB to a queue. The consumers are using sync
>> receive() calls at the moment. When I start the test, I immediately
>> get lots of exceptions from both brokers, like below.
>>
>> Any suggestions? If I do 1 producer, 1 consumer, and 1 message, no
>> problems. But even bumping that to 10 messages results in a couple
>> instance of the error. If I put in a delay of 100ms after every send
>> with 1 producer and consumer, no problem, but then if I go up to 10
>> producers and 10 consumers I get the problem again.
>>
>> Thanks,
>> Aaron
>>
>> ERROR MasterBroker - Slave Failed
>> javax.jms.JMSException: Slave broker out of sync with master:
>> Acknowledgment (MessageAck {commandId = 190, responseRequired = true,
>> ackType = 2, consumerId = ID:server-50670-1210985573156-0:14:1:1,
>> firstMessageId = null, lastMessageId =
>> ID:server-50670-1210985573156-0:8:1:1:1, destination =
>> queue://PerfTest, transactionId = null, messageCount = 1}) was not in
>> the dispatch list: []
>> at org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:365)
>> at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:364)
>> at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:470)
>> at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
>> at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
>> at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
>> at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:84)
>> at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:443)
>> at org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
>> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
>> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:180)
>> at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
>> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
>> at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:205)
>> at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
>> at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
>
>
> What type of master/slave are you using? Also, have you tried clearing
> the data directory from each broker and starting fresh to see if the
> problem manifests itself again?
>
> Bruce
> --
> perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> Apache ActiveMQ - http://activemq.org/
> Apache Camel - http://activemq.org/camel/
> Apache ServiceMix - http://servicemix.org/
> Apache Geronimo - http://geronimo.apache.org/
>
> Blog: http://bruceblog.org/
>
>
Re: Master/Slave Problem with v5.1.0
Posted by Bruce Snyder <br...@gmail.com>.
On Fri, May 16, 2008 at 7:05 PM, Aaron Mulder
<am...@alumni.princeton.edu> wrote:
> I've set up a master/slave cluster, and I'm running a test that
> launches threads for 20 producers and 20 consumers and they read/write
> persistent messages of 10kB to a queue. The consumers are using sync
> receive() calls at the moment. When I start the test, I immediately
> get lots of exceptions from both brokers, like below.
>
> Any suggestions? If I do 1 producer, 1 consumer, and 1 message, no
> problems. But even bumping that to 10 messages results in a couple
> instance of the error. If I put in a delay of 100ms after every send
> with 1 producer and consumer, no problem, but then if I go up to 10
> producers and 10 consumers I get the problem again.
>
> Thanks,
> Aaron
>
> ERROR MasterBroker - Slave Failed
> javax.jms.JMSException: Slave broker out of sync with master:
> Acknowledgment (MessageAck {commandId = 190, responseRequired = true,
> ackType = 2, consumerId = ID:server-50670-1210985573156-0:14:1:1,
> firstMessageId = null, lastMessageId =
> ID:server-50670-1210985573156-0:8:1:1:1, destination =
> queue://PerfTest, transactionId = null, messageCount = 1}) was not in
> the dispatch list: []
> at org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:365)
> at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:364)
> at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:470)
> at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
> at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:84)
> at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:443)
> at org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:180)
> at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:205)
> at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
> at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
What type of master/slave are you using? Also, have you tried clearing
the data directory from each broker and starting fresh to see if the
problem manifests itself again?
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache Camel - http://activemq.org/camel/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Blog: http://bruceblog.org/