You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by brendanr <br...@gmail.com> on 2009/07/29 12:42:35 UTC
Master/Slave: Slave Failed, async error occurred
Hi,
I am having an issue with a simple master/slave setup. My master is running
on one machine (192.168.10.141), and my slave is running on a separate
machine (192.168.10.1). Attached are the config files for both master and
slave, and the log files for master and slave. I am running broker version
5.2.0 and using activemq CPP library 3.0.1. My producer and consumers are
using the supplied sample code.
On the master broker I get an exception saying the following:
ERROR MasterBroker - Slave Failed: ...
On the slave broker I get an exception daying teh following:
ERROR Service - Async error occurred: ...
Please see attached log files for complete exceptions.
I can easily reproduce the errors under the following conditions:
1. Start the master broker
2. Start a producer that connects to the master broker and sends 10 messages
to queue TEST.FOO. Producer disconnects.
3. Start the slave broker. It connects to teh master broker successfully.
3. Start a consumer that connects to the master broker. It will retrieve the
messages successfuly, but will cause the exceptions on both brokers.
It doesn't matter if the producer and consumer connect to the broker using
the failover mechanism (using 2 addresses) or not.
If I start both brokers first, before connecting any producers/consumers
then I do not get the exceptions, but in the application I would like to use
activemq in, it is not unreasonable that the master broker will have some
activity before the slave broker is ready to connect.
Should activemq work the way I would like it to work, and if so what am I
doing wrong? Any help would be much appreciated.
Thanks
Brendan
http://www.nabble.com/file/p24716703/activemq-master.log activemq-master.log
http://www.nabble.com/file/p24716703/activemq-master.xml activemq-master.xml
http://www.nabble.com/file/p24716703/activemq-slave.log activemq-slave.log
http://www.nabble.com/file/p24716703/activemq-slave.xml activemq-slave.xml
--
View this message in context: http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave: Slave Failed, async error occurred
Posted by Gary Tully <ga...@gmail.com>.
pure master slave is handy for data replication but is lock step rather than
catch up, is limited to a single pair and has no auto recovery strategy.
Master slave with a shared file system or shared database is what you want
for fault tolerance. The first to obtain a lock is a master and the next to
acquire the lock will assume mastership in the event of failure. This setup
along with the failover transport will give you a simple and reliable fault
tolerant solution.
2009/7/29 Norbert Pfistner <np...@picturesafe.de>
> We also plan to use master/slave.
>
> The 'waitForSlave' seems a bit strange for me, since our main reason for
> adding some slaves is to avoid a single point of failure (i.e. the master).
> But what if the slave fails? Does the required 'waitForSlave' mean that we
> can't run a master without his slaves? Doesn't this even increase the risk
> of failure?
>
> In an ideal world i would expect that using master slave would meen simply
> start a borker (first will be the master), add any slaves you want (which
> should be configured by the clients) any time later and beeing able to
> indipendly start/stop/crash one of them: The whole messaging system should
> remain stable as long at least one broker is up.
>
> Is this possible with activeMQ? Which version should we use (currently we
> use 5.2, not yet in Master/Slave, JDBC persistance).
>
> Are there any special settings we need to set to achieve the goal discribed
> above?
>
> Greetings,
> Norbert
>
> brendanr schrieb:
>
> Thank you for your reply.
>> I know about the 'waitForSlave' attribute, but it is not really what I
>> need.
>> I would like to be able to add a backup machine (and hence a slave broker)
>> at some later time, without having to restart the master broker and all
>> the
>> producer/consumer connections to it. I would have expected that when a
>> slave
>> connected that it could learn about all messages/producers/consumers
>> connected to the master, and continue operating correctly.
>>
>> But from what you are saying, this is not possible? Will it be available
>> when the Kaha DB feature is available?
>> Many thanks again.
>>
>> Gary Tully wrote:
>>
>>> the brokers need to be in sync from the start. there is a waitForSlave
>>> attribute that can force the master to stall till the slave connects.
>>> see:
>>> https://issues.apache.org/activemq/browse/AMQ-596
>>>
>>> 2009/7/29 brendanr <br...@gmail.com>
>>>
>>> Hi,
>>>>
>>>> I am having an issue with a simple master/slave setup. My master is
>>>> running
>>>> on one machine (192.168.10.141), and my slave is running on a separate
>>>> machine (192.168.10.1). Attached are the config files for both master
>>>> and
>>>> slave, and the log files for master and slave. I am running broker
>>>> version
>>>> 5.2.0 and using activemq CPP library 3.0.1. My producer and consumers
>>>> are
>>>> using the supplied sample code.
>>>>
>>>> On the master broker I get an exception saying the following:
>>>> ERROR MasterBroker - Slave Failed: ...
>>>> On the slave broker I get an exception daying teh following:
>>>> ERROR Service - Async error occurred: ...
>>>>
>>>> Please see attached log files for complete exceptions.
>>>>
>>>> I can easily reproduce the errors under the following conditions:
>>>> 1. Start the master broker
>>>> 2. Start a producer that connects to the master broker and sends 10
>>>> messages
>>>> to queue TEST.FOO. Producer disconnects.
>>>> 3. Start the slave broker. It connects to teh master broker
>>>> successfully.
>>>> 3. Start a consumer that connects to the master broker. It will retrieve
>>>> the
>>>> messages successfuly, but will cause the exceptions on both brokers.
>>>>
>>>> It doesn't matter if the producer and consumer connect to the broker
>>>> using
>>>> the failover mechanism (using 2 addresses) or not.
>>>> If I start both brokers first, before connecting any producers/consumers
>>>> then I do not get the exceptions, but in the application I would like to
>>>> use
>>>> activemq in, it is not unreasonable that the master broker will have
>>>> some
>>>> activity before the slave broker is ready to connect.
>>>>
>>>> Should activemq work the way I would like it to work, and if so what am
>>>> I
>>>> doing wrong? Any help would be much appreciated.
>>>>
>>>> Thanks
>>>> Brendan
>>>>
>>>>
>>>>
>>>> http://www.nabble.com/file/p24716703/activemq-master.logactivemq-master.log
>>>>
>>>> http://www.nabble.com/file/p24716703/activemq-master.xmlactivemq-master.xml
>>>> http://www.nabble.com/file/p24716703/activemq-slave.log
>>>> activemq-slave.log
>>>> http://www.nabble.com/file/p24716703/activemq-slave.xml
>>>> activemq-slave.xml
>>>> --
>>>> View this message in context:
>>>>
>>>> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>>
>>> --
>>> http://blog.garytully.com
>>>
>>> Open Source Integration
>>> http://fusesource.com
>>>
>>>
>>>
>>
> --
>
> Dipl.-Ing. Norbert Pfistner
> Softwareentwicklung
>
> picturesafe GmbH
> Simon-von-Utrecht-Straße 31-37
> D-20359 Hamburg
> http://www.picturesafe.de
>
> fon: +49 40 374127 901
> fax: +49 40 374127 999
> npfistner@picturesafe.de
>
> Sitz der Gesellschaft: Hannover
> Geschäftsführer: Herbert Wirth
> HR: Amtsgericht Hannover HR B 53 366
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Master/Slave: Slave Failed, async error occurred
Posted by Norbert Pfistner <np...@picturesafe.de>.
We also plan to use master/slave.
The 'waitForSlave' seems a bit strange for me, since our main reason for
adding some slaves is to avoid a single point of failure (i.e. the
master). But what if the slave fails? Does the required 'waitForSlave'
mean that we can't run a master without his slaves? Doesn't this even
increase the risk of failure?
In an ideal world i would expect that using master slave would meen
simply start a borker (first will be the master), add any slaves you
want (which should be configured by the clients) any time later and
beeing able to indipendly start/stop/crash one of them: The whole
messaging system should remain stable as long at least one broker is up.
Is this possible with activeMQ? Which version should we use (currently
we use 5.2, not yet in Master/Slave, JDBC persistance).
Are there any special settings we need to set to achieve the goal
discribed above?
Greetings,
Norbert
brendanr schrieb:
> Thank you for your reply.
> I know about the 'waitForSlave' attribute, but it is not really what I need.
> I would like to be able to add a backup machine (and hence a slave broker)
> at some later time, without having to restart the master broker and all the
> producer/consumer connections to it. I would have expected that when a slave
> connected that it could learn about all messages/producers/consumers
> connected to the master, and continue operating correctly.
>
> But from what you are saying, this is not possible? Will it be available
> when the Kaha DB feature is available?
> Many thanks again.
>
>
> Gary Tully wrote:
>> the brokers need to be in sync from the start. there is a waitForSlave
>> attribute that can force the master to stall till the slave connects. see:
>> https://issues.apache.org/activemq/browse/AMQ-596
>>
>> 2009/7/29 brendanr <br...@gmail.com>
>>
>>> Hi,
>>>
>>> I am having an issue with a simple master/slave setup. My master is
>>> running
>>> on one machine (192.168.10.141), and my slave is running on a separate
>>> machine (192.168.10.1). Attached are the config files for both master and
>>> slave, and the log files for master and slave. I am running broker
>>> version
>>> 5.2.0 and using activemq CPP library 3.0.1. My producer and consumers are
>>> using the supplied sample code.
>>>
>>> On the master broker I get an exception saying the following:
>>> ERROR MasterBroker - Slave Failed: ...
>>> On the slave broker I get an exception daying teh following:
>>> ERROR Service - Async error occurred: ...
>>>
>>> Please see attached log files for complete exceptions.
>>>
>>> I can easily reproduce the errors under the following conditions:
>>> 1. Start the master broker
>>> 2. Start a producer that connects to the master broker and sends 10
>>> messages
>>> to queue TEST.FOO. Producer disconnects.
>>> 3. Start the slave broker. It connects to teh master broker successfully.
>>> 3. Start a consumer that connects to the master broker. It will retrieve
>>> the
>>> messages successfuly, but will cause the exceptions on both brokers.
>>>
>>> It doesn't matter if the producer and consumer connect to the broker
>>> using
>>> the failover mechanism (using 2 addresses) or not.
>>> If I start both brokers first, before connecting any producers/consumers
>>> then I do not get the exceptions, but in the application I would like to
>>> use
>>> activemq in, it is not unreasonable that the master broker will have some
>>> activity before the slave broker is ready to connect.
>>>
>>> Should activemq work the way I would like it to work, and if so what am I
>>> doing wrong? Any help would be much appreciated.
>>>
>>> Thanks
>>> Brendan
>>>
>>>
>>> http://www.nabble.com/file/p24716703/activemq-master.logactivemq-master.log
>>> http://www.nabble.com/file/p24716703/activemq-master.xmlactivemq-master.xml
>>> http://www.nabble.com/file/p24716703/activemq-slave.log
>>> activemq-slave.log
>>> http://www.nabble.com/file/p24716703/activemq-slave.xml
>>> activemq-slave.xml
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>> --
>> http://blog.garytully.com
>>
>> Open Source Integration
>> http://fusesource.com
>>
>>
>
--
Dipl.-Ing. Norbert Pfistner
Softwareentwicklung
picturesafe GmbH
Simon-von-Utrecht-Straße 31-37
D-20359 Hamburg
http://www.picturesafe.de
fon: +49 40 374127 901
fax: +49 40 374127 999
npfistner@picturesafe.de
Sitz der Gesellschaft: Hannover
Geschäftsführer: Herbert Wirth
HR: Amtsgericht Hannover HR B 53 366
Re: Master/Slave: Slave Failed, async error occurred
Posted by Gary Tully <ga...@gmail.com>.
You may need to look at share file system or JDBC master slave
configuration.
The group clustering type replication with automatic group additions is
something that will be addressed in the 6.0 time frame. Some initial support
for replicating persistent messages is already part of KahaDB, it may be
work giving it a whirl using a current 5.3-SNAPSHOT or trunk build; see:
http://activemq.apache.org/kahadb-master-slave.html - the basics are
already there. Stuff like replicating non persistent messages is on the to
do list.
2009/7/29 brendanr <br...@gmail.com>
>
> Thank you for your reply.
> I know about the 'waitForSlave' attribute, but it is not really what I
> need.
> I would like to be able to add a backup machine (and hence a slave broker)
> at some later time, without having to restart the master broker and all the
> producer/consumer connections to it. I would have expected that when a
> slave
> connected that it could learn about all messages/producers/consumers
> connected to the master, and continue operating correctly.
>
> But from what you are saying, this is not possible? Will it be available
> when the Kaha DB feature is available?
> Many thanks again.
>
>
> Gary Tully wrote:
> >
> > the brokers need to be in sync from the start. there is a waitForSlave
> > attribute that can force the master to stall till the slave connects.
> see:
> > https://issues.apache.org/activemq/browse/AMQ-596
> >
> > 2009/7/29 brendanr <br...@gmail.com>
> >
> >>
> >> Hi,
> >>
> >> I am having an issue with a simple master/slave setup. My master is
> >> running
> >> on one machine (192.168.10.141), and my slave is running on a separate
> >> machine (192.168.10.1). Attached are the config files for both master
> and
> >> slave, and the log files for master and slave. I am running broker
> >> version
> >> 5.2.0 and using activemq CPP library 3.0.1. My producer and consumers
> are
> >> using the supplied sample code.
> >>
> >> On the master broker I get an exception saying the following:
> >> ERROR MasterBroker - Slave Failed: ...
> >> On the slave broker I get an exception daying teh following:
> >> ERROR Service - Async error occurred: ...
> >>
> >> Please see attached log files for complete exceptions.
> >>
> >> I can easily reproduce the errors under the following conditions:
> >> 1. Start the master broker
> >> 2. Start a producer that connects to the master broker and sends 10
> >> messages
> >> to queue TEST.FOO. Producer disconnects.
> >> 3. Start the slave broker. It connects to teh master broker
> successfully.
> >> 3. Start a consumer that connects to the master broker. It will retrieve
> >> the
> >> messages successfuly, but will cause the exceptions on both brokers.
> >>
> >> It doesn't matter if the producer and consumer connect to the broker
> >> using
> >> the failover mechanism (using 2 addresses) or not.
> >> If I start both brokers first, before connecting any producers/consumers
> >> then I do not get the exceptions, but in the application I would like to
> >> use
> >> activemq in, it is not unreasonable that the master broker will have
> some
> >> activity before the slave broker is ready to connect.
> >>
> >> Should activemq work the way I would like it to work, and if so what am
> I
> >> doing wrong? Any help would be much appreciated.
> >>
> >> Thanks
> >> Brendan
> >>
> >>
> >>
> http://www.nabble.com/file/p24716703/activemq-master.logactivemq-master.log
> >>
> http://www.nabble.com/file/p24716703/activemq-master.xmlactivemq-master.xml
> >> http://www.nabble.com/file/p24716703/activemq-slave.log
> >> activemq-slave.log
> >> http://www.nabble.com/file/p24716703/activemq-slave.xml
> >> activemq-slave.xml
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24718147.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Master/Slave: Slave Failed, async error occurred
Posted by brendanr <br...@gmail.com>.
Thank you for your reply.
I know about the 'waitForSlave' attribute, but it is not really what I need.
I would like to be able to add a backup machine (and hence a slave broker)
at some later time, without having to restart the master broker and all the
producer/consumer connections to it. I would have expected that when a slave
connected that it could learn about all messages/producers/consumers
connected to the master, and continue operating correctly.
But from what you are saying, this is not possible? Will it be available
when the Kaha DB feature is available?
Many thanks again.
Gary Tully wrote:
>
> the brokers need to be in sync from the start. there is a waitForSlave
> attribute that can force the master to stall till the slave connects. see:
> https://issues.apache.org/activemq/browse/AMQ-596
>
> 2009/7/29 brendanr <br...@gmail.com>
>
>>
>> Hi,
>>
>> I am having an issue with a simple master/slave setup. My master is
>> running
>> on one machine (192.168.10.141), and my slave is running on a separate
>> machine (192.168.10.1). Attached are the config files for both master and
>> slave, and the log files for master and slave. I am running broker
>> version
>> 5.2.0 and using activemq CPP library 3.0.1. My producer and consumers are
>> using the supplied sample code.
>>
>> On the master broker I get an exception saying the following:
>> ERROR MasterBroker - Slave Failed: ...
>> On the slave broker I get an exception daying teh following:
>> ERROR Service - Async error occurred: ...
>>
>> Please see attached log files for complete exceptions.
>>
>> I can easily reproduce the errors under the following conditions:
>> 1. Start the master broker
>> 2. Start a producer that connects to the master broker and sends 10
>> messages
>> to queue TEST.FOO. Producer disconnects.
>> 3. Start the slave broker. It connects to teh master broker successfully.
>> 3. Start a consumer that connects to the master broker. It will retrieve
>> the
>> messages successfuly, but will cause the exceptions on both brokers.
>>
>> It doesn't matter if the producer and consumer connect to the broker
>> using
>> the failover mechanism (using 2 addresses) or not.
>> If I start both brokers first, before connecting any producers/consumers
>> then I do not get the exceptions, but in the application I would like to
>> use
>> activemq in, it is not unreasonable that the master broker will have some
>> activity before the slave broker is ready to connect.
>>
>> Should activemq work the way I would like it to work, and if so what am I
>> doing wrong? Any help would be much appreciated.
>>
>> Thanks
>> Brendan
>>
>>
>> http://www.nabble.com/file/p24716703/activemq-master.logactivemq-master.log
>> http://www.nabble.com/file/p24716703/activemq-master.xmlactivemq-master.xml
>> http://www.nabble.com/file/p24716703/activemq-slave.log
>> activemq-slave.log
>> http://www.nabble.com/file/p24716703/activemq-slave.xml
>> activemq-slave.xml
>> --
>> View this message in context:
>> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24718147.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave: Slave Failed, async error occurred
Posted by Gary Tully <ga...@gmail.com>.
the brokers need to be in sync from the start. there is a waitForSlave
attribute that can force the master to stall till the slave connects. see:
https://issues.apache.org/activemq/browse/AMQ-596
2009/7/29 brendanr <br...@gmail.com>
>
> Hi,
>
> I am having an issue with a simple master/slave setup. My master is running
> on one machine (192.168.10.141), and my slave is running on a separate
> machine (192.168.10.1). Attached are the config files for both master and
> slave, and the log files for master and slave. I am running broker version
> 5.2.0 and using activemq CPP library 3.0.1. My producer and consumers are
> using the supplied sample code.
>
> On the master broker I get an exception saying the following:
> ERROR MasterBroker - Slave Failed: ...
> On the slave broker I get an exception daying teh following:
> ERROR Service - Async error occurred: ...
>
> Please see attached log files for complete exceptions.
>
> I can easily reproduce the errors under the following conditions:
> 1. Start the master broker
> 2. Start a producer that connects to the master broker and sends 10
> messages
> to queue TEST.FOO. Producer disconnects.
> 3. Start the slave broker. It connects to teh master broker successfully.
> 3. Start a consumer that connects to the master broker. It will retrieve
> the
> messages successfuly, but will cause the exceptions on both brokers.
>
> It doesn't matter if the producer and consumer connect to the broker using
> the failover mechanism (using 2 addresses) or not.
> If I start both brokers first, before connecting any producers/consumers
> then I do not get the exceptions, but in the application I would like to
> use
> activemq in, it is not unreasonable that the master broker will have some
> activity before the slave broker is ready to connect.
>
> Should activemq work the way I would like it to work, and if so what am I
> doing wrong? Any help would be much appreciated.
>
> Thanks
> Brendan
>
>
> http://www.nabble.com/file/p24716703/activemq-master.logactivemq-master.log
> http://www.nabble.com/file/p24716703/activemq-master.xmlactivemq-master.xml
> http://www.nabble.com/file/p24716703/activemq-slave.log activemq-slave.log
> http://www.nabble.com/file/p24716703/activemq-slave.xml activemq-slave.xml
> --
> View this message in context:
> http://www.nabble.com/Master-Slave%3A-Slave-Failed%2C-async-error-occurred-tp24716703p24716703.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com