You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by icfantv <ad...@readytalk.com> on 2010/04/13 19:33:59 UTC
Running ActiveMQ and having a database failover
We learned the hard way that using ActiveMQ with a database store doesn't
play nicely with vacuums in that ActiveMQ uses open transactions to "lock"
the relevant tables so secondary ActiveMQ servers wait to take over should
the primary ActiveMQ server fail. The open transaction causes vacuums to
never finish - very bad on production databases.
Learning from our mistakes, we set up a separate database server for use
solely by ActiveMQ. Now ActiveMQ can have open transactions to its heart's
content and we don't care because we don't need to vacuum these
tables...much, if ever.
The problem we are facing now is that we run the ActiveMQ database server in
a primary/secondary configuration such that if the primary fails, the
secondary server gets the IP address of the primary and in theory, the
service corrects itself. However, ActiveMQ also doesn't appear to like
having it's database connection ripped out from underneath it and instead of
failing gracefully, it pukes and while the process still runs, it no longer
accepts connections from ActiveMQ clients. We have to physically kill the
ActiveMQ process and restart it.
Does anyone have a solution whereby we can use primary/failover database
servers in conjunction with ActiveMQ such that ActiveMQ can recover without
user intervention? We are currently using ActiveMQ 5.1.
Thanks,
--adam
--
View this message in context: http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28233397.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Running ActiveMQ and having a database failover
Posted by icfantv <ad...@readytalk.com>.
Sorry, I never saw your reply. Can we still use this attribute if we run
ActiveMQ in failover mode? If so, how does the backup broker know not to
process messages or do both brokers actually process messages and it "just
works?" Or, can we not run in failover mode?
Thanks,
--adam
Dejan Bosanac wrote:
>
> Hi,
>
> if you're sure that only one broker will use database, you can try
> setting useDatabaseLock to false
>
> Cheers
> --
> Dejan Bosanac - http://twitter.com/dejanb
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
>
> On Wed, Apr 14, 2010 at 5:31 PM, icfantv <ad...@readytalk.com>
> wrote:
>
>>
>> That's a neat idea, but unfortunately for us it does not appear that
>> Postgresql supports a failover parameter in the connection URL.
>> Additionally, even if it did, it would depend on how it handled the
>> failover
>> so as to not cause ActiveMQ to puke.
>>
>> We use a floating IP address to point to the "primary" database server
>> and
>> use HA to failover to the "secondary" (by simply changing to where the
>> floating IP address points) if the primary becomes unresponsive.
>>
>> I'm willing to bet that the problem here is that ActiveMQ uses an open
>> transaction to "lock" the table or tables necessary to prevent a failover
>> ActiveMQ server from taking over.
>>
>> If anyone else as any ideas/thoughts/solutions I would appreciate it.
>> Thanks.
>>
>> --adam
>> --
>> View this message in context:
>> http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28244251.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> -----
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
--
View this message in context: http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28506984.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Running ActiveMQ and having a database failover
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi,
if you're sure that only one broker will use database, you can try
setting useDatabaseLock to false
Cheers
--
Dejan Bosanac - http://twitter.com/dejanb
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Wed, Apr 14, 2010 at 5:31 PM, icfantv <ad...@readytalk.com> wrote:
>
> That's a neat idea, but unfortunately for us it does not appear that
> Postgresql supports a failover parameter in the connection URL.
> Additionally, even if it did, it would depend on how it handled the
> failover
> so as to not cause ActiveMQ to puke.
>
> We use a floating IP address to point to the "primary" database server and
> use HA to failover to the "secondary" (by simply changing to where the
> floating IP address points) if the primary becomes unresponsive.
>
> I'm willing to bet that the problem here is that ActiveMQ uses an open
> transaction to "lock" the table or tables necessary to prevent a failover
> ActiveMQ server from taking over.
>
> If anyone else as any ideas/thoughts/solutions I would appreciate it.
> Thanks.
>
> --adam
> --
> View this message in context:
> http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28244251.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Running ActiveMQ and having a database failover
Posted by icfantv <ad...@readytalk.com>.
That's a neat idea, but unfortunately for us it does not appear that
Postgresql supports a failover parameter in the connection URL.
Additionally, even if it did, it would depend on how it handled the failover
so as to not cause ActiveMQ to puke.
We use a floating IP address to point to the "primary" database server and
use HA to failover to the "secondary" (by simply changing to where the
floating IP address points) if the primary becomes unresponsive.
I'm willing to bet that the problem here is that ActiveMQ uses an open
transaction to "lock" the table or tables necessary to prevent a failover
ActiveMQ server from taking over.
If anyone else as any ideas/thoughts/solutions I would appreciate it.
Thanks.
--adam
--
View this message in context: http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28244251.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Running ActiveMQ and having a database failover
Posted by Sebastian Rodriguez <sr...@gmail.com>.
I'm not sure rather this solution can be used in your case. If you want to
use a primary / secondary db, you can normally specify them in the driver
configuration:
<bean id="mssql-ds" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName"
value="com.microsoft.sqlserver.jdbc.SQLServerDriver"/>
<property name="url"
value="jdbc:sqlserver://PRINCIPALDBSERVER;databaseName=activemq;instanceName=PRINCIPAL;user=activemq;password=activemq;failoverPartner=MIRRORDBSERVER\MIRROR"/>
</bean>
This is the configuration for mysql for eg.
In your case, your primary and secondary db have the same IP. Maybe you can
consider this solution? :)
On 14 April 2010 01:33, icfantv <ad...@readytalk.com> wrote:
>
> We learned the hard way that using ActiveMQ with a database store doesn't
> play nicely with vacuums in that ActiveMQ uses open transactions to "lock"
> the relevant tables so secondary ActiveMQ servers wait to take over should
> the primary ActiveMQ server fail. The open transaction causes vacuums to
> never finish - very bad on production databases.
>
> Learning from our mistakes, we set up a separate database server for use
> solely by ActiveMQ. Now ActiveMQ can have open transactions to its heart's
> content and we don't care because we don't need to vacuum these
> tables...much, if ever.
>
> The problem we are facing now is that we run the ActiveMQ database server
> in
> a primary/secondary configuration such that if the primary fails, the
> secondary server gets the IP address of the primary and in theory, the
> service corrects itself. However, ActiveMQ also doesn't appear to like
> having it's database connection ripped out from underneath it and instead
> of
> failing gracefully, it pukes and while the process still runs, it no longer
> accepts connections from ActiveMQ clients. We have to physically kill the
> ActiveMQ process and restart it.
>
> Does anyone have a solution whereby we can use primary/failover database
> servers in conjunction with ActiveMQ such that ActiveMQ can recover without
> user intervention? We are currently using ActiveMQ 5.1.
>
> Thanks,
>
> --adam
> --
> View this message in context:
> http://old.nabble.com/Running-ActiveMQ-and-having-a-database-failover-tp28233397p28233397.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
Sebastien Rodriguez