You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by mhempleman <ma...@alstom.com> on 2015/09/27 00:59:14 UTC
Network of Brokers: XAException on Failover
Jboss 6.1 EAP
ActiveMQ 5.1.2
I'm trying to connect Jboss to an ActiveMQ high availability cluster. When
I kill one ActiveMQ server and Jboss fails over to the backup server
everything works as expected (messages are still passed), but I see the
following error message every two minutes:
[com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016027: Local
XARecoveryModule.xaRecovery got XA exception XAException.XAER_RMFAIL:
javax.transaction.xa.XAException: The JMS connection has failed:
java.io.EOFException
at
org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:793)
at
org.apache.activemq.TransactionContext.recover(TransactionContext.java:662)
at
org.apache.activemq.ra.LocalAndXATransaction.recover(LocalAndXATransaction.java:131)
at
org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.recover(XAResourceWrapperImpl.java:165)
at
com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoveryFirstPass(XARecoveryModule.java:534)
[jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
at
com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkFirstPass(XARecoveryModule.java:168)
[jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
at
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:743)
[jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
at
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371)
[jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
Caused by: org.apache.activemq.ConnectionFailedException: The JMS connection
has failed: java.io.EOFException
at
org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1483)
at
org.apache.activemq.TransactionContext.recover(TransactionContext.java:648)
... 6 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
[rt.jar:1.7.0_79]
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
This is the standalone.xml configuration:
<http://activemq.2283324.n4.nabble.com/file/n4702355/config.png>
Any idea why I'm seeing this message and how I can fix the issue?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Network of Brokers: XAException on Failover
Posted by mhempleman <ma...@alstom.com>.
I did mean 5.12.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355p4702359.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Network of Brokers: XAException on Failover
Posted by Tim Bain <tb...@alumni.duke.edu>.
Please tell me you meant 5.12, not 5.1.2.
On Sep 26, 2015 5:08 PM, "mhempleman" <ma...@alstom.com> wrote:
> Jboss 6.1 EAP
> ActiveMQ 5.1.2
>
> I'm trying to connect Jboss to an ActiveMQ high availability cluster. When
> I kill one ActiveMQ server and Jboss fails over to the backup server
> everything works as expected (messages are still passed), but I see the
> following error message every two minutes:
>
> [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016027: Local
> XARecoveryModule.xaRecovery got XA exception XAException.XAER_RMFAIL:
> javax.transaction.xa.XAException: The JMS connection has failed:
> java.io.EOFException
> at
>
> org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:793)
> at
> org.apache.activemq.TransactionContext.recover(TransactionContext.java:662)
> at
>
> org.apache.activemq.ra.LocalAndXATransaction.recover(LocalAndXATransaction.java:131)
> at
>
> org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.recover(XAResourceWrapperImpl.java:165)
> at
>
> com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoveryFirstPass(XARecoveryModule.java:534)
> [jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
> at
>
> com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkFirstPass(XARecoveryModule.java:168)
> [jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
> at
>
> com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:743)
> [jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
> at
>
> com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371)
> [jbossjts-jacorb-4.17.4.Final-redhat-2.jar:4.17.4.Final-redhat-2]
> Caused by: org.apache.activemq.ConnectionFailedException: The JMS
> connection
> has failed: java.io.EOFException
> at
>
> org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1483)
> at
> org.apache.activemq.TransactionContext.recover(TransactionContext.java:648)
> ... 6 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> [rt.jar:1.7.0_79]
> at
>
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
> at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
>
>
>
>
> This is the standalone.xml configuration:
>
> <http://activemq.2283324.n4.nabble.com/file/n4702355/config.png>
>
>
> Any idea why I'm seeing this message and how I can fix the issue?
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Network of Brokers: XAException on Failover
Posted by Tim Bain <tb...@alumni.duke.edu>.
BTW, I'm not convinced that a different network topology will avoid the
exception you first asked about. Someone who knows XA transactions (which
is not me, sorry) needs to look at that.
On Oct 1, 2015 7:06 AM, "Tim Bain" <tb...@alumni.duke.edu> wrote:
> A network of brokers can be used for HA as long as the network topology
> will remain fully connected in the face of N failures (for whatever value
> of N you choose to support, probably 1) and the clients' failover URIs will
> always contain a live broker in the face of the same failure(s).
>
> But you could also consider master/slave pairs using LevelDB as the
> storage technology, which doesn't have a single point of failure.
>
> Tim
> On Sep 29, 2015 3:45 PM, "mhempleman" <ma...@alstom.com>
> wrote:
>
>> Maybe I'm not using the network of brokers in the correct manner. Should
>> a
>> network of brokers be used for HA, or just scalability and load balancing?
>>
>> We are not concerned with the loss of a few messages if one broker fails;
>> however, we want to make sure there is not a single point of failure in
>> the
>> system (one broker node), and we want the client to failover seamlessly in
>> the event of a failure (which is not currently happening... see previous
>> message). I looked into master/slave setups, but each seems to have a
>> single point of failure. The failure points in the shared filesystem and
>> shared db configurations are obvious. Any advice would be greatly
>> appreciated. Thanks!
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355p4702445.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
Re: Network of Brokers: XAException on Failover
Posted by Tim Bain <tb...@alumni.duke.edu>.
A network of brokers can be used for HA as long as the network topology
will remain fully connected in the face of N failures (for whatever value
of N you choose to support, probably 1) and the clients' failover URIs will
always contain a live broker in the face of the same failure(s).
But you could also consider master/slave pairs using LevelDB as the storage
technology, which doesn't have a single point of failure.
Tim
On Sep 29, 2015 3:45 PM, "mhempleman" <ma...@alstom.com> wrote:
> Maybe I'm not using the network of brokers in the correct manner. Should a
> network of brokers be used for HA, or just scalability and load balancing?
>
> We are not concerned with the loss of a few messages if one broker fails;
> however, we want to make sure there is not a single point of failure in the
> system (one broker node), and we want the client to failover seamlessly in
> the event of a failure (which is not currently happening... see previous
> message). I looked into master/slave setups, but each seems to have a
> single point of failure. The failure points in the shared filesystem and
> shared db configurations are obvious. Any advice would be greatly
> appreciated. Thanks!
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355p4702445.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Network of Brokers: XAException on Failover
Posted by mhempleman <ma...@alstom.com>.
Maybe I'm not using the network of brokers in the correct manner. Should a
network of brokers be used for HA, or just scalability and load balancing?
We are not concerned with the loss of a few messages if one broker fails;
however, we want to make sure there is not a single point of failure in the
system (one broker node), and we want the client to failover seamlessly in
the event of a failure (which is not currently happening... see previous
message). I looked into master/slave setups, but each seems to have a
single point of failure. The failure points in the shared filesystem and
shared db configurations are obvious. Any advice would be greatly
appreciated. Thanks!
--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-Brokers-XAException-on-Failover-tp4702355p4702445.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.