You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Jonathan Halliday (JIRA)" <ji...@apache.org> on 2018/10/23 12:33:00 UTC

[jira] [Updated] (ARTEMIS-2144) tx begin failure in ra doesn't get cleaned up

     [ https://issues.apache.org/jira/browse/ARTEMIS-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Halliday updated ARTEMIS-2144:
---------------------------------------
    Description: 
org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage calls to endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE) (i.e. Spring framework) to begin a Transaction (associating it to the Thread) and enlist the XAResource to that Transaction. These operations may fail independently:
{code:java}
// AbstractMessageEndpointFactory.java
public void beginTransaction() throws Exception {
  if (transactionFactory != null && this.xaResource != null) {
    // This line works, associating the tx to the Thread, because that's a local operation.
    this.transaction = transactionFactory.createTransaction(transactionName, transactionTimeout);
    // This line fails, because it's a remote call to a dead MQ server
    this.transaction.enlistResource(this.xaResource);
  }
}
{code}
{code:java}
# example trace (wildfly application server, artemis 1.5.5)
ERROR [org.apache.activemq.artemis.ra] (Thread-40 (ActiveMQ-client-global-threads)) AMQ154004: Failed to deliver message: javax.resource.spi.ApplicationServerInternalException: Failed to begin transaction
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:221) at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage(ActiveMQMessageHandler.java:293)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124)
 at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
Caused by: javax.transaction.SystemException: WFTXN0089: Failed to configure transaction timeout of 299
 at org.wildfly.transaction.client.LocalTransaction.enlistResource(LocalTransaction.java:155)
 at org.springframework.transaction.jta.ManagedTransactionAdapter.enlistResource(ManagedTransactionAdapter.java:83)
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$TransactionDelegate.beginTransaction(AbstractMessageEndpointFactory.java:313)
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:218)
 ... 8 more

{code}
In such case, it is necessary that ActiveMQMessageHandler ensure termination of the transaction before returning control. This can be accomplished by e.g.
|{code:java}
try {
   endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE);
} catch(ApplicationServerInternalException asie) {
   Throwable cause = asie.getCause();
   if(cause instanceof SystemException) {
      if(tm.getStatus() != Status.STATUS_NO_TRANSACTION) {
         ActiveMQRALogger.LOGGER.trace("HornetQMessageHandler::tx begin failed uncleanly with "+cause.getMessage()+". Trying to clean up.");
         tm.setRollbackOnly();
         endpoint.afterDelivery();
      }
   }
   throw asie;
}
{code}|

Failure to include this cleanup step results in the broken Transaction remaining associated to the Thread, which thereafter cannot be reused in a pooled environment.

  was:
org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage calls to endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE) (i.e. Spring framework) to begin a Transaction (associating it to the Thread) and enlist the XAResource to that Transaction. These operations may fail independently:
{code:java}
// AbstractMessageEndpointFactory.java
public void beginTransaction() throws Exception {
  if (transactionFactory != null && this.xaResource != null) {
    // This line works, associating the tx to the Thread, because that's a local operation.
    this.transaction = transactionFactory.createTransaction(transactionName, transactionTimeout);
    // This line fails, because it's a remote call to a dead MQ server
    this.transaction.enlistResource(this.xaResource);
  }
}
{code}
{code:java}
# example trace (wildfly application server, artemis 1.5.5)
ERROR [org.apache.activemq.artemis.ra] (Thread-40 (ActiveMQ-client-global-threads)) AMQ154004: Failed to deliver message: javax.resource.spi.ApplicationServerInternalException: Failed to begin transaction
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:221) at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage(ActiveMQMessageHandler.java:293)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49)
 at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124)
 at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
Caused by: javax.transaction.SystemException: WFTXN0089: Failed to configure transaction timeout of 299
 at org.wildfly.transaction.client.LocalTransaction.enlistResource(LocalTransaction.java:155)
 at org.springframework.transaction.jta.ManagedTransactionAdapter.enlistResource(ManagedTransactionAdapter.java:83)
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$TransactionDelegate.beginTransaction(AbstractMessageEndpointFactory.java:313)
 at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:218)
 ... 8 more

{code}
In such case, it is necessary that ActiveMQMessageHandler ensure termination of the transaction before returning control. This can be accomplished by e.g.
|{code:java}
try {
   endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE);
} catch(ApplicationServerInternalException asie) {
   Throwable cause = asie.getCause();
   if(cause instanceof SystemException) {
      ActiveMQRALogger.LOGGER.trace("HornetQMessageHandler::tx begin failed with "+cause.getMessage()+". Calling release (rollback) to ensure clean up.");
      endpoint.release();
   }
   throw asie;
}
{code}|

[https://gist.github.com/jhalliday/733b0b034509f3a1c92f192fae6a8bf9]  (patch against 1.5.5)

Failure to include this cleanup step results in the broken Transaction remaining associated to the Thread, which thereafter cannot be reused in a pooled environment.


> tx begin failure in ra doesn't get cleaned up
> ---------------------------------------------
>
>                 Key: ARTEMIS-2144
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-2144
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 1.5.5, 2.6.3
>            Reporter: Jonathan Halliday
>            Priority: Major
>
> org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage calls to endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE) (i.e. Spring framework) to begin a Transaction (associating it to the Thread) and enlist the XAResource to that Transaction. These operations may fail independently:
> {code:java}
> // AbstractMessageEndpointFactory.java
> public void beginTransaction() throws Exception {
>   if (transactionFactory != null && this.xaResource != null) {
>     // This line works, associating the tx to the Thread, because that's a local operation.
>     this.transaction = transactionFactory.createTransaction(transactionName, transactionTimeout);
>     // This line fails, because it's a remote call to a dead MQ server
>     this.transaction.enlistResource(this.xaResource);
>   }
> }
> {code}
> {code:java}
> # example trace (wildfly application server, artemis 1.5.5)
> ERROR [org.apache.activemq.artemis.ra] (Thread-40 (ActiveMQ-client-global-threads)) AMQ154004: Failed to deliver message: javax.resource.spi.ApplicationServerInternalException: Failed to begin transaction
>  at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:221) at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.onMessage(ActiveMQMessageHandler.java:293)
>  at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1001)
>  at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:49)
>  at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1124)
>  at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.transaction.SystemException: WFTXN0089: Failed to configure transaction timeout of 299
>  at org.wildfly.transaction.client.LocalTransaction.enlistResource(LocalTransaction.java:155)
>  at org.springframework.transaction.jta.ManagedTransactionAdapter.enlistResource(ManagedTransactionAdapter.java:83)
>  at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$TransactionDelegate.beginTransaction(AbstractMessageEndpointFactory.java:313)
>  at org.springframework.jca.endpoint.AbstractMessageEndpointFactory$AbstractMessageEndpoint.beforeDelivery(AbstractMessageEndpointFactory.java:218)
>  ... 8 more
> {code}
> In such case, it is necessary that ActiveMQMessageHandler ensure termination of the transaction before returning control. This can be accomplished by e.g.
> |{code:java}
> try {
>    endpoint.beforeDelivery(ActiveMQActivation.ONMESSAGE);
> } catch(ApplicationServerInternalException asie) {
>    Throwable cause = asie.getCause();
>    if(cause instanceof SystemException) {
>       if(tm.getStatus() != Status.STATUS_NO_TRANSACTION) {
>          ActiveMQRALogger.LOGGER.trace("HornetQMessageHandler::tx begin failed uncleanly with "+cause.getMessage()+". Trying to clean up.");
>          tm.setRollbackOnly();
>          endpoint.afterDelivery();
>       }
>    }
>    throw asie;
> }
> {code}|
> Failure to include this cleanup step results in the broken Transaction remaining associated to the Thread, which thereafter cannot be reused in a pooled environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)