[jira] Commented: (AMQ-2880) Exception thrown during commit leads to message loss

["Stan Lewis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/activemq/browse/AMQ-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61454#action_61454 ] 

Stan Lewis commented on AMQ-2880:
---------------------------------

Just a note to simulate the real lock timeout exception using MySQL you can just lock some rows in the table, for example:

start transaction;
select id from activemq_msgs limit 10 for update;

then try and consume some messages.  ActiveMQ will get block and then get a lock timeout when it hits these locked rows.  Then release the lock with:

commit;

after consuming all of the messages in the queue that you can you'll see those locked rows are still in the table and hadn't been consumed.

> Exception thrown during commit leads to message loss
> ----------------------------------------------------
>
>                 Key: AMQ-2880
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2880
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.4.0
>            Reporter: Stan Lewis
>         Attachments: test-case.txt
>
>
> In cases where JDBC persistence is used and the database server is under a fair bit of load it's sometimes possible for table/row locks to time out, which means you'll see exceptions such as:
> java.sql.BatchUpdateException: Lock wait timeout exceeded; try restarting transaction
> at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1693)
> at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1108)
> at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
> at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
> at org.apache.activemq.store.jdbc.TransactionContext.executeBatch(TransactionContext.java:98)
> at org.apache.activemq.store.jdbc.TransactionContext.executeBatch(TransactionContext.java:80)
> at org.apache.activemq.store.jdbc.TransactionContext.commit(TransactionContext.java:161)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.commitTransaction(JDBCPersistenceAdapter.java:504)
> at org.apache.activemq.store.memory.MemoryTransactionStore$Tx.commit(MemoryTransactionStore.java:109)
> at org.apache.activemq.store.memory.MemoryTransactionStore.commit(MemoryTransactionStore.java:220)
> at org.apache.activemq.transaction.XATransaction.commit(XATransaction.java:73)
> at org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:176)
> at org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:103)
> at org.apache.activemq.broker.TransportConnection.processCommitTransactionTwoPhase(TransportConnection.java:430)
> at org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:102)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
> at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
> at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201)
> at java.lang.Thread.run(Thread.java:636)
> In this case the connection to the database is fine and what we should do is retry the transaction as it's a temporary failure condition.  Instead what happens is the broker moves to the next message in the queue, leaving the current message in the database.  The message won't show up in the web console and cannot be consumed by any consumers until the broker is restarted.
> Attached is a test case that simulates the error condition in a controlled fashion by using a subclassed JDBCPersistenceAdapter that will throw an exception in commitTransaction as necessary.  So the producer sends 10 messages and then the consumer tries to consume them, during this time the persistence adapter will throw an exception during commitTransaction.  Then the condition is cleared and the consumer can consume all 10 messages, however the consumer only consumes 9 messages, the 1st message in the sequence is still in the database.  Hopefully the logging makes this clear.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.