[jira] Updated: (AMQ-2573) Failover with async rollback of message audit can lead to over eager duplicate suppression (auto acking) and apparent missing messages

["Gary Tully (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/activemq/browse/AMQ-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully updated AMQ-2573:
----------------------------

    Description: 
with multiple unordered message consumers on a connection and failover with default prefetch recovery processing can lead to incorrect duplicate detection.
Duplicates are expected as all unacked messages will be redelivered on recovery. An active consumer may have unconsumed messages which need to be rolled back in the message audit. currently this rollback is occurring at an indeterminate time w.r.t to future dispatch.
given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be rolled back prior to recovery of consumer 3 which may get any message in the range 0 - 20 delivered to it. Currently, rollbacks are done on the next delivery. 
All audit rollbacks need to be complete before any new delivery occurs on a recovered connection.

The result of over eager duplicate suppression is missing messages as the duplicates are auto acked.
Symptom: missing messages from a consumer perspective. With debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code} it is possible to catch the duplicate suppression. If your missing messages are in this set, you may have this problem.

workaround is to reduce the prefetch or even use a pull consumer, prefetch=0 such that unconsumed messages do not back up in the consumers dispatch queue pending receive() calls. With async listeners where the consumers can keep up with dispatch, this will not be a problem,

  was:
with multiple unordered message consumers on a connection and failover with default prefetch recovery processing can lead to incorrect duplicate detection.
Duplicates are expected as all unacked messages will be redelivered on recovery. An active consumer may have unconsumed messages which need to be rolled back in the message audit. currently this rollback is occurring at an indeterminate time w.r.t to future dispatch.
given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be rolled back prior to recovery of consumer 3 which may get an message in the range 0 - 20 delivered to it. Currently, rollbacks are done on the next delivery. 
All audit rollbacks need to be complete before any new delivery occurs on a recovered connection.

The result of over eager duplicate suppression is missing messages as the duplicates are auto acked.
Symptom: with debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code} it is possible to catch the duplicate suppression. If your missing messages are in this set, you may have this problem.


> Failover with async rollback of message audit can lead to over eager duplicate suppression (auto acking) and apparent missing messages
> --------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2573
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2573
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Gary Tully
>            Assignee: Gary Tully
>             Fix For: 5.3.1, 5.4.0
>
>
> with multiple unordered message consumers on a connection and failover with default prefetch recovery processing can lead to incorrect duplicate detection.
> Duplicates are expected as all unacked messages will be redelivered on recovery. An active consumer may have unconsumed messages which need to be rolled back in the message audit. currently this rollback is occurring at an indeterminate time w.r.t to future dispatch.
> given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be rolled back prior to recovery of consumer 3 which may get any message in the range 0 - 20 delivered to it. Currently, rollbacks are done on the next delivery. 
> All audit rollbacks need to be complete before any new delivery occurs on a recovered connection.
> The result of over eager duplicate suppression is missing messages as the duplicates are auto acked.
> Symptom: missing messages from a consumer perspective. With debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code} it is possible to catch the duplicate suppression. If your missing messages are in this set, you may have this problem.
> workaround is to reduce the prefetch or even use a pull consumer, prefetch=0 such that unconsumed messages do not back up in the consumers dispatch queue pending receive() calls. With async listeners where the consumers can keep up with dispatch, this will not be a problem,

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.