[jira] [Created] (QPID-5642) Message sequence numbers (enabled with qpid.msg_sequence) should be persistent across broker restarts

["Pavel Moravec (JIRA)" <ji...@apache.org>]

Pavel Moravec created QPID-5642:
-----------------------------------

             Summary: Message sequence numbers (enabled with qpid.msg_sequence) should be persistent across broker restarts
                 Key: QPID-5642
                 URL: https://issues.apache.org/jira/browse/QPID-5642
             Project: Qpid
          Issue Type: Bug
          Components: C++ Broker
    Affects Versions: 0.26
            Reporter: Pavel Moravec
            Assignee: Pavel Moravec
            Priority: Minor


The C++ broker supports currently only at-least-once and at-most-once reliability. Therefore duplicate messages may occur, especially in situations involving failovers, reconnects etc. Currently, the broker doesn't offer any specific solution for duplicate detection. For many messaging scenarios, it is crucial to process every message only once to avoid errors. 

The broker supports message sequencing on exchange level. The message sequencing assigns a sequence number to every message which is routed via the exchange. In theory, this can be used to detect duplicates. Unfortunately, the sequence number isn't stored persistently and is restarted with every restart of the broker (HA cluster) and starts again from 1. That makes the use of the sequence number for duplicate detection quite complicated, especially since the restart of the broker is also the situation when the duplicates may occur.

Therefore, it is required to update message sequence numbers (enabled with qpid.msg_sequence) to store/journal after every (pre)route of a durable exchange, in order to persist the sequence numbers accross broker restart.
	



Reproducer:

rm -rf /var/lib/qpidd/* output.txt
service qpidd restart
qpid-config add exchange fanout myFanout --sequence --durable
qpid-receive -a myFanout --connection-options="{reconnect:true}" -f --print-content=no --print-header=yes -m 6 > output.txt 2>/dev/null &
sleep 1
qpid-send -a myFanout -m 3
service qpidd restart
sleep 1
qpid-send -a myFanout -m 3
cat output.txt


Current output:
Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:1, sn:1, ts:1394611004142278196, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:2, sn:2, ts:1394611004142340093, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:3, sn:3, ts:1394611004142354743, x-amqp-0-10.routing-key:}


Expected output:
Properties: {qpid.msg_sequence:1, sn:1, ts:1394611002529779545, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:2, sn:2, ts:1394611002529879589, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:3, sn:3, ts:1394611002529896423, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:4, sn:1, ts:1394611004142278196, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:5, sn:2, ts:1394611004142340093, x-amqp-0-10.routing-key:}

Properties: {qpid.msg_sequence:6, sn:3, ts:1394611004142354743, x-amqp-0-10.routing-key:}

(sn comes from qpid-send so it will be re-set to 1)




--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org