Active / Passive in WAN

[Sebastien Briquet <sb...@apache.org>]

Hi,

We have the need for message queuing where the message order is the most
important point in our use case. RocketMQ seems the best choices here and
we will give it a try in the coming weeks (at my daily work).

However, I need to be able to answer a question first: How does it work in
a active/passive multi-site environment?

For instance: I have a cluster located in a site A, which is active. The
site B is passive until a switch is decided. So between the time the site A
is still active (because of message already queued) and the site B becomes
active and start queuing new messages, how can I guarantee messages are
ordered between sites?

I cannot find any documentation about this specifically (or I didn't
understood the existing one :p)

Many thanks in advance!
Sebastien.

Re: Active / Passive in WAN

[yukon <yu...@apache.org>]

Hi,

Thanks for your interests in RocketMQ. Let's start with the ordered message
of RocketMQ.

Each topic has multiple message queues spread over multiple broker server
nodes, messages in the same queue can be consumed in order if you use
OrderConsumer. The concept message queue is similar to the known partition,
and it's a FIFO queue.

By default, a topic has 8 queues in each broker node, the producer client
is responsible for sending a kind of messages to the fixed partition with a
sharding key. For example, The chat messages of the same chat channel
should be sent to the same message queue in order to consume them orderly.

Obviously,  if we adjust the message queue number to 1 of a topic, all the
messages sent to the topic can be consumed in order.

We have an example shows how to use ordered message[1]. In the example, we
use `OrderId` as sharding key and modular hashing to select a message queue.

Now, back to your question, we assume that the both BrokerA in site A and
BrokerB in site B both have TopicX. When site A is active, all the messages
are sent to BrokerA and consumed from BrokerA, TopicX in passive BrokerB
has no read or write perm.

In switch stage:

1. Firstly, set TopicX perm in BrokerB to write-only, set TopicX perm in
BrokerA to read-only. Now, all the new messages are sent to the BrokerB but
can't be consumed yet.
2. Then, set TopicX perm in BrokerB to read-write after the messages
already queued in BrokerA consumed, then disable the perm in BrokerA. Now,
BrokerA is passive.

All the operations can be finished by the MQAdmin tool, please have a try
and let me know if you have any other question.

[1]. http://rocketmq.apache.org/docs/order-example/

Regards,
yukon

On Thu, Feb 1, 2018 at 5:56 PM, Sebastien Briquet <sb...@apache.org>
wrote:

> Hi,
>
> We have the need for message queuing where the message order is the most
> important point in our use case. RocketMQ seems the best choices here and
> we will give it a try in the coming weeks (at my daily work).
>
> However, I need to be able to answer a question first: How does it work in
> a active/passive multi-site environment?
>
> For instance: I have a cluster located in a site A, which is active. The
> site B is passive until a switch is decided. So between the time the site A
> is still active (because of message already queued) and the site B becomes
> active and start queuing new messages, how can I guarantee messages are
> ordered between sites?
>
> I cannot find any documentation about this specifically (or I didn't
> understood the existing one :p)
>
> Many thanks in advance!
> Sebastien.
>
>