[DISCUSS] How about the scheme of TubeMQ replica

[Goson zhang <go...@apache.org>]

I want to discuss the feature of TubeMQ replica

In our environment, there is a situation where the number of concurrent
filter consumption of Topic in some clusters has reached 100, and it is
likely to exceed 100 to 200, or even 300, but the number of concurrent
consumption of our single node is limited. So, we want to add a replica
solution to TubeMQ to support over 100 concurrent filtered consumption.

Considering that the current TubeMQ mainly supports high-performance
scenarios, we consider the asynchronous replication copy scheme, which is
similar to Kafka's replica scheme, but is slightly different:
[image: image.png]
1. Cluster's Broker nodes that support the replication scheme are organized
according to Group, and the nodes in the Group are assigned as one Leader
and more Follower nodes; Follower nodes pulls data asynchronously from
Leader node and saves it locally;
2. The Leader node in the Group undertakes production and a small amount of
consumption; the Follower nodes only do pure consumption processing. When
the Leader node fails, the Follower nodes can be switched to the Leader
node and perform related change processing;

To see if you have any good ideas and ideas, I'll create an issue in the
apache's issues.

Thanks!