You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2021/09/06 02:33:54 UTC
[GitHub] [incubator-inlong] gosonzhang opened a new issue #1524: The feature of the TubeMQ replica
gosonzhang opened a new issue #1524:
URL: https://github.com/apache/incubator-inlong/issues/1524
In our environment, there is a situation where the number of concurrent filter consumption of Topic in some clusters has reached 100, and it is likely to exceed 100 to 200, or even 300, but the number of concurrent consumption of our single node is limited. So, we want to add a replica solution to TubeMQ to support over 100 concurrent filtered consumption.
Considering that the current TubeMQ mainly supports high-performance scenarios, we consider the asynchronous replication copy scheme, which is similar to Kafka's replica scheme, but is slightly different:
1. Cluster's Broker nodes that support the replication scheme are organized according to Group, and the nodes in the Group are assigned as one Leader and more Follower nodes; Follower nodes pulls data asynchronously from Leader node and saves it locally;
2. The Leader node in the Group undertakes production and a small amount of consumption; the Follower nodes only do pure consumption processing. When the Leader node fails, the Follower nodes can be switched to the Leader node and perform related change processing;
3. The producer and consumer have no perception of the replica, and still maintain the existing realization for production and consumption
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] gosonzhang commented on issue #1524: The feature of the TubeMQ replica
Posted by GitBox <gi...@apache.org>.
gosonzhang commented on issue #1524:
URL: https://github.com/apache/incubator-inlong/issues/1524#issuecomment-913299473
I will create a development branch for this feature and build it separately. If you have good suggestions, please discuss them in the mail group[1]:
1. https://lists.apache.org/thread.html/rdb3b75bd59bc9c8962c134a2b24c978577ac79df6054c28a46dfb765%40%3Cdev.inlong.apache.org%3E
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org