You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by "imjyz (via GitHub)" <gi...@apache.org> on 2025/12/10 08:04:38 UTC

[D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

GitHub user imjyz created a discussion: 通过消费者标签支持灰度发布和多环境

rocketmq 不支持 多环境、灰度发布等场景,虽然可以通过动态划分 consumer group 来实现,但是会带来额外的运维成本,且不够灵活,无法支持降级场景

希望通过 consumer-client 支持配置 **消费者标签** 和 **过滤条件** 来支持,consumer 拉取消息时,broken 通过匹配**消费者标签**路由消息

完整内容:https://github.com/imjyz/rocketmq/blob/develop/docs/en/RIP-84-Consumer-Tag.md


GitHub link: https://github.com/apache/rocketmq/discussions/9917

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz edited a comment on the discussion: 通过消费者标签支持灰度发布和多环境

优化后的技术方案如下:
[Rocketmq支持消息路由](https://github.com/imjyz/rocketmq/blob/develop/docs/cn/support_message_router.md)


GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-16925517

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz edited a discussion: 通过消费者标签支持灰度发布和多环境

rocketmq 不支持 多环境、灰度发布等场景,虽然可以通过动态划分 consumer group 来实现,但是会带来额外的运维成本,且不够灵活,无法支持降级场景

希望通过 consumer-client 支持配置 **消费者标签** 和 **过滤条件** 来支持,consumer 拉取消息时,broken 通过匹配**消费者标签**路由消息

完整内容:https://github.com/imjyz/rocketmq/blob/develop/docs/en/RIP-84-Consumer-Tag.md
RIP 邮件:https://lists.apache.org/thread/77opwn1oyy52cc89lnrzy01ht3qpxsc7
这里用来同步讨论过程


GitHub link: https://github.com/apache/rocketmq/discussions/9917

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz added a comment to the discussion: 通过消费者标签支持灰度发布和多环境

我已经更新了 RIP 文档,补充了更详细的设计说明:

- [中文版](https://github.com/imjyz/rocketmq/blob/develop/docs/cn/RIP-84-%E6%B6%88%E8%B4%B9%E8%80%85%E6%A0%87%E7%AD%BE.md)
- [英文版](https://github.com/imjyz/rocketmq/blob/develop/docs/en/RIP-84-Consumer-Tag.md)

关于轻量化的问题:本方案仅新增了缓存机制,未引入任何新组件,客户端使用依然足够轻量。
其他问题因内容较多,已在文档中详细解答,请参见:

- [Broker 端变化](https://github.com/imjyz/rocketmq/blob/develop/docs/cn/RIP-84-%E6%B6%88%E8%B4%B9%E8%80%85%E6%A0%87%E7%AD%BE.md#broker-%E7%AB%AF)
- [被否定的替代方案](https://github.com/imjyz/rocketmq/blob/develop/docs/cn/RIP-84-%E6%B6%88%E8%B4%B9%E8%80%85%E6%A0%87%E7%AD%BE.md#%E8%A2%AB%E5%90%A6%E5%86%B3%E7%9A%84%E6%9B%BF%E4%BB%A3%E6%96%B9%E6%A1%88)

GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-15324966

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz added a comment to the discussion: 通过消费者标签支持灰度发布和多环境

优化后的技术方案如下:
[support_message_router.md](https://github.com/user-attachments/files/27784703/support_message_router.md)


GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-16925517

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz edited a discussion: 通过消费者标签支持灰度发布和多环境

rocketmq 不支持 多环境、灰度发布等场景,虽然可以通过动态划分 consumer group 来实现,但是会带来额外的运维成本,且不够灵活,无法支持降级场景

希望通过 consumer-client 支持配置 **消费者标签** 和 **过滤条件** 来支持,consumer 拉取消息时,broken 通过匹配**消费者标签**路由消息

完整内容:https://github.com/imjyz/rocketmq/blob/develop/docs/en/RIP-84-Consumer-Tag.md
RIP 邮件:https://lists.apache.org/thread/77opwn1oyy52cc89lnrzy01ht3qpxsc7

**这里用来同步讨论过程**


GitHub link: https://github.com/apache/rocketmq/discussions/9917

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz added a comment to the discussion: 通过消费者标签支持灰度发布和多环境

方案被否定,原因:带来了写放大,会降低MQ性能和稳定性。

GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-16925534

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz added a comment to the discussion: 通过消费者标签支持灰度发布和多环境

社区回复:

该提案可能需要更详细的设计,尤其是在 Broker 端。

在当前模型中,同一个消费者组内的所有消费者共享相同的订阅关系,从队列拉取的消息可以被分配给组内的任意消费者。然而,在这个新场景中,每个消费者实际上拥有不同的“订阅”(通过基于 Tag 的过滤实现)。这引发了一些关键问题:

* 消息缓存策略:当不同消费者具有差异化的过滤逻辑时,拉取到的消息应如何缓存?
* 容错与可靠性:若某个消费者发生故障,如何确保消息不丢失并尽可能减少重复消费?
* 资源风险:如果某个消费者卡住或严重滞后,其关联的缓存是否会无限增长?
* 轻量化目标:考虑到本方案旨在支持灰度发布(canary deployment)能力,理想情况下,这部分实现应尽可能轻量。

此外,还有一种替代方案:为每个灰度版本创建独立的消费者组,并配置不同的订阅过滤规则。该方案也能实现类似效果(尽管在细粒度流量控制方面存在一定局限),且实现更简单,无需改动现有核心逻辑。

能否请你进一步说明:相比上述替代方案,本提案具体有哪些优势?

GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-15267061

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org


Re: [D] 通过消费者标签支持灰度发布和多环境 [rocketmq]

Posted by "imjyz (via GitHub)" <gi...@apache.org>.
GitHub user imjyz added a comment to the discussion: 通过消费者标签支持灰度发布和多环境

把 client 从方案中抽离出来,client支持携带扩展属性,类似 pulsar 的 `producer.properties` 和 `consumer.properties` ,issue:
https://github.com/apache/rocketmq-apis/issues/107

GitHub link: https://github.com/apache/rocketmq/discussions/9917#discussioncomment-16925548

----
This is an automatically sent email for dev@rocketmq.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@rocketmq.apache.org