You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by Yin James <yw...@hotmail.com> on 2021/06/13 10:14:09 UTC
答复: RIP23: Support gRPC protocol
从我的经验来看，grpc不是为双向对等通信设计的，对rich client很不友好（包括rebalance，transaction，request-reply）。而且使用grpc就要将状态与tcp connection解耦，重新创建一个session layer，具体怎么实现需要更深入的讨论（比如重新设计协议层）。
综上，我认为grpc在多语言客户端方面不具有显著优势，而且会为多语言客户端的维护带来更大的负担。
James Yin.
________________________________
发件人: Zhanhui Li <li...@apache.org>
发送时间: 2021年6月9日 11:10
收件人: dev <de...@rocketmq.apache.org>
主题: Re: RIP23: Support gRPC protocol
Hi,
This proposal, in general, is in the right direction that helps RocketMQ
provide full-fledged SDK for popular languages and platforms. Taking full
advantage of gRPC does save a lot of effort in terms of serialization and
RPC tiers. Obviously, this proposal also brings complexities and potential
compatibility issues.
One of the potential issues is that gRPC does not expose Channel in the
implementation while RocketMQ processors make heavy use of it, even if both
of them are built on top of Netty 4.x. Will this an issue when reuse
existing code?
Zhanhui Li
On Tue, Jun 8, 2021 at 8:28 PM i yangkun <ya...@outlook.com> wrote:
> Background & Motivation
> What do we need to do
>
>
> * Will we add a new module?
> maybe.
> * Will we add new APIs?
> Yes.
>
> * Will we add new feature?
> Yes.
>
>
> Why should we do that
>
>
> * Are there any problems of our current project?
>
> a. Remoting module is too complicated to maintain, gRPC makes it easier to
> establish a robust communication layer, the current remoting module would
> be simplified radically.
>
> b. gRPC has been the de-facto standard in CloudNative, service mesh would
> be easily applied if gRPC is enabled.
>
> c. The private protocol of RocketMQ depends on the FastJson, it is
> difficult to adapt for other language.
>
> On the other side, since the pop consumer has been merged, we could
> implement new SDK based on gRPC and pop, which is easier to develop and
> maintain.
>
> Chinese Version:
>
> a. Remoting 模块对于长期的维护而言过于复杂了，我们可以使用 gRPC 更轻松地建立起一个健壮的通信层，这会使得现有的 remoting
> 模块从根本上得到简化。
>
> b. gRPC 目前已经是云原生时代的事实标准，使用 gRPC 可以使得我们天然获取一些云原生的能力，比如 Service Mesh。
>
> c. 目前 RocketMQ 的私有协议强烈依赖 FastJson，多语言的适配将会变得困难。
>
>
> 从另外一个角度来说，鉴于 pop 消费者已经被合并，我们可以基于 gRPC 和 pop 实现新的 SDK，新的 SDK 将会更加易于开发和维护。
>
> Goals
>
>
> * What problem is this proposal designed to solve?
>
> Support gRPC's protocol, simplify current communication layer oof
> RocketMQ, make it easier to adapt for other language, which is not limited
> to CPP/GO/C#/GO。
>
> Chinese Version:
>
> 支持 gRPC 协议，简化 RocketMQ 现有的通信层，复用 gRPC 的能力，简化多语言适配成本，不限于 CPP/GO/C#/GO。
>
> * To what degree should we solve the problem?
> This RIP must guarantee below point:
>
> 1. Compatibility: Both of gRPC and RemotingCommand should be supported.
> 2. High performance: This implementation does not affects latency and
> throughput.
>
>
> Chinese Version:
>
> 新方案需要保证两点：
>
> 1. 兼容性：同时支持 gRPC 和 RemotingCommand 协议，不影响现有功能。
> 2. 高性能：基于 gRPC 的实现不影响整理的延时和吞吐量。
>
>
> Non-Goals
>
>
> * What problem is this proposal NOT designed to solve?
> Nothing specific.
> * Are there any limits of this proposal?
> Nothing specific.
>
>
> Changes
> Architecture
>
>
> Current broker processor and client.
>
> [
> https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fintranetproxy.alipay.com%2Fskylark%2Flark%2F0%2F2021%2Fpng%2F200096%2F1623142547507-128b85f5-98f4-4568-85f8-28ef32982b7c.png&data=04%7C01%7C%7C88e7d63cad0a43c62e9208d92af424d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637588050352180814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SNH6pBuq9sTsNX2AczGtHnZba3B3CDazr1XJRNahSkU%3D&reserved=0
> ]
>
> Proposed gRPC processor and client.
>
> [
> https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fintranetproxy.alipay.com%2Fskylark%2Flark%2F0%2F2021%2Fpng%2F200096%2F1623142552491-a7f58ac0-cd7d-4ddd-936e-fb296b667196.png&data=04%7C01%7C%7C88e7d63cad0a43c62e9208d92af424d3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637588050352180814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QP7D3pN5IHXoHd%2FmfJqbky81nrrJkkBMFM69KIRdXXw%3D&reserved=0
> ]
>
> Broker would provide a protocol negotiate procedure to distinguish
> RemotingCommand from gRPC protocol. two kinds or processor in broker would
> re-use the same port to serve for RPC from different SDK.
>
>
> Chinese Version:
>
> broker 本身提供协议协商机制用于区分 RemotingCommnad 和 gRPC 协议，broker 针对 gRPC 和
> RemotingCommand 提供不同的 processor 为各自的 SDK 服务。
>
> Interface Design/Change
>
>
> * Method signature changes
> Nothing specific.
> * Method behavior changes
> Nothing specific.
>
> * CLI command changes
> Nothing specific.
> * Log format or content changes
> Nothing specific.
>
>
> Compatibility, Deprecation, and Migration Plan
>
>
> * Are backward and forward compatibility taken into consideration?
>
> Broker support processor of RemotingCommand and gRPC simultaneously, so
> there are one compatibility situations:
>
> If user migrates from original SDK to gRPC SDK in push mode, the
> re-balance policy should make sure that it would not cause repeated
> consumption for a lot of messages.
>
> * Are there deprecated APIs?
> Nothing specific.
> * How do we do migration?
> Nothing specific.
>
>
> Implementation Outline
>
>
> We will implement the proposed changes by 4 phases.
>
>
> Phase 1
>
> 1. Provides gRPC protocol definition(IDL)
>
> Phase 2
>
> 1. Implement gRPC processor of broker.
> 2. Implement protocol negotiation of two kinds of protocol(gRPC and
> RemotingCommand)
>
> Phase 3
>
> 1. Implement new JAVA and CPP native SDK based on gRPC
>
> Phase 4
>
> 1. Implement native SDK base on gRPC for other language.
>
>
> Rejected Alternatives
>
>
> How does alternatives solve the issue you proposed?
>
>
> Thrift? not so much impact as gRPC in community.
>
>
> Pros and Cons of alternatives
>
>
> Nothing specific.
>
> Why should we reject above alternatives
>
>