You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by yukon <yu...@apache.org> on 2021/10/01 01:40:15 UTC
Re: RIP 24: Introduce multi-protocol support
Hello, community,
Any further opinions about this RIP? If not, we would like to propose a
draft of the new protocol spec.
Regards,
yukon
On Tue, Sep 28, 2021 at 8:14 AM vongosling <vo...@apache.org> wrote:
> Subscribe to the dev mailing list before sending emails. Otherwise, emails
> cannot be sent to the community.
>
>
> i yangkun <ya...@outlook.com> 于2021年9月28日周二 上午8:11写道：
>
> > Status
> > ________________________________
> >
> > * Current Status: Draft
> > * Authors: [aaron-ai](https://github.com/aaron-ai)
> >
> > * Shepherds: Zhanhui Li<ma...@apache.org>, Yukon<mailto:
> > yukon@apache.org>, duhengforever<ma...@gmail.com>
> > * Mailing List discussion: <dev@rocketmq.apache.org<mailto:
> > dev@rocketmq.apache.org>>
> >
> > * Pull Request: #PR_NUMBER
> > * Released: <released_version>
> >
> > Background & Motivation
> > ________________________________
> > What do we need to do
> >
> > * will we add a new module? yes.
> > * will we add new APIs? no.
> >
> > * will we add a new feature? yes.
> >
> > Why should we do that
> >
> > * Are there any problems with our current project?
> > Currently, the RocketMQ kernel supports only one transport layer protocol
> > available, aka, Remoting. Despite its simplicity and high performance,
> the
> > remoting protocol has some disadvantages:
> >
> >
> > 1. Lack of formal protocol specification. Remoting is loosely defined,
> > lacking a necessary specification document to describe each protocol
> > detail. This indicates much uncertainty, obstructing community
> developers,
> > from different backgrounds, from building uniformly-behaved SDKs and
> > further raises the bar of participation because they have to read the
> > existing source codebase before understanding the ins and outs.
> >
> > 2. Remoting is built on top of Netty, not every language has a Netty-like
> > library. Languages like C/C++ have to pave a lot of "preparation" to
> stand
> > at the same start-line. It would be awesome for RocketMQ to have gRPC,
> > which provides a unified foundation and concepts. Based on this unified
> > foundation, SDKs of different languages can be built with shared models
> and
> > designs.
> >
> > 3. Remoting builds virtually everything from the ground, including
> > auto-retry, connection-multiplexing, coding/decoding, and more. Note
> these
> > all have been properly defined and implemented in gRPC-like libraries.
> > Further, these common parts bring a significant portion of the SDK
> > complexity, whereas they are already available in gRPC for all mainstream
> > programming languages in the field.
> >
> > * What can we benefit from the proposed changes?
> > In the future, RocketMQ is supposed to have multi-protocol support, to
> > embrace new technology trends, service mesh for example, and enhance
> > interoperability among different communities, like gRPC, HTTP, MQTT, etc.
> > So we shall introduce a multi-protocol mechanism and add gRPC as a new
> > supported protocol.
> >
> > As we all know, gRPC is widely adopted, and now is a CNCF incubation
> > project. It has a great yet mature ecosystem, including a number of
> > successful community projects. The gRPC-based protocol will bring us more
> > productive features:
> >
> > 1. Build RocketMQ Client SDK with a unified interface.
> >
> > 2. Focus on the logic of RocketMQ itself without much effort on
> the
> > transportation layer.
> >
> > 3. The HTTP2-based gRPC protocol makes RocketMQ more accessible to
> > cloud-native ecology.
> >
> > Goals
> >
> > * What problem is this proposal designed to solve?
> > Introduce a multi-protocol mechanism and add gRPC as a new supported
> > protocol
> > * To what degree should we solve the problem?
> >
> > Support gRPC and Remoting protocol both
> >
> > Non-Goals
> >
> > * What problem is this proposal NOT designed to solve?
> > Nothing specific.
> > * Are there any limits of this proposal?
> > Nothing specific.
> >
> > Changes
> > ________________________________
> > Architecture
> >
> > Nothing specific.
> >
> > Interface Design/Change
> >
> > * Method signature changes. No
> > * Method behavior changes. No
> >
> > * CLI command changes. No
> > * Log format or content changes. No
> >
> > Compatibility, Deprecation, and Migration Plan
> >
> > * Are backward and forward compatibility taken into consideration?
> > Yes, new broker server will support both two protocols
> > * Are there deprecated APIs?
> > Nothing specific.
> >
> > * How do we do the migration?
> > Nothing specific.
> >
> > Implementation Outline
> >
> > We will implement the proposed changes by two phases.
> >
> > Phase 1
> >
> > 1. Define the new protocol based on gRPC and reach a consensus in the
> > community.
> > 2. Implement the gRPC server in the rocketmq kernel
> >
> > 1. Implement Java SDK based on gRPC protocol
> >
> > Phase 2
> >
> > 1. Implement multilingual SDK, like CPP, Golang, python, C#, etc.
> >
> >
>
> --
> Best Regards :-)
>
Re: RIP 24: Introduce multi-protocol support
Posted by yukon <yu...@apache.org>.
We have drafted the new rocketmq protocol and APIs, any feedback is
welcome.
Please refer https://github.com/apache/rocketmq-apis/pull/1/files for more
details.
Regards
On Fri, Oct 1, 2021 at 9:40 AM yukon <yu...@apache.org> wrote:
> Hello, community,
>
> Any further opinions about this RIP? If not, we would like to propose a
> draft of the new protocol spec.
>
> Regards,
> yukon
>
> On Tue, Sep 28, 2021 at 8:14 AM vongosling <vo...@apache.org> wrote:
>
>> Subscribe to the dev mailing list before sending emails. Otherwise, emails
>> cannot be sent to the community.
>>
>>
>> i yangkun <ya...@outlook.com> 于2021年9月28日周二 上午8:11写道：
>>
>> > Status
>> > ________________________________
>> >
>> > * Current Status: Draft
>> > * Authors: [aaron-ai](https://github.com/aaron-ai)
>> >
>> > * Shepherds: Zhanhui Li<ma...@apache.org>, Yukon<mailto:
>> > yukon@apache.org>, duhengforever<ma...@gmail.com>
>> > * Mailing List discussion: <dev@rocketmq.apache.org<mailto:
>> > dev@rocketmq.apache.org>>
>> >
>> > * Pull Request: #PR_NUMBER
>> > * Released: <released_version>
>> >
>> > Background & Motivation
>> > ________________________________
>> > What do we need to do
>> >
>> > * will we add a new module? yes.
>> > * will we add new APIs? no.
>> >
>> > * will we add a new feature? yes.
>> >
>> > Why should we do that
>> >
>> > * Are there any problems with our current project?
>> > Currently, the RocketMQ kernel supports only one transport layer
>> protocol
>> > available, aka, Remoting. Despite its simplicity and high performance,
>> the
>> > remoting protocol has some disadvantages:
>> >
>> >
>> > 1. Lack of formal protocol specification. Remoting is loosely defined,
>> > lacking a necessary specification document to describe each protocol
>> > detail. This indicates much uncertainty, obstructing community
>> developers,
>> > from different backgrounds, from building uniformly-behaved SDKs and
>> > further raises the bar of participation because they have to read the
>> > existing source codebase before understanding the ins and outs.
>> >
>> > 2. Remoting is built on top of Netty, not every language has a
>> Netty-like
>> > library. Languages like C/C++ have to pave a lot of "preparation" to
>> stand
>> > at the same start-line. It would be awesome for RocketMQ to have gRPC,
>> > which provides a unified foundation and concepts. Based on this unified
>> > foundation, SDKs of different languages can be built with shared models
>> and
>> > designs.
>> >
>> > 3. Remoting builds virtually everything from the ground, including
>> > auto-retry, connection-multiplexing, coding/decoding, and more. Note
>> these
>> > all have been properly defined and implemented in gRPC-like libraries.
>> > Further, these common parts bring a significant portion of the SDK
>> > complexity, whereas they are already available in gRPC for all
>> mainstream
>> > programming languages in the field.
>> >
>> > * What can we benefit from the proposed changes?
>> > In the future, RocketMQ is supposed to have multi-protocol support, to
>> > embrace new technology trends, service mesh for example, and enhance
>> > interoperability among different communities, like gRPC, HTTP, MQTT,
>> etc.
>> > So we shall introduce a multi-protocol mechanism and add gRPC as a new
>> > supported protocol.
>> >
>> > As we all know, gRPC is widely adopted, and now is a CNCF incubation
>> > project. It has a great yet mature ecosystem, including a number of
>> > successful community projects. The gRPC-based protocol will bring us
>> more
>> > productive features:
>> >
>> > 1. Build RocketMQ Client SDK with a unified interface.
>> >
>> > 2. Focus on the logic of RocketMQ itself without much effort on
>> the
>> > transportation layer.
>> >
>> > 3. The HTTP2-based gRPC protocol makes RocketMQ more accessible
>> to
>> > cloud-native ecology.
>> >
>> > Goals
>> >
>> > * What problem is this proposal designed to solve?
>> > Introduce a multi-protocol mechanism and add gRPC as a new supported
>> > protocol
>> > * To what degree should we solve the problem?
>> >
>> > Support gRPC and Remoting protocol both
>> >
>> > Non-Goals
>> >
>> > * What problem is this proposal NOT designed to solve?
>> > Nothing specific.
>> > * Are there any limits of this proposal?
>> > Nothing specific.
>> >
>> > Changes
>> > ________________________________
>> > Architecture
>> >
>> > Nothing specific.
>> >
>> > Interface Design/Change
>> >
>> > * Method signature changes. No
>> > * Method behavior changes. No
>> >
>> > * CLI command changes. No
>> > * Log format or content changes. No
>> >
>> > Compatibility, Deprecation, and Migration Plan
>> >
>> > * Are backward and forward compatibility taken into consideration?
>> > Yes, new broker server will support both two protocols
>> > * Are there deprecated APIs?
>> > Nothing specific.
>> >
>> > * How do we do the migration?
>> > Nothing specific.
>> >
>> > Implementation Outline
>> >
>> > We will implement the proposed changes by two phases.
>> >
>> > Phase 1
>> >
>> > 1. Define the new protocol based on gRPC and reach a consensus in the
>> > community.
>> > 2. Implement the gRPC server in the rocketmq kernel
>> >
>> > 1. Implement Java SDK based on gRPC protocol
>> >
>> > Phase 2
>> >
>> > 1. Implement multilingual SDK, like CPP, Golang, python, C#, etc.
>> >
>> >
>>
>> --
>> Best Regards :-)
>>
>