You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dubbo.apache.org by hy...@apache.org on 2019/10/16 05:42:03 UTC
[dubbo-website] branch master updated: translate
blog/dubbo-protocol.md to English (#495)
This is an automated email from the ASF dual-hosted git repository.
hyunkun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dubbo-website.git
The following commit(s) were added to refs/heads/master by this push:
new dd30e24 translate blog/dubbo-protocol.md to English (#495)
dd30e24 is described below
commit dd30e24528d58732a391248c009eeeb49cd75534
Author: catxu <ca...@foxmail.com>
AuthorDate: Wed Oct 16 13:41:56 2019 +0800
translate blog/dubbo-protocol.md to English (#495)
---
blog/en-us/dubbo-protocol.md | 182 +++++++++++++++++++++++++++++++++++++++++++
blog/zh-cn/dubbo-protocol.md | 34 ++++----
site_config/blog.js | 15 +++-
3 files changed, 210 insertions(+), 21 deletions(-)
diff --git a/blog/en-us/dubbo-protocol.md b/blog/en-us/dubbo-protocol.md
new file mode 100644
index 0000000..4946dc9
--- /dev/null
+++ b/blog/en-us/dubbo-protocol.md
@@ -0,0 +1,182 @@
+---
+title: Introduction to the Dubbo protocol
+keywords: Dubbo, Protocol, RPC
+description: This article introduces the design of the Dubbo protocol.
+---
+
+## The concept of the protocol
+The protocol is the foundation of communication between two network entities, and data is transmitted from one entity to another in the form of a byte stream over the network. In the world of byte streams, this one-dimensional byte stream cannot be reshaped into two-dimensional or multi-dimensional data structures and domain objects without a protocol.
+
+
+### What is the protocol
+The protocol is the semantics determined by both parties for the communication. For example, we design a protocol for string transmission, which allows the client to send a string and the server receives the corresponding string.
+
+This protocol is quite simple, first 4-byte represent the total length of the message, followed by 1-byte represents the length of the charset, and then followed by the message payload, charset name and string body.
+
+For example, Send a string `abc` encoded by `iso-8859-1` to the server end. After protocol trans-coding, the content is: `18 = 4 + 1 + 10 + 3|10|iso-8859-1|abc`, when the server receives these byte streams, it first reads 4-byte and converts it to int, in this case is 18, which is the length of message. Then continue to read the remaining 14 bytes, the remaining first byte read will be the length of the charset name, which is 10, the after 10-byte content is converted to a string, the co [...]
+
+In the previous example of a custom string transfer protocol, we have already seen the role of protocols in data transmission. Without a protocol, the data exchange cannot be completed. The following is Wikipedia's definition of the communication protocol.
+
+> In telecommunication, a communication protocol is a system of rules that allow two or more entities of a communications system to transmit information via any kind of variation of a physical quantity. The protocol defines the rules syntax, semantics and synchronization of communication and possible error recovery methods. Protocols may be implemented by hardware, software, or a combination of both.
+
+Communication protocols need to define syntax, semantics, and communication synchronization operations. The content described here is actually a formal description of the previous custom string transfer protocol.
+
+### The definition of `Codec`
+
+`org.apache.dubbo.remoting.Codec2` define the **Codec**'s I/O processing, so the main methods are `encode` and `decode`, as defined below:
+
+```java
+@SPI
+public interface Codec2 {
+
+ @Adaptive({Constants.CODEC_KEY})
+ void encode(Channel channel, ChannelBuffer buffer, Object message) throws IOException;
+
+ @Adaptive({Constants.CODEC_KEY})
+ Object decode(Channel channel, ChannelBuffer buffer) throws IOException;
+
+
+ enum DecodeResult {
+ NEED_MORE_INPUT, SKIP_SOME_INPUT
+ }
+
+}
+```
+
+ `Codec` works on a protocol, `encode` method means to encode the communication object to the `ByteBufferWrapper`, and `decode` method decodes the `ChannelBuffer` read from the network into `Object`, which is the communication object.
+
+
+## Common protocol mode
+Protocols in application layer have three general forms: fixed-length protocol, special delimiter protocol and protocol header + payload mode. The specific contents of these forms are described below.
+
+When reading data from the network in the form of a byte stream, it is necessary to determine when a meaningful transmission content ends. Moreover, the transmission of data on the network, there are sticky packets and fragment packets problem. The solution to this problem is that the protocol can accurately identify, when the sticky packets or fragment packets occurs, it can handle it correctly.
+
+
+### Fixed-length protocol
+
+The fixed-length protocol means that the length of the protocol content is fixed. For example, the protocol byte length is 50. When 50 bytes are read from the network, the decoding operation is performed. The fixed-length protocol is more efficient when reading or writing, because the size of the data cache is basically determined, just like an array. The defect is the lack of adaptability. Taking the RPC scene as an example, it is difficult to estimate the length.
+
+> Refer to Netty's `FixedLengthFrameDecoder`
+
+### Special delimiter protocol
+Compared to fixed-length protocols, if we can define a special character as the end of each protocol unit, we can communicate in a variable length, so as to balance the data transmission and efficiency, such as the delimiter \n.
+
+The problem with the special delimiter method is that it think about the process of protocol transmission in a simple way. For a protocol unit, it must be read all before it can be processed. In addition, it must be prevented that the data transmitted by the user cannot be the same as the delimiter, otherwise it will be disordered.
+
+> Refer to Netty's `DelimiterBasedFrameDecoder`
+
+### Variable length protocol(protocol head + payload)
+
+Generally, it is a custom protocol, which is composed of a fixed length and a variable content. The fixed length part needs to specify the length of the variable content part.
+
+```sh
++————————————————+
+|fixed-length |
++————————————————+
+|variable content|
++————————————————+
+```
+
+> Refer to Netty's `LengthFieldBasedFrameDecoder`
+
+The Dubbo protocol is actually a variable length protocol, which is covered in more detail in later chapters.
+
+## The Dubbo protocol
+
+### Overview to the Dubbo protocol
+The Dubbo framework defines a proprietary RPC protocol in which the specific content of the request and response protocols is presented using a table.
+
+
+
+### Dubbo protocol details
+
+- Magic - Magic High & Magic Low (16 bits)
+
+ Identify protocol version, Dubbo protocol: 0xdabb
+
+- Req/Res (1 bit)
+
+ Identify a request or response. request: 1; response: 0.
+
+- 2 Way (1 bit)
+
+ Only useful when Req/Res is 1(request), identifying if you expect to return a value from the server. Set to 1 if a return value from the server is required.
+
+- Event (1 bit)
+
+ Identifies whether it is an event message, for example, a heartbeat event. Set to 1 if this is an event.
+
+- Serialization ID (5 bit)
+
+ Identifies the serialization type: for example, the value of fastjson is 6.
+
+- Status (8 bits)
+
+ Only useful when Req/Res is 0 (response), used to identify the status of the response
+
+ - 20 - OK
+ - 30 - CLIENT_TIMEOUT
+ - 31 - SERVER_TIMEOUT
+ - 40 - BAD_REQUEST
+ - 50 - BAD_RESPONSE
+ - 60 - SERVICE_NOT_FOUND
+ - 70 - SERVICE_ERROR
+ - 80 - SERVER_ERROR
+ - 90 - CLIENT_ERROR
+ - 100 - SERVER_THREADPOOL_EXHAUSTED_ERROR
+
+- Request ID (64 bits)
+
+ Identifies the only request, the type is long.
+
+- Data Length (32 bits)
+
+ The length of the serialized content (variable part), counted in bytes, the type is int.
+
+- Variable Part
+
+ After being serialized by a specific serialization type (identified by the serialization ID), each part is a byte [] or byte.
+
+ - If it is a request packet (Req/Res = 1), each part is:
+ - Dubbo version
+ - Service name
+ - Service version
+ - Method name
+ - Method parameter types
+ - Method arguments
+ - Attachments
+ - If it is a response packet (Req/Res = 0), each part is:
+ - Return value's type (byte), identifying the type of value returned from the server:
+ - Return null: RESPONSE_NULL_VALUE 2
+ - Normal response value: RESPONSE_VALUE 1
+ - Exception: RESPONSE_WITH_EXCEPTION 0
+ - Return value: response bytes returned from the server
+
+**Note: ** For the variable part, when uses json serialization in current version of Dubbo framework, an additional line break is added as a separator between each part of the content. Please add a new line break after each part of the variable part, such as:
+
+```
+Dubbo version bytes (line break)
+Service name bytes (line break)
+...
+```
+
+## Advantages and disadvantages about the Dubbo protocol
+
+### Advantages
+
+- The protocol is designed to be very compact. The data type can be represented by 1 bit will not be represented by a byte, such as a boolean type identifier.
+- The request header is the same as response header, and the specific content is assembled to the variable part through the serializer, and it is simple to implement.
+
+### To be improved
+
+- Similar to the http request, the header can determine the resource to be accessed, and Dubbo needs to involve the specific serializer to resolve the service name, method, method signature, and these resource locators are string type or string array. It's easy to convert to bytes, so it can be assembled into the header. Similar to http2 header compression, resources called for rpc can also be negotiated with an int to identify, which improves performance. This is easier to implement if [...]
+
+- Use req/res to determine if it is a request, then you can refine the protocol, remove some unwanted identifiers and add some specific identifiers. For example, `status`, `twoWay` can be strictly customized to remove redundant identifiers. There is also a timeout identifier that is transmitted as the `attachment` of the Dubbo. In theory, it should be placed in the header of the request protocol, because the timeout is essential in the network request. Referring to `attachment`, you can [...]
+
+- Dubbo will convert the service name `com.alibaba.middleware.hsf.guide.api.param.ModifyOrderPriceParam` to `Lcom/alibaba/middleware/hsf/guide/api/param/ModifyOrderPriceParam;`, which is theoretically unnecessary, simply append a `;` will be fine.
+
+- The Dubbo protocol does not reserve extension fields, and cannot add new identifiers. The scalability is not very good. For example, the only way to add the `response context` function is to change the protocol version, but this requires both the client and server versions to be upgraded. Very unfriendly in distributed scenarios.
+
+## Conclusion
+
+This article mainly introduces the concept of the protocol and the commonly used protocol mode. Then it analyzes the Dubbo protocol in detail and also mentions some shortcomings. However, compared with its simplicity and ease of implementation, the above shortcomings are not enough to drive us to redesign a new version of the protocol, so we welcome suggestions and features for protocol optimization.
diff --git a/blog/zh-cn/dubbo-protocol.md b/blog/zh-cn/dubbo-protocol.md
index f978bec..39f142e 100644
--- a/blog/zh-cn/dubbo-protocol.md
+++ b/blog/zh-cn/dubbo-protocol.md
@@ -1,7 +1,7 @@
---
-title: DUBBO协议详解
+title: Dubbo 协议详解
keywords: Dubbo, Protocol, RPC
-description: 本文介绍了dubbo协议的设计
+description: 本文介绍了 Dubbo 协议的设计
---
## 协议的概念
@@ -12,7 +12,7 @@ description: 本文介绍了dubbo协议的设计
### 协议是什么
-协议是双方确定的交流语义，比如：我们设计一个字符串传输的协议，它允许客户端发送一个字符串，服务端接收到对应的字符 串。这个协议很简单，首先发送一个4字节的消息总长度，然后再发送1字节的字符集charset长度，接下来就是消息的payload，字符集名称和字符串正文。
+协议是双方确定的交流语义，比如：我们设计一个字符串传输的协议，它允许客户端发送一个字符串，服务端接收到对应的字符串。这个协议很简单，首先发送一个4字节的消息总长度，然后再发送1字节的字符集charset长度，接下来就是消息的payload，字符集名称和字符串正文。
发送一个`iso-8859-1`的字符串`abc`到对端。经过协议编码，内容是：`18 = 4 + 1 + 10 + 3|10|iso-8859-1|abc`，当这些字节流发往服务端后，当服务端收到字节流后，首先读取4个字节，将其转换为int，在这个例子中是18，接下来继续读14个字节，将首个字节得到字符集名称长度10，将后续内容的前10字节转换为字符串，内容是`iso-8859-1`，使用该字符集将后续的字节数组造型成为字符串`new String(bytes, "iso-8859-1")`。
@@ -88,13 +88,13 @@ public interface Codec2 {
> 可以参考Netty的`LengthFieldBasedFrameDecoder`
-DUBBO协议实际上就是一种变长协议，后面的章节会详细介绍。
+Dubbo 协议实际上就是一种变长协议，后面的章节会详细介绍。
-## DUBBO协议
+## Dubbo 协议
### 协议概览
-DUBBO框架定义了私有的RPC协议，其中请求和响应协议的具体内容我们使用表格来展示。
+Dubbo 框架定义了私有的RPC协议，其中请求和响应协议的具体内容我们使用表格来展示。
@@ -102,7 +102,7 @@ DUBBO框架定义了私有的RPC协议，其中请求和响应协议的具体内
- Magic - Magic High & Magic Low (16 bits)
- 标识协议版本号，dubbo 协议：0xdabb
+ 标识协议版本号，Dubbo 协议：0xdabb
- Req/Res (1 bit)
@@ -122,7 +122,7 @@ DUBBO框架定义了私有的RPC协议，其中请求和响应协议的具体内
- Status (8 bits)
- 仅在 Req/Res 为0（响应）时有用，用于标识响应的状态
+ 仅在 Req/Res 为0（响应）时有用，用于标识响应的状态。
- 20 - OK
- 30 - CLIENT_TIMEOUT
@@ -162,7 +162,7 @@ DUBBO框架定义了私有的RPC协议，其中请求和响应协议的具体内
- 异常：RESPONSE_WITH_EXCEPTION 0
- 返回值：从服务端返回的响应bytes
-**注意：**对于(Variable Part)变长部分，当前版本的dubbo框架使用json序列化时，在每部分内容间额外增加了换行符作为分隔，请选手在Variable Part的每个part后额外增加换行符， 如：
+**注意：**对于(Variable Part)变长部分，当前版本的Dubbo 框架使用json序列化时，在每部分内容间额外增加了换行符作为分隔，请在Variable Part的每个part后额外增加换行符， 如：
```
Dubbo version bytes (换行符)
@@ -170,27 +170,27 @@ Service name bytes (换行符)
...
```
-## DUBBO协议的优缺点
+## Dubbo 协议的优缺点
### 优点
-- 协议设计上很紧凑，可用用 1 个 bit 表示的，不会用一个 byte 来表示，比如 boolean 类型的标识
-- 请求、响应的 header 一致，通过序列化器对 content 组装特定的内容，代码实现起来简单
+- 协议设计上很紧凑，能用 1 个 bit 表示的，不会用一个 byte 来表示，比如 boolean 类型的标识。
+- 请求、响应的 header 一致，通过序列化器对 content 组装特定的内容，代码实现起来简单。
### 可以改进的点
-- 类似于 http 请求，通过 header 就可以确定要访问的资源，而 dubbo 需要涉及到用特定序列化协议才可以将服务名、方法、方法签名解析出来，并且这些资源定位符是 string 类型或者 string 数组，很容易转成 bytes，因此可以组装到 header 中。类似于 http2 的 header 压缩，对于 rpc 调用的资源也可以协商出来一个int来标识，从而提升性能，如果在`header`上组装资源定位符的话，该功能则更易实现。
+- 类似于 http 请求，通过 header 就可以确定要访问的资源，而 Dubbo 需要涉及到用特定序列化协议才可以将服务名、方法、方法签名解析出来，并且这些资源定位符是 string 类型或者 string 数组，很容易转成 bytes，因此可以组装到 header 中。类似于 http2 的 header 压缩，对于 rpc 调用的资源也可以协商出来一个int来标识，从而提升性能，如果在`header`上组装资源定位符的话，该功能则更易实现。
-- 通过 req/res 是否是请求后，可以精细定制协议，去掉一些不需要的标识和添加一些特定的标识。比如`status`,`twoWay`标识可以严格定制，去掉冗余标识。还有超时时间是作为 DUBBO 的 `attachment` 进行传输的，理论上应该放到请求协议的header中，因为超时是网络请求中必不可少的。提到 `attachment` ，通过实现可以看到 `attachment` 中有一些是跟协议 `content`中已有的字段是重复的，比如 `path`和`version`等字段，这些会增大协议尺寸。
+- 通过 req/res 是否是请求后，可以精细定制协议，去掉一些不需要的标识和添加一些特定的标识。比如`status`,`twoWay`标识可以严格定制，去掉冗余标识。还有超时时间是作为 Dubbo 的 `attachment` 进行传输的，理论上应该放到请求协议的header中，因为超时是网络请求中必不可少的。提到 `attachment` ，通过实现可以看到 `attachment` 中有一些是跟协议 `content`中已有的字段是重复的，比如 `path`和`version`等字段，这些会增大协议尺寸。
-- DUBBO 会将服务名`com.alibaba.middleware.hsf.guide.api.param.ModifyOrderPriceParam`，转换为`Lcom/alibaba/middleware/hsf/guide/api/param/ModifyOrderPriceParam;`，理论上是不必要的，最后追加一个`;`即可。
+- Dubbo 会将服务名`com.alibaba.middleware.hsf.guide.api.param.ModifyOrderPriceParam`，转换为`Lcom/alibaba/middleware/hsf/guide/api/param/ModifyOrderPriceParam;`，理论上是不必要的，最后追加一个`;`即可。
-- DUBBO协议没有预留扩展字段，没法新增标识，扩展性不太好，比如新增`响应上下文`的功能，只有改协议版本号的方式，但是这样要求客户端和服务端的版本都进行升级，对于分布式场景很不友好。
+- Dubbo 协议没有预留扩展字段，没法新增标识，扩展性不太好，比如新增`响应上下文`的功能，只有改协议版本号的方式，但是这样要求客户端和服务端的版本都进行升级，对于分布式场景很不友好。
## 总结
-本文主要介绍了协议的概念和常用的协议模式，后面对DUBBO协议进行了详细分析，也提到了一些不足的地方，但是相对于其简洁性和易于实现性，以上提出的缺点不足以有动力设计出一个新版本的协议，所以欢迎大家提出对协议优化方面的建议和特性。
+本文主要介绍了协议的概念和常用的协议模式，后面对 Dubbo 协议进行了详细分析，也提到了一些不足的地方，但是相对于其简洁性和易于实现性，以上提出的缺点不足以有动力设计出一个新版本的协议，所以欢迎大家提出对协议优化方面的建议和特性。
diff --git a/site_config/blog.js b/site_config/blog.js
index 2a93c6f..4a861e4 100644
--- a/site_config/blog.js
+++ b/site_config/blog.js
@@ -125,7 +125,7 @@ export default {
{
title: 'The first Dubbo meetup has been held in Beijing',
author: 'Huxing Zhang',
- dateStr: 'May 12nd，2018',
+ dateStr: 'May 12th，2018',
desc: 'The first Dubbo meetup has successfully been held in Beijing, over 400+ people were present. What a great event! ',
link: '/en-us/blog/dubbo-meetup-beijing-may-12th-2018.html',
},
@@ -146,7 +146,7 @@ export default {
{
title: 'Dubbo roadmap is announced in QCon Beijing 2018',
author: '@Huxing Zhang',
- dateStr: 'April 22nd，2018',
+ dateStr: 'April 22th，2018',
desc: 'Ian Luo has delivered a great talk at QCon Beijing 2018, where the roadmap of Dubbo has also be announced',
link: '/en-us/blog/qcon-beijing-2018.html',
},
@@ -184,6 +184,13 @@ export default {
dateStr: 'August 7th，2018',
desc: 'Your First Dubbo Demo',
link: '/en-us/blog/dubbo-101.html',
+ },
+ {
+ title: 'Dubbo protocol',
+ author: '@catxu',
+ dateStr: 'Oct 11th, 2019',
+ desc: 'This article introduces the common protocol mode and the design of Dubbo protocol',
+ link: '/en-us/blog/dubbo-protocol.html',
}
]
},
@@ -222,7 +229,7 @@ export default {
{
title: 'Dubbo2.7 三大新特性详解',
author: '@lexburner',
- dateStr: 'Mar 22nd, 2019',
+ dateStr: 'Mar 22th, 2019',
desc: '异步化改造,三大中心改造,服务治理增强',
link: '/zh-cn/blog/dubbo-27-features.html',
},
@@ -236,7 +243,7 @@ export default {
{
title: 'Dubbo优雅停机介绍',
author: '@guohao',
- dateStr: 'Feb 22nd, 2019',
+ dateStr: 'Feb 22th, 2019',
desc: 'Dubbo优雅停机的实现背景和实践',
link: '/zh-cn/blog/dubbo-gracefully-shutdown.html',
},