You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by me...@apache.org on 2021/11/01 16:52:21 UTC
[shardingsphere] branch master updated: Update shadow document
(#13399)
This is an automated email from the ASF dual-hosted git repository.
menghaoran pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new b6c0b8e Update shadow document (#13399)
b6c0b8e is described below
commit b6c0b8ebb3a593b67bdf608eec5f5be7ce03b2bf
Author: Liang Zhang <te...@163.com>
AuthorDate: Tue Nov 2 00:51:34 2021 +0800
Update shadow document (#13399)
* Update index of shadow
* Update concept of shadow
* Update concept of shadow
* Update use-norms of shadow
* Update concept of shadow
---
docs/document/content/features/shadow/_index.cn.md | 12 +++--
docs/document/content/features/shadow/_index.en.md | 21 ++++----
.../document/content/features/shadow/concept.cn.md | 39 ++++----------
.../document/content/features/shadow/concept.en.md | 38 +++++--------
.../content/features/shadow/use-norms.cn.md | 61 +++++++++------------
.../content/features/shadow/use-norms.en.md | 62 +++++++++-------------
6 files changed, 92 insertions(+), 141 deletions(-)
diff --git a/docs/document/content/features/shadow/_index.cn.md b/docs/document/content/features/shadow/_index.cn.md
index 604147b..6d0fe2b 100644
--- a/docs/document/content/features/shadow/_index.cn.md
+++ b/docs/document/content/features/shadow/_index.cn.md
@@ -7,17 +7,19 @@ chapter = true
## 背景
-在基于微服务的分布式应用架构下，业务需要多个服务是通过一系列的服务、中间件的调用来完成，所以单个服务的压力测试已经不能代表真实场景。
-在测试环境中，如果重新搭建一整套与生产环境类似的压测环境，成本太高，并且往往无法模拟线上环境的体量以及复杂度。
-这种场景下，业内通常选择全链路压测的方式，即在生产环境进行压测，这样所获得的测试结果能够较为准确地反应系统真实容量水平和性能。
+在基于微服务的分布式应用架构下，业务需要多个服务是通过一系列的服务、中间件的调用来完成，所以单个服务的压力测试已无法代表真实场景。
+在测试环境中，如果重新搭建一整套与生产环境类似的压测环境，成本过高，并且往往无法模拟线上环境的复杂度以及流量。
+因此，业内通常选择全链路压测的方式，即在生产环境进行压测，这样所获得的测试结果能够准确地反应系统真实容量和性能水平。
## 挑战
-全链路压测是一项复杂而庞大的工作。需要各个微服务、中间件之间配合与调整，以应对不同流量以及压测标识的透传。通常会搭建一整套压测平台以适用不同测试计划。
+全链路压测是一项复杂而庞大的工作。
+需要各个微服务、中间件之间配合与调整，以应对不同流量以及压测标识的透传。
+通常会搭建一整套压测平台以适用不同测试计划。
在数据库层面需要做好数据隔离，为了保证生产数据的可靠性与完整性，需要将压测产生的数据路由到压测环境数据库，防止压测数据对生产数据库中真实数据造成污染。
这就要求业务应用在执行 SQL 前，能够根据透传的压测标识，做好数据分类，将相应的 SQL 路由到与之对应的数据源。
## 目标
Apache ShardingSphere 关注于全链路压测场景下，数据库层面的解决方案。
-基于内核的 SQL 解析能力，以及可插拔平台架构，实现压测数据与生产数据的隔离，帮助应用自动路由，支持全链路压测，是 Apache ShardingSphere 影子数据库模块的主要设计目标。
+将压测数据自动路由至用户指定的数据库库，是 Apache ShardingSphere 影子库模块的主要设计目标。
diff --git a/docs/document/content/features/shadow/_index.en.md b/docs/document/content/features/shadow/_index.en.md
index 7c12dd6..c91c90d 100644
--- a/docs/document/content/features/shadow/_index.en.md
+++ b/docs/document/content/features/shadow/_index.en.md
@@ -7,25 +7,28 @@ chapter = true
## Background
-Under the distributed application architecture based on microservices, business requires multiple services to be completed through a series of services and middleware calls. The pressure testing of a single service can no longer reflect the real scenario.
+Under the distributed application architecture based on microservices, business requires multiple services to be completed through a series of services and middleware calls.
+The pressure testing of a single service can no longer reflect the real scenario.
-In the test environment, if you rebuild a complete set of pressure test environment similar to the production environment, the cost is too high. It is usually impossible to simulate the scale and complexity of the production environment.
+In the test environment, the cost of rebuild complete set of pressure test environment similar to the production environment is too high.
+It is usually impossible to simulate the complexity and data of the production environment.
-In this scenario, industry usually chooses the full-link pressure test method, that is, pressure test in the production environment. So the test results obtained can more accurately reflect the real capacity level and performance of the system.
+So, it is the better way to use the production environment for pressure test.
+The test results obtained real capacity and performance of the system accurately.
## Challenges
-Full-link pressure testing is a complex and huge task. Coordination and adjustments between microservices and middleware required to cope with the transparent transmission of different flow rates and pressure test marks. Usually we will build a complete set of pressure testing platform for different test plans.
+pressure testing on production environment is a complex and huge task.
+Coordination and adjustments between microservices and middlewares required to cope with the transparent transmission of different flow rates and pressure test tags.
+Usually we will build a complete set of pressure testing platform for different test plans.
-Data isolation have to be done at the database-level, in order to ensure the reliability and integrity of the production data, the data generated by pressure testing routed to the test database.
+Data isolation have to be done at the database-level, in order to ensure the reliability and integrity of the production data, data generated by pressure testing routed to test database.
Prevent test data from polluting the real data in the production database.
This requires business applications to perform data classification based on the transparently transmitted pressure test identification before executing SQL, and route the corresponding SQL to the corresponding data source.
## Goal
-Apache ShardingSphere focuses on database-level solutions in full-link pressure testing scenarios.
+Apache ShardingSphere focuses on data solutions in pressure testing on production environment.
-Kernel-based SQL analysis capabilities and a pluggable platform architecture realize the isolation of pressure test data and production data, help application automatic routing, and support full-link pressure testing.
-
-It is the main design goal of the Apache ShardingSphere shadow DB module.
+The main goal of the Apache ShardingSphere shadow Database module is routing pressure testing data to user defined database automatically.
diff --git a/docs/document/content/features/shadow/concept.cn.md b/docs/document/content/features/shadow/concept.cn.md
index 17d081e..1b05f17 100644
--- a/docs/document/content/features/shadow/concept.cn.md
+++ b/docs/document/content/features/shadow/concept.cn.md
@@ -3,45 +3,28 @@ title = "核心概念"
weight = 1
+++
-## 影子库开关
+## 压测开关
-影子库开关。压力测试是一个特定时段的需求，在需要时开启即可。
+压力测试是一个特定时段的需求，在需要时开启即可。
-## 生产数据库
+## 生产库
生产数据使用的数据库。
-## 影子数据库
+## 影子库
压测数据隔离的影子数据库，与生产数据库应当使用相同的配置。
-## 影子表
-
-进行压测数据相关的表，与生产数据库中对应表具有相同表结构。
-
## 影子算法
-目前提供 2 种类型影子算法。
-由于影子算法和业务实现紧密相关，因此并未提供默认的影子算法。
-
-- 列影子算法
-
-对应 ColumnShadowAlgorithm，适用于用户压测过程中，对压测执行链路上执行的SQL涉及的某个字段的值满足一定匹配条件的场景。
-
-优点：用户只需要控制流量数据不需要修改代码和SQL就可以完成测试。
-
-不足：仅支持DML语句
-
-- 注解影子算法
-
-对应 NoteShadowAlgorithm，适用于用户压测过程中，对压测执行链路上执行的SQL不能确定涉及字段的值的场景。
-
-优点：用户可以不确定链路上执行SQL细节，只要知道那个SQL执行即可。
+影子算法和业务实现紧密相关，目前提供 2 种类型影子算法。
-不足：用户需要改代码或者SQL
+- 基于列的影子算法
-## 默认影子算法
+通过识别 SQL 中的数据，匹配路由至影子库的场景。
+适用于由压测数据名单驱动的压测场景。
-默认影子算法，选配项。对于没有配置影子算法表的默认匹配算法。
+- 基于标记的影子算法
-**注意**：默认影子算法仅支持注解影子算法。
+通过识别 SQL 中的注释，匹配路由至影子库的场景。
+适用于由上游系统透传标识驱动的压测场景。
diff --git a/docs/document/content/features/shadow/concept.en.md b/docs/document/content/features/shadow/concept.en.md
index ec9f9b1..20f6b10 100644
--- a/docs/document/content/features/shadow/concept.en.md
+++ b/docs/document/content/features/shadow/concept.en.md
@@ -3,42 +3,28 @@ title = "Core Concept"
weight = 1
+++
-## Shadow DB Switch
+## Pressure Testing Switcher
-Shadow DB switch.
+Pressure testing is a requirement for a specific period, turned on when needed.
-Pressure testing is a requirement for a specific period of time, turned on when needed.
-
-## Production DB
+## Production Database
The database used for production data.
-## Shadow DB
-
-The Shadow database for Pressure testing data isolation.
-
-## Shadow Table
-
-Pressure testing data related tables.
+## Shadow Database
-The shadow table has the same table structure in the production DB and shadow DB.
+The database for pressure testing data isolation.
## Shadow Algorithm
-Provides 2 types of shadow algorithms.
-
-Since the shadow algorithm is closely related to business, no default shadow algorithm provided.
-
-- Column shadow algorithm
-
-It is suitable for scenarios where the value of a field involved in the executed SQL satisfies certain matching conditions in the testing.
-
-- Note shadow algorithm
+The shadow algorithms are closely related to business, there are 2 types of shadow algorithms provided.
-It is suitable for scenarios where the field values involved in executing SQL cannot meet certain matching conditions in the testing.
+- Column based shadow algorithm
-## Default Shadow Algorithm
+Recognize data from SQL and route to shadow databases.
+Suitable for test data driven scenario.
-Default shadow algorithm, optional item. The default matching algorithm for table that is not configured with the shadow algorithm.
+- Note based shadow algorithm
-**Note**: The default shadow algorithm only supports note shadow algorithm.
+Recognize comment from SQL and route to shadow databases.
+Suitable for identify passed by upstream system scenario.
diff --git a/docs/document/content/features/shadow/use-norms.cn.md b/docs/document/content/features/shadow/use-norms.cn.md
index 9213c86..0349ed4 100644
--- a/docs/document/content/features/shadow/use-norms.cn.md
+++ b/docs/document/content/features/shadow/use-norms.cn.md
@@ -3,50 +3,39 @@ title = "使用规范"
weight = 2
+++
-## 影子数据库
+## 支持项
-### 支持项
+* 基于标记的影子算法支持全部 SQL；
+* 基于列的影子算法仅支持部分 SQL。
-* 后端数据库为 MySQL、Oracle、PostgreSQL、SQLServer；
+## 不支持项
-### 不支持项
+### 基于标记的影子算法
-* NoSQL 数据库；
+* 无
-## 影子算法
+### 基于列的影子算法
-### 支持项
+* 不支持 DDL 语句；
+* 不支持范围、分组和子查询，如：BETWEEN、GROUP BY ... HAVING 等。
-* 注解影子算法支持 MDL，DDL 语句；
-* 列影子算法基本支持常用的 MDL 语句；
+SQL 支持列表：
-### 不支持项
+- INSERT
-* 列影子算法不支持 DDL 语句。
-* 列影子算法不支持范围值匹配操作，比如：子查询、BETWEEN、GROUP BY ... HAVING等；
-* 使用影子库功能+分库分表功能，部分特殊SQL不支持，请参考[SQL使用规范](https://shardingsphere.apache.org/document/current/cn/features/sharding/use-norms/sql/)
+| *SQL* | *是否支持* |
+| -------------------------------------------------------------------------------- | --------- |
+| INSERT INTO table (column,...) VALUES (value,...) | 支持 |
+| INSERT INTO table (column,...) VALUES (value,...),(value,...),... | 支持 |
+| INSERT INTO table (column,...) SELECT column1 from table1 where column1 = value1 | 不支持 |
-## 列影子算法 DML 语句支持度列表
+- SELECT/UPDATE/DELETE
-* INSERT 语句
-
-对 `INSERT` 插入的字段和插入的值判定
-
-| *操作类型* | *SQL语句* | *是否支持* |
-| -------- | --------- | --------- |
-| INSERT | INSERT INTO table (column,...) VALUES (value,...) | 支持 |
-| INSERT | INSERT INTO table (column,...) VALUES (value,...),(value,...),... | 支持 |
-| INSERT | INSERT INTO table (column,...) SELECT column1 from table1 where column1 = value1 | 不支持 |
-
-* SELECT/UPDATE/DELETE语句
-
-对 `WHERE` 条件中包含的字段和值进行判定
-
-| *条件类型* | *SQL语句* | *是否支持* |
-| -------- | --------- | --------- |
-| = | SELECT/UPDATE/DELETE ... WHERE column = value | 支持 |
-| LIKE/NOT LIKE | SELECT/UPDATE/DELETE ... WHERE column LIKE/NOT LIKE value | 支持 |
-| IN/NOT IN | SELECT/UPDATE/DELETE ... WHERE column IN/NOT IN (value1,value2,...) | 支持 |
-| BETWEEN | SELECT/UPDATE/DELETE ... WHERE column BETWEEN value1 AND value2 | 不支持 |
-| GROUP BY ... HAVING... | SELECT/UPDATE/DELETE ... WHERE ... GROUP BY column HAVING column > value; | 不支持 |
-| 子查询 | SELECT/UPDATE/DELETE ... WHERE column = (SELECT column FROM table WHERE column = value) | 不支持 |
+| *条件类型* | *SQL* | *是否支持* |
+| ---------------------- | --------------------------------------------------------------------------------------- | --------- |
+| = | SELECT/UPDATE/DELETE ... WHERE column = value | 支持 |
+| LIKE/NOT LIKE | SELECT/UPDATE/DELETE ... WHERE column LIKE/NOT LIKE value | 支持 |
+| IN/NOT IN | SELECT/UPDATE/DELETE ... WHERE column IN/NOT IN (value1,value2,...) | 支持 |
+| BETWEEN | SELECT/UPDATE/DELETE ... WHERE column BETWEEN value1 AND value2 | 不支持 |
+| GROUP BY ... HAVING... | SELECT/UPDATE/DELETE ... WHERE ... GROUP BY column HAVING column > value | 不支持 |
+| 子查询 | SELECT/UPDATE/DELETE ... WHERE column = (SELECT column FROM table WHERE column = value) | 不支持 |
diff --git a/docs/document/content/features/shadow/use-norms.en.md b/docs/document/content/features/shadow/use-norms.en.md
index c0022aa..ac716bd 100644
--- a/docs/document/content/features/shadow/use-norms.en.md
+++ b/docs/document/content/features/shadow/use-norms.en.md
@@ -3,51 +3,39 @@ title = "Use Norms"
weight = 2
+++
-## Shadow database
+## Supported
-### Supported
+* Note based shadow algorithm support all SQL;
+* Column based shadow algorithm support part of SQL.
-* The database is MySQL, Oracle, PostgreSQL, SQLServer;
+## Unsupported
-### Unsupported
+### Note based shadow algorithm
-* NoSQL database;
+* None
-## Shadow algorithm
+### Column based shadow algorithm
-### Supported
+* Does not support DDL;
+* Does not support range, group and subquery, for example: BETWEEN, GROUP BY ... HAVING...;
-* The note shadow algorithm supports MDL and DDL statements;
-* The column shadow algorithm basically supports commonly used MDL statements;
+SQL support list:
-### Unsupported
+- INSERT
-* Column shadow algorithm does not support DDL statements；
-* The column shadow algorithm does not support range value matching operations, for example: subQuery, BETWEEN, GROUP BY ... HAVING...;
-* Use shadow library function + sub-library sub-table function, some special SQL is not supported,
- please refer to[SQL Usage Specification](https://shardingsphere.apache.org/document/current/en/features/sharding/use-norms/sql/)
+| *SQL* | *Supported* |
+| -------------------------------------------------------------------------------- | ------------ |
+| INSERT INTO table (column,...) VALUES (value,...) | Y |
+| INSERT INTO table (column,...) VALUES (value,...),(value,...),... | Y |
+| INSERT INTO table (column,...) SELECT column1 from table1 where column1 = value1 | N |
-## Column shadow algorithm DML statement support list
+- SELECT/UPDATE/DELETE
-* INSERT statement
-
-Judge the inserted column and inserted value of `INSERT` operation
-
-| *Operation* | *SQL* | *Support* |
-| -------- | --------- | --------- |
-| INSERT | INSERT INTO table (column,...) VALUES (value,...) | true |
-| INSERT | INSERT INTO table (column,...) VALUES (value,...),(value,...),... | true |
-| INSERT | INSERT INTO table (column,...) SELECT column1 from table1 where column1 = value1 | false |
-
-* SELECT/UPDATE/DELETE statement
-
-Judge the column and values included in the `WHERE` condition
-
-| *Condition* | *SQL* | *Support* |
-| -------- | --------- | --------- |
-| = | SELECT/UPDATE/DELETE ... WHERE column = value | true |
-| LIKE/NOT LIKE | SELECT/UPDATE/DELETE ... WHERE column LIKE/NOT LIKE value | true |
-| IN/NOT IN | SELECT/UPDATE/DELETE ... WHERE column IN/NOT IN (value1,value2,...) | true |
-| BETWEEN | SELECT/UPDATE/DELETE ... WHERE column BETWEEN value1 AND value2 | false |
-| GROUP BY ... HAVING... | SELECT/UPDATE/DELETE ... WHERE ... GROUP BY column HAVING column > value; | false |
-| subQuery | SELECT/UPDATE/DELETE ... WHERE column = (SELECT column FROM table WHERE column = value) | false |
+| *Condition* | *SQL* | *Supported* |
+| ---------------------- | --------------------------------------------------------------------------------------- | ----------- |
+| = | SELECT/UPDATE/DELETE ... WHERE column = value | Y |
+| LIKE/NOT LIKE | SELECT/UPDATE/DELETE ... WHERE column LIKE/NOT LIKE value | Y |
+| IN/NOT IN | SELECT/UPDATE/DELETE ... WHERE column IN/NOT IN (value1,value2,...) | Y |
+| BETWEEN | SELECT/UPDATE/DELETE ... WHERE column BETWEEN value1 AND value2 | N |
+| GROUP BY ... HAVING... | SELECT/UPDATE/DELETE ... WHERE ... GROUP BY column HAVING column > value | N |
+| Subquery | SELECT/UPDATE/DELETE ... WHERE column = (SELECT column FROM table WHERE column = value) | N |