You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by me...@apache.org on 2022/01/20 10:37:34 UTC
[shardingsphere] branch master updated: Update scaling build doc and spi doc (#14943)
This is an automated email from the ASF dual-hosted git repository.
menghaoran pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new 923382e Update scaling build doc and spi doc (#14943)
923382e is described below
commit 923382e536a4ee40f01702072f439ae56c3fdbd7
Author: Hongsheng Zhong <sa...@126.com>
AuthorDate: Thu Jan 20 18:36:43 2022 +0800
Update scaling build doc and spi doc (#14943)
* Update scaling data consistency check SPI doc
* Add CREATE SHARDING SCALING RULE in scaling build doc
---
docs/document/content/dev-manual/scaling.cn.md | 19 +++++++++++---
docs/document/content/dev-manual/scaling.en.md | 19 +++++++++++---
.../user-manual/shardingsphere-scaling/build.cn.md | 29 ++++++++++++++++++++--
.../user-manual/shardingsphere-scaling/build.en.md | 29 ++++++++++++++++++++--
4 files changed, 84 insertions(+), 12 deletions(-)
diff --git a/docs/document/content/dev-manual/scaling.cn.md b/docs/document/content/dev-manual/scaling.cn.md
index b2377d5..e2888da 100644
--- a/docs/document/content/dev-manual/scaling.cn.md
+++ b/docs/document/content/dev-manual/scaling.cn.md
@@ -33,7 +33,18 @@ chapter = true
| ------------------------------------------- | ------------------------------------------- |
| DataConsistencyCheckAlgorithm | 数据一致性校验算法 |
-| *已知实现类* | *详细说明* |
-| ------------------------------------------- | ------------------------------------------- |
-| DataMatchDataConsistencyCheckAlgorithm | 基于数据匹配的一致性校验算法 |
-| CRC32MatchDataConsistencyCheckAlgorithm | 基于数据CRC32匹配的一致性校验算法 |
+| *已知实现类* | *详细说明* |
+| ------------------------------------------- | ---------------------------------------------------- |
+| DataMatchDataConsistencyCheckAlgorithm | 基于数据匹配的一致性校验算法。类型名：DATA_MATCH。 |
+| CRC32MatchDataConsistencyCheckAlgorithm | 基于数据CRC32匹配的一致性校验算法。类型名：CRC32_MATCH。 |
+
+## SingleTableDataCalculator
+
+| *SPI 名称* | *详细说明* |
+| ------------------------------------------- | ---------------------------------------------------- |
+| SingleTableDataCalculator | 给数据一致性校验使用的单表数据计算算法 |
+
+| *已知实现类* | *详细说明* |
+| ------------------------------------------- | ---------------------------------------------------------------------- |
+| DataMatchSingleTableDataCalculator | 给 DATA_MATCH 数据一致性校验算法使用的单表数据计算算法。适用于所有数据库。 |
+| CRC32MatchMySQLSingleTableDataCalculator | 给 CRC32_MATCH 数据一致性校验算法使用的单表数据计算算法。适用于MySQL。 |
diff --git a/docs/document/content/dev-manual/scaling.en.md b/docs/document/content/dev-manual/scaling.en.md
index 202fb43..eb05632 100644
--- a/docs/document/content/dev-manual/scaling.en.md
+++ b/docs/document/content/dev-manual/scaling.en.md
@@ -33,7 +33,18 @@ chapter = true
| ------------------------------------------- | ------------------------------------------- |
| DataConsistencyCheckAlgorithm | Data consistency check algorithm on source and target database cluster |
-| *Implementation Class* | *Description* |
-| ------------------------------------------- | ------------------------------------------- |
-| DataMatchDataConsistencyCheckAlgorithm | Records content match implementation |
-| CRC32MatchDataConsistencyCheckAlgorithm | Records CRC32 match implementation |
+| *Implementation Class* | *Description* |
+| ------------------------------------------- | ---------------------------------------------------------------------- |
+| DataMatchDataConsistencyCheckAlgorithm | Records content match implementation. Type name: DATA_MATCH. |
+| CRC32MatchDataConsistencyCheckAlgorithm | Records CRC32 match implementation. Type name: CRC32_MATCH. |
+
+## SingleTableDataCalculator
+
+| *SPI Name* | *Description* |
+| ------------------------------------------- | ---------------------------------------------------------------------- |
+| SingleTableDataCalculator | Single table data calculator for data consistency check |
+
+| *Implementation Class* | *Description* |
+| ------------------------------------------- | ---------------------------------------------------------------------- |
+| DataMatchSingleTableDataCalculator | Single table data calculator for DATA_MATCH data consistency check |
+| CRC32MatchMySQLSingleTableDataCalculator | Single table data calculator for CRC32_MATCH data consistency check |
diff --git a/docs/document/content/user-manual/shardingsphere-scaling/build.cn.md b/docs/document/content/user-manual/shardingsphere-scaling/build.cn.md
index b4e4379..d420975 100644
--- a/docs/document/content/user-manual/shardingsphere-scaling/build.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-scaling/build.cn.md
@@ -42,7 +42,9 @@ mode:
overwrite: false
```
-4. 修改配置文件 `conf/config-sharding.yaml` 的 `scalingName` 和 `scaling` 部分。
+4. 开启 scaling
+
+方法1：修改配置文件 `conf/config-sharding.yaml` 的 `scalingName` 和 `scaling` 部分。
配置项说明：
```yaml
@@ -118,7 +120,30 @@ rules:
chunk-size: 1000
```
-以上的 `rateLimiter`，`completionDetector`，`sourceWritingStopper`，`dataConsistencyChecker` 和 `checkoutLocker` 都可以通过实现SPI自定义。可以参考现有实现，详情请参见[开发者手册#弹性伸缩](/cn/dev-manual/scaling/)。
+以上的 `completionDetector`，`dataConsistencyChecker` 都可以通过实现 SPI 自定义。可以参考现有实现，详情请参见[开发者手册#弹性伸缩](/cn/dev-manual/scaling/)。
+
+方法2：通过 DistSQL 配置 scaling
+
+创建 scaling 配置示例：
+```sql
+CREATE SHARDING SCALING RULE default_scaling (
+INPUT(
+ WORKER_THREAD=40,
+ BATCH_SIZE=1000,
+ RATE_LIMITER(TYPE(NAME=QPS, PROPERTIES("qps"=50)))
+),
+OUTPUT(
+ WORKER_THREAD=40,
+ BATCH_SIZE=1000,
+ RATE_LIMITER(TYPE(NAME=TPS, PROPERTIES("tps"=2000)))
+),
+STREAM_CHANNEL(TYPE(NAME=MEMORY, PROPERTIES("block-queue-size"=10000))),
+COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=3))),
+DATA_CONSISTENCY_CHECKER(TYPE(NAME=DATA_MATCH, PROPERTIES("chunk-size"=1000)))
+);
+```
+
+详情请参见[RDL#数据分片](/cn/user-manual/shardingsphere-proxy/distsql/syntax/rdl/rule-definition/sharding/)。
5. 启动 ShardingSphere-Proxy：
diff --git a/docs/document/content/user-manual/shardingsphere-scaling/build.en.md b/docs/document/content/user-manual/shardingsphere-scaling/build.en.md
index e9f7af9..fc8f343 100644
--- a/docs/document/content/user-manual/shardingsphere-scaling/build.en.md
+++ b/docs/document/content/user-manual/shardingsphere-scaling/build.en.md
@@ -41,7 +41,9 @@ mode:
overwrite: false
```
-4. Modify `scalingName` and `scaling` configuration in `conf/config-sharding.yaml`.
+4. Enable scaling
+
+Way 1. Modify `scalingName` and `scaling` configuration in `conf/config-sharding.yaml`.
Configuration Items Explanation:
```yaml
@@ -117,7 +119,30 @@ rules:
chunk-size: 1000
```
-You could customize `rateLimiter`, `completionDetector`, `sourceWritingStopper`, `dataConsistencyChecker` and `checkoutLocker` algorithm by implementing SPI. Current implementation could be referenced, please refer to [Dev Manual#Scaling](/en/dev-manual/scaling/) for more details.
+You could customize `completionDetector`, `dataConsistencyChecker` algorithm by implementing SPI. Current implementation could be referenced, please refer to [Dev Manual#Scaling](/en/dev-manual/scaling/) for more details.
+
+Way 2: Configure scaling by DistSQL
+
+Create scaling configuration example:
+```sql
+CREATE SHARDING SCALING RULE default_scaling (
+INPUT(
+ WORKER_THREAD=40,
+ BATCH_SIZE=1000,
+ RATE_LIMITER(TYPE(NAME=QPS, PROPERTIES("qps"=50)))
+),
+OUTPUT(
+ WORKER_THREAD=40,
+ BATCH_SIZE=1000,
+ RATE_LIMITER(TYPE(NAME=TPS, PROPERTIES("tps"=2000)))
+),
+STREAM_CHANNEL(TYPE(NAME=MEMORY, PROPERTIES("block-queue-size"=10000))),
+COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=3))),
+DATA_CONSISTENCY_CHECKER(TYPE(NAME=DATA_MATCH, PROPERTIES("chunk-size"=1000)))
+);
+```
+
+Please refer to [RDL#Sharding](/en/user-manual/shardingsphere-proxy/distsql/syntax/rdl/rule-definition/sharding/) for more details.
5. Start up ShardingSphere-Proxy: