You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by ch...@apache.org on 2025/05/21 01:50:39 UTC
(shardingsphere) branch master updated: Update doc for e2e sql (#35477)
This is an automated email from the ASF dual-hosted git repository.
chengzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new fd6514221fb Update doc for e2e sql (#35477)
fd6514221fb is described below
commit fd6514221fba6c96a9f845186913ccc622fd551c
Author: Zhengqiang Duan <du...@apache.org>
AuthorDate: Wed May 21 09:50:34 2025 +0800
Update doc for e2e sql (#35477)
---
.../test-manual/integration-test/_index.cn.md | 81 ++++++++++++++-------
.../test-manual/integration-test/_index.en.md | 83 +++++++++++++++-------
2 files changed, 114 insertions(+), 50 deletions(-)
diff --git a/docs/document/content/test-manual/integration-test/_index.cn.md b/docs/document/content/test-manual/integration-test/_index.cn.md
index ed7470e8260..c4b704ef177 100644
--- a/docs/document/content/test-manual/integration-test/_index.cn.md
+++ b/docs/document/content/test-manual/integration-test/_index.cn.md
@@ -25,11 +25,10 @@ weight = 1
- Embed 环境由测试框架自动搭建嵌入式 MySQL，适用于 ShardingSphere-JDBC 的本地环境测试。
当前默认采用 Docker 环境，使用 Testcontainer 创建运行时环境并执行测试用例。
-未来将采用 Embed 环境的 ShardingSphere-JDBC + MySQL，替换 Native 执行测试用例的默认环境类型。
数据库类型目前支持 MySQL、PostgreSQL、SQLServer 和 Oracle，并且可以支持使用 ShardingSphere-JDBC 或是使用 ShardingSphere-Proxy 执行测试用例。
-场景用于对 ShardingSphere 支持规则进行测试，目前支持数据分片和读写分离的相关场景，未来会不断完善场景的组合。
+场景用于对 ShardingSphere 支持规则进行测试，目前支持数据分片、数据加密、数据脱敏和读写分离的相关场景，未来会不断完善场景的组合。
### 测试引擎
@@ -53,20 +52,28 @@ weight = 1
### 测试用例配置
-SQL 用例在 `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xml`。
+SQL 用例在 `resources/cases/${SQL-TYPE}/e2e-${SQL-TYPE}-${cases-description}.xml`。
用例文件格式如下：
```xml
-<integration-test-cases>
+<e2e-test-cases>
+ <test-case sql="${SQL}">
+ <!-- 查询场景 -->
+ <assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-source-name="{datasource-name}" />
+ <!-- 非查询场景 -->
+ <assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-file="${dataset_file_1}.xml" />
+ <!-- ... more assertions -->
+ </test-case>
+
<test-case sql="${SQL}">
<assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-file="${dataset_file_1}.xml" />
<!-- ... more assertions -->
<assertion parameters="${value_3}:${type_3}, ${value_4}:${type_4}" expected-data-file="${dataset_file_2}.xml" />
- </test-case>
+ </test-case>
<!-- ... more test cases -->
-</integration-test-cases>
+</e2e-test-cases>
```
`expected-data-file` 的查找规则是：
@@ -90,8 +97,6 @@ SQL 用例在 `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xm
</dataset>
```
-> e2e operation 为 E2E 测试，并不包含 <dataset> 这类断言
-
### 环境配置
`${SCENARIO-TYPE}` 表示场景名称，在测试引擎运行中用于标识唯一场景。
@@ -99,20 +104,37 @@ SQL 用例在 `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xm
#### Native 环境配置
-目录：`src/test/resources/env/scenario/${SCENARIO-TYPE}`
+修改 `e2e-sql` 模块 `src/test/resources/env/it-env.properties` 文件中 `it.cluster.env.type` 为 `NATIVE` 模式，然后修改如下的属性为本地数据库地址和账号。
- - `scenario-env.properties`: 数据源配置；
- - `rules.yaml`: 规则配置；
- - `databases.xml`: 真实库名称；
- - `dataset.xml`: 初始化数据；
- - `init-sql\${DATABASE-TYPE}\init.sql`: 初始化数据库表结构；
- - `authority.xml`: 待补充。
+```properties
+it.native.storage.host=127.0.0.1
+it.native.storage.port=3306
+it.native.storage.username=root
+it.native.storage.password=123456
+```
+
+修改完成后，可以再调整 `it-env.properties` 中其他属性，测试 ShardingSphere 的 Proxy、JDBC 接入端，或者测试单机、集群模式。
#### Docker 环境配置
-目录：`src/test/resources/env/${SCENARIO-TYPE}`
+修改 `e2e-sql` 模块 `src/test/resources/env/it-env.properties` 文件中 `it.cluster.env.type` 为 `DOCKER` 模式，如果执行 Proxy 接入端测试，需要执行如下的命令打包 Proxy 镜像。
+
+```bash
+./mvnw -B clean install -am -pl test/e2e/sql -Pit.env.docker -DskipTests -Dspotless.apply.skip=true -Drat.skip=true
+```
- - `proxy/conf/database-${SCENARIO-TYPE}.yaml`: 规则配置。
+如果是 Mac 平台 M 系列芯片，在打包 Proxy 镜像前，需要先执行如下的命令，然后再进行 Proxy 镜像打包。
+
+```bash
+# 安装 socat
+brew install socat
+socat TCP-LISTEN:2375,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock
+
+# 在创建镜像的窗口中执行
+export DOCKER_HOST=tcp://127.0.0.1:2375
+```
+
+修改完成后，可以再调整 `it-env.properties` 中其他属性，测试 ShardingSphere 的 Proxy、JDBC 接入端，或者测试单机、集群模式。
**Docker 环境配置为 ShardingSphere-Proxy 提供了远程调试端口，可以在 `test/e2e/fixture/src/test/assembly/bin/start.sh` 文件的 `JAVA_OPTS` 中找到第 2 个暴露的端口用于远程调试。**
@@ -120,7 +142,7 @@ SQL 用例在 `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xm
#### 配置测试引擎运行环境
-通过配置 `src/test/resources/env/engine-env.properties` 控制测试引擎。
+通过配置 `src/test/resources/env/it-env.properties` 控制测试引擎。
所有的属性值都可以通过 Maven 命令行 `-D` 的方式动态注入。
@@ -133,26 +155,37 @@ it.scenarios=db,tbl,dbtbl_with_replica_query,replica_query
# 是否运行附加测试用例
it.run.additional.cases=false
+# 是否运行冒烟测试
+it.run.smoke=false
-# 配置环境类型，只支持单值。可选值：docker或空，默认值：空
+# 配置环境类型，只支持单值。可选值：DOCKER, NATIVE
it.cluster.env.type=${it.env}
# 待测试的接入端类型，多个值可用逗号分隔。可选值：jdbc, proxy, 默认值：jdbc
it.cluster.adapters=jdbc
-# 场景类型，多个值可用逗号分隔。可选值：H2, MySQL, Oracle, SQLServer, PostgreSQL
-it.cluster.databases=H2,MySQL,Oracle,SQLServer,PostgreSQL
+# 场景类型，多个值可用逗号分隔。可选值：H2, MySQL, PostgreSQL，openGauss
+it.cluster.databases=H2,MySQL,PostgreSQL,openGauss
+
+# 测试数据库的镜像版本
+it.cluster.database.mysql.image=mysql:8.2.0
+
+# NATIVE 模式下数据库连接信息及账号
+it.native.storage.host=127.0.0.1
+it.native.storage.port=3306
+it.native.storage.username=root
+it.native.storage.password=123456
```
#### 运行调试模式
- 标准测试引擎
- 运行 `org.apache.shardingsphere.test.integration.engine.${SQL-TYPE}.General${SQL-TYPE}E2EIT` 以启动不同 SQL 类型的测试引擎。
+ 运行 `org.apache.shardingsphere.test.e2e.it.sql.${SQL-TYPE}.General${SQL-TYPE}E2EIT` 以启动不同 SQL 类型的测试引擎。
- 批量测试引擎
- 运行 `org.apache.shardingsphere.test.integration.engine.dml.BatchDMLE2EIT`，以启动为 DML 语句提供的测试 `addBatch()` 的批量测试引擎。
+ 运行 `org.apache.shardingsphere.test.e2e.it.sql.dml.BatchDMLE2EIT`，以启动为 DML 语句提供的测试 `addBatch()` 的批量测试引擎。
- 附加测试引擎
- 运行 `org.apache.shardingsphere.test.integration.engine.${SQL-TYPE}.Additional${SQL-TYPE}E2EIT` 以启动使用更多 JDBC 方法调用的测试引擎。
+ 运行 `org.apache.shardingsphere.test.e2e.it.sql.${SQL-TYPE}.Additional${SQL-TYPE}E2EIT` 以启动使用更多 JDBC 方法调用的测试引擎。
附加测试引擎需要通过设置 `it.run.additional.cases=true` 开启。
#### 运行 Docker 模式
diff --git a/docs/document/content/test-manual/integration-test/_index.en.md b/docs/document/content/test-manual/integration-test/_index.en.md
index d5353d36959..de47ca024ff 100644
--- a/docs/document/content/test-manual/integration-test/_index.en.md
+++ b/docs/document/content/test-manual/integration-test/_index.en.md
@@ -26,11 +26,10 @@ Environment preparation mode is divided into Native and Docker, and Embed type w
Currently, the Native environment is adopted by default, and ShardingSphere-JDBC + H2 database is used to run test cases.
Maven's `-pit. Env.docker` parameter specifies how the Docker environment is run.
-In the future, ShardingSphere-JDBC + MySQL of the Embed environment will be adopted to replace the default environment type used when Native executes test cases.
Database types currently support MySQL, PostgreSQL, SQLServer, and Oracle, and test cases can be executed using ShardingSphere-JDBC or ShardingSphere-Proxy.
-Scenarios are used to test the supporting rules of ShardingSphere. Currently, data sharding and read/write splitting and other related scenarios are supported, and the combination of scenarios will be improved continuously in the future.
+Scenarios are used to test the supporting rules of ShardingSphere. Currently, data sharding, data encrypt, data mask and read/write splitting and other related scenarios are supported, and the combination of scenarios will be improved continuously in the future.
### Test engine
@@ -55,20 +54,28 @@ Module path：`test/e2e/sql`
### Test case configuration
-SQL test case is in `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xml`.
+SQL test case is in `resources/cases/${SQL-TYPE}/e2e-${SQL-TYPE}-${cases-description}.xml`.
The case file format is as follows:
```xml
-<integration-test-cases>
+<e2e-test-cases>
+ <test-case sql="${SQL}">
+ <!-- select case -->
+ <assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-source-name="{datasource-name}" />
+ <!-- not select case -->
+ <assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-file="${dataset_file_1}.xml" />
+ <!-- ... more assertions -->
+ </test-case>
+
<test-case sql="${SQL}">
<assertion parameters="${value_1}:${type_1}, ${value_2}:${type_2}" expected-data-file="${dataset_file_1}.xml" />
<!-- ... more assertions -->
<assertion parameters="${value_3}:${type_3}, ${value_4}:${type_4}" expected-data-file="${dataset_file_2}.xml" />
- </test-case>
+ </test-case>
<!-- ... more test cases -->
-</integration-test-cases>
+</e2e-test-cases>
```
The lookup rule of `expected-data-file`is as follows:
@@ -92,8 +99,6 @@ The assertion file format is as follows:
</dataset>
```
-> e2e operation module is E2E test，does not contains the assertion for `</dataset>` tag
-
### Environment configuration
`${SCENARIO-TYPE}` Refers to the scenario name used to identify a unique scenario during the test engine run.
@@ -101,29 +106,43 @@ The assertion file format is as follows:
#### Native environment configuration
-Directory: `src/test/resources/env/${SCENARIO-TYPE}`
+Modify `it.cluster.env.type` in `src/test/resources/env/it-env.properties` file of `e2e-sql` module to `NATIVE` mode, and then modify the following properties to the local database address and account.
- - `scenario-env.properties`: data source configuration；
- - `rules.yaml`: rule configuration；
- - `databases.xml`: name of the real database；
- - `dataset.xml`: initialize the data；
- - `init-sql\${DATABASE-TYPE}\init.sql`: initialize the database and table structure；
- - `authority.xml`: to be supplemented.
+```properties
+it.native.storage.host=127.0.0.1
+it.native.storage.port=3306
+it.native.storage.username=root
+it.native.storage.password=123456
+```
+
+After the modification is completed, you can adjust other properties in `it-env.properties` to test ShardingSphere's Proxy, JDBC access terminal, or test the stand-alone and cluster modes.
#### Docker environment configuration
-Directory: `src/test/resources/docker/${SCENARIO-TYPE}`
+Modify `it.cluster.env.type` in the `src/test/resources/env/it-env.properties` file of the `e2e-sql` module to `DOCKER` mode. If you perform a Proxy access end test, you need to execute the following command to package the Proxy image.
+
+```bash
+./mvnw -B clean install -am -pl test/e2e/sql -Pit.env.docker -DskipTests -Dspotless.apply.skip=true -Drat.skip=true
+```
- - `docker-compose.yml`: Docker-Compose config files, used for Docker environment startup；
- - `proxy/conf/database-${SCENARIO-TYPE}.yaml`: rule configuration。
+If it is a Mac platform M series chip, before packaging the Proxy image, you need to execute the following command first, and then package the Proxy image.
-**The Docker environment configuration provides a remote debugging port for ShardingSphere-Proxy. You can find the second exposed port for remote debugging in `shardingsphere-proxy` of the `docker-comemage. yml` file. **
+```bash
+# Install socat
+brew install socat
+socat TCP-LISTEN:2375,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock
+
+# Execute in the terminal where the image is created
+export DOCKER_HOST=tcp://127.0.0.1:2375
+```
+
+After the modification is completed, you can adjust other properties in `it-env.properties` to test ShardingSphere's Proxy, JDBC access terminal, or test the stand-alone and cluster modes.
### Run the test engine
#### Configure the running environment of the test engine
-Control the test engine by configuring `src/test/resources/env/engine-env.properties`.
+Control the test engine by configuring `src/test/resources/env/it-env.properties`.
All attribute values can be dynamically injected via Maven command line `-D`.
@@ -135,25 +154,37 @@ it.scenarios=db,tbl,dbtbl_with_replica_query,replica_query
# Whether to run additional test cases
it.run.additional.cases=false
-# Configure the environment type. Only one value is supported. Optional value: docker or null. The default value: null.
+# Whether to run smoke test
+it.run.smoke=false
+
+# Configure the environment type. Only one value is supported. Optional value: DOCKER, NATIVE
it.cluster.env.type=${it.env}
# Access port types to be tested. Multiple values can be separated by commas. Optional value: jdbc, proxy. The default value: jdbc
it.cluster.adapters=jdbc
-# Scenario type. Multiple values can be separated by commas. Optional value: H2, MySQL, Oracle, SQLServer, PostgreSQL
-it.cluster.databases=H2,MySQL,Oracle,SQLServer,PostgreSQL
+# Scenario type. Multiple values can be separated by commas. Optional value: H2, MySQL, PostgreSQL, openGauss
+it.cluster.databases=H2,MySQL,PostgreSQL,openGauss
+
+# The mirror version of the database
+it.cluster.database.mysql.image=mysql:8.2.0
+
+# Database connection information and account in NATIVE mode
+it.native.storage.host=127.0.0.1
+it.native.storage.port=3306
+it.native.storage.username=root
+it.native.storage.password=123456
```
#### Run debugging mode
- Standard test engine
- Run `org.apache.shardingsphere.test.integration.engine.${SQL-TYPE}.General${SQL-TYPE}E2EIT` to start the test engines of different SQL types.
+ Run `org.apache.shardingsphere.test.e2e.it.sql.${SQL-TYPE}.General${SQL-TYPE}E2EIT` to start the test engines of different SQL types.
- Batch test engine
- Run `org.apache.shardingsphere.test.integration.engine.dml.BatchDMLE2EIT` to start the batch test engine for the test `addBatch()` provided for DML statements.
+ Run `org.apache.shardingsphere.test.e2e.it.sql.dml.BatchDMLE2EIT` to start the batch test engine for the test `addBatch()` provided for DML statements.
- Additional test engine
- Run `org.apache.shardingsphere.test.integration.engine.${SQL-TYPE}.Additional${SQL-TYPE}E2EIT` to start the test engine with more JDBC method calls.
+ Run `org.apache.shardingsphere.test.e2e.it.sql.${SQL-TYPE}.Additional${SQL-TYPE}E2EIT` to start the test engine with more JDBC method calls.
Additional test engines need to be enabled by setting `it.run.additional.cases=true`.
#### Run Docker mode