You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by yi...@apache.org on 2023/04/14 13:15:15 UTC
[doris] branch master updated: [test][typo](alter) enhance the suspicious rollup case for alter LSC and add experimental docs (#18612)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 30a783908e [test][typo](alter) enhance the suspicious rollup case for alter LSC and add experimental docs (#18612)
30a783908e is described below
commit 30a783908e23f60d774d28af716e949f2b933902
Author: 奕冷 <82...@users.noreply.github.com>
AuthorDate: Fri Apr 14 21:15:04 2023 +0800
[test][typo](alter) enhance the suspicious rollup case for alter LSC and add experimental docs (#18612)
* improve rollup case
* add docs and mark it as experimental
* adjust the unique key in uniq-rollup-case to add flexibility
---
.../Alter/ALTER-TABLE-PROPERTY.md | 8 +++++
.../Alter/ALTER-TABLE-PROPERTY.md | 8 +++++
.../test_agg_rollup_schema_change.groovy | 35 ++++++++++--------
.../test_dup_rollup_schema_change.groovy | 42 +++++++++++++---------
.../test_uniq_rollup_schema_change.groovy | 36 ++++++++++---------
5 files changed, 81 insertions(+), 48 deletions(-)
diff --git a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
index b520c823c8..9888cc03fd 100644
--- a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
+++ b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
@@ -169,6 +169,14 @@ Note:
changed.
3. For partitioned tables, the actual replica distribution of the table is at the partition level, that is, each partition has its own replica distribution, which can be viewed through the `show partitions from tbl` statement. If you want to modify the actual replica distribution, see `ALTER TABLE PARTITION`.
+13\. **[Experimental]** turn on `light_schema_change`
+
+ For tables that were not created with light_schema_change enabled, you can enable it by using the following statement.
+
+```sql
+ALTER TABLE example_db.mysql_table SET ("light_schema_change" = "true");
+```
+
### Example
1. Modify the bloom filter column of the table
diff --git a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
index 958c5c97a7..de67e5e5cb 100644
--- a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
+++ b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-PROPERTY.md
@@ -162,6 +162,14 @@ ALTER TABLE example_db.mysql_table SET ("default.replication_allocation" = "tag.
2. 对于非分区表，修改不带 default 前缀的副本分布属性，会同时修改表的默认副本分布和实际副本分布。即修改后，通过 `show create table` 和 `show partitions from tbl` 语句可以看到副本分布数据都被修改了。
3. 对于分区表，表的实际副本分布是分区级别的，即每个分区有自己的副本分布，可以通过 `show partitions from tbl` 语句查看。如果想修改实际副本分布，请参阅 `ALTER TABLE PARTITION`。
+13\. **[Experimental]** 打开`light_schema_change`
+
+ 对于建表时未开启light_schema_change的表，可以通过如下方式打开。
+
+```sql
+ALTER TABLE example_db.mysql_table SET ("light_schema_change" = "true");
+```
+
### Example
1. 修改表的 bloom filter 列
diff --git a/regression-test/suites/schema_change_p0/test_agg_rollup_schema_change.groovy b/regression-test/suites/schema_change_p0/test_agg_rollup_schema_change.groovy
index 7f4e4ab034..286dd112b4 100644
--- a/regression-test/suites/schema_change_p0/test_agg_rollup_schema_change.groovy
+++ b/regression-test/suites/schema_change_p0/test_agg_rollup_schema_change.groovy
@@ -28,6 +28,20 @@ suite ("test_agg_rollup_schema_change") {
def jobStateResult = sql """ SHOW ALTER TABLE COLUMN WHERE IndexName='${tbName}' ORDER BY createtime DESC LIMIT 1 """
return jobStateResult[0][9]
}
+ def waitForMVJob = (tbName, timeout) -> {
+ while (timeout--){
+ String result = getMVJobState(tbName)
+ if (result == "FINISHED") {
+ sleep(3000)
+ break
+ } else {
+ sleep(100)
+ if (timeout < 1){
+ assertEquals(1,2)
+ }
+ }
+ }
+ }
try {
@@ -87,20 +101,8 @@ suite ("test_agg_rollup_schema_change") {
//add rollup
def rollupName = "rollup_cost"
- sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`city`,`age`, cost);"
- int max_try_time = 3000
- while (max_try_time--){
- String result = getMVJobState(tableName)
- if (result == "FINISHED") {
- sleep(3000)
- break
- } else {
- sleep(100)
- if (max_try_time < 1){
- assertEquals(1,2)
- }
- }
- }
+ sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`age`, `cost`);"
+ waitForMVJob(tableName, 3000)
sql """ INSERT INTO ${tableName} VALUES
(1, '2017-10-01', 'Beijing', 10, 1, 1, 30, 20, hll_hash(1), to_bitmap(1))
@@ -113,6 +115,11 @@ suite ("test_agg_rollup_schema_change") {
// alter and test light schema change
sql """ALTER TABLE ${tableName} SET ("light_schema_change" = "true");"""
+ //add rollup
+ def rollupName2 = "rollup_city"
+ sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName2}(`user_id`,`city`);"
+ waitForMVJob(tableName, 3000)
+
sql """ INSERT INTO ${tableName} VALUES
(2, '2017-10-01', 'Beijing', 10, 1, 1, 31, 21, hll_hash(2), to_bitmap(2))
"""
diff --git a/regression-test/suites/schema_change_p0/test_dup_rollup_schema_change.groovy b/regression-test/suites/schema_change_p0/test_dup_rollup_schema_change.groovy
index 2445045643..6e546eeb63 100644
--- a/regression-test/suites/schema_change_p0/test_dup_rollup_schema_change.groovy
+++ b/regression-test/suites/schema_change_p0/test_dup_rollup_schema_change.groovy
@@ -26,6 +26,21 @@ suite ("test_dup_rollup_schema_change") {
def jobStateResult = sql """ SHOW ALTER TABLE COLUMN WHERE IndexName='${tableName}' ORDER BY createtime DESC LIMIT 1 """
return jobStateResult[0][9]
}
+ def waitForMVJob = (tbName, timeout) -> {
+ while (timeout--){
+ String result = getMVJobState(tbName)
+ if (result == "FINISHED") {
+ sleep(3000)
+ break
+ } else {
+ sleep(100)
+ if (timeout < 1){
+ assertEquals(1,2)
+ }
+ }
+ }
+ }
+
def tableName = "schema_change_dup_rollup_regression_test"
try {
@@ -83,30 +98,23 @@ suite ("test_dup_rollup_schema_change") {
PROPERTIES ( "replication_num" = "1", "light_schema_change" = "false" );
"""
- // alter and test light schema change
- sql """ALTER TABLE ${tableName} SET ("light_schema_change" = "true");"""
-
//add rollup
def rollupName = "rollup_cost"
- sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`city`,`age`, cost);"
- int max_try_time = 3000
- while (max_try_time--){
- String result = getMVJobState(tableName)
- if (result == "FINISHED") {
- sleep(3000)
- break
- } else {
- sleep(100)
- if (max_try_time < 1){
- assertEquals(1,2)
- }
- }
- }
+ sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`age`, `cost`);"
+ waitForMVJob(tableName, 3000)
sql """ INSERT INTO ${tableName} VALUES
(1, '2017-10-01', 'Beijing', 10, 1, '2020-01-01', '2020-01-01', '2020-01-01', 1, 30, 20)
"""
+ // alter and test light schema change
+ sql """ALTER TABLE ${tableName} SET ("light_schema_change" = "true");"""
+
+ //add rollup
+ def rollupName2 = "rollup_city"
+ sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName2}(`user_id`,`city`);"
+ waitForMVJob(tableName, 3000)
+
sql """ INSERT INTO ${tableName} VALUES
(1, '2017-10-01', 'Beijing', 10, 1, '2020-01-02', '2020-01-02', '2020-01-02', 1, 31, 19)
"""
diff --git a/regression-test/suites/schema_change_p0/test_uniq_rollup_schema_change.groovy b/regression-test/suites/schema_change_p0/test_uniq_rollup_schema_change.groovy
index f955f3b611..3a47bc9d63 100644
--- a/regression-test/suites/schema_change_p0/test_uniq_rollup_schema_change.groovy
+++ b/regression-test/suites/schema_change_p0/test_uniq_rollup_schema_change.groovy
@@ -27,6 +27,20 @@ suite ("test_uniq_rollup_schema_change") {
def jobStateResult = sql """ SHOW ALTER TABLE COLUMN WHERE IndexName='${tbName}' ORDER BY createtime DESC LIMIT 1 """
return jobStateResult[0][9]
}
+ def waitForMVJob = (tbName, timeout) -> {
+ while (timeout--){
+ String result = getMVJobState(tbName)
+ if (result == "FINISHED") {
+ sleep(3000)
+ break
+ } else {
+ sleep(100)
+ if (timeout < 1){
+ assertEquals(1,2)
+ }
+ }
+ }
+ }
try {
String[][] backends = sql """ show backends; """
@@ -91,30 +105,18 @@ suite ("test_uniq_rollup_schema_change") {
(1, '2017-10-01', 'Beijing', 10, 1, '2020-01-02', '2020-01-02', '2020-01-02', 1, 31, 19)
"""
- // alter and test light schema change
- sql """ALTER TABLE ${tableName} SET ("light_schema_change" = "true");"""
-
//add rollup
def rollupName = "rollup_cost"
- sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`city`,`sex`, `age`, cost);"
- int max_try_time = 3000
- while (max_try_time--){
- String result = getMVJobState(tableName)
- if (result == "FINISHED") {
- sleep(3000)
- break
- } else {
- sleep(100)
- if (max_try_time < 1){
- assertEquals(1,2)
- }
- }
- }
+ sql "ALTER TABLE ${tableName} ADD ROLLUP ${rollupName}(`user_id`,`date`,`city`,`sex`, `age`,`cost`);"
+ waitForMVJob(tableName, 3000)
sql """ INSERT INTO ${tableName} VALUES
(2, '2017-10-01', 'Beijing', 10, 1, '2020-01-02', '2020-01-02', '2020-01-02', 1, 31, 21)
"""
+ // alter and test light schema change
+ sql """ALTER TABLE ${tableName} SET ("light_schema_change" = "true");"""
+
sql """ INSERT INTO ${tableName} VALUES
(2, '2017-10-01', 'Beijing', 10, 1, '2020-01-03', '2020-01-03', '2020-01-03', 1, 32, 20)
"""
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org