You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2023/06/27 03:18:12 UTC
[GitHub] [doris] Yukang-Lian opened a new pull request, #21223: [Enhancement](regression-test)Add regression test for backup and restore
Yukang-Lian opened a new pull request, #21223:
URL: https://github.com/apache/doris/pull/21223
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21223:
URL: https://github.com/apache/doris/pull/21223#issuecomment-1621131522
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #21223:
URL: https://github.com/apache/doris/pull/21223#issuecomment-1616346715
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #21223:
URL: https://github.com/apache/doris/pull/21223
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #21223:
URL: https://github.com/apache/doris/pull/21223#issuecomment-1620952595
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zhannngchen commented on a diff in pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #21223:
URL: https://github.com/apache/doris/pull/21223#discussion_r1250779792
##########
regression-test/suites/backup_restore/test_MoW_backup_restore.groovy:
##########
@@ -0,0 +1,136 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.regression.suite
+
+suite("test_MoW_backup_restore", "p2") {
+
+ def syncer = getSyncer()
+ def repo = "__keep_on_local__"
+ def tableName = "demo_MoW"
+ sql """drop table if exists ${tableName}"""
+ sql """CREATE TABLE IF NOT EXISTS ${tableName}
+ ( `user_id` INT NOT NULL, `value` INT NOT NULL)
+ UNIQUE KEY(`user_id`)
+ DISTRIBUTED BY HASH(`user_id`)
+ BUCKETS 1
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1",
+ "disable_auto_compaction" = "true",
+ "enable_unique_key_merge_on_write" = "true");"""
+
+ // version1 (1,1)(2,2)
+ sql """insert into ${tableName} values(1,1),(2,2)"""
+ sql """backup snapshot ${context.dbName}.snapshot1 to ${repo} on (${tableName}) properties("type"="full")"""
+ while(checkSnapshotFinish()==false){
+ Thread.sleep(3000)
+ }
+ qt_3 """select * from ${tableName}"""
Review Comment:
All select should add `order by`, to make your query result stable
##########
regression-test/suites/backup_restore/test_MoW_backup_restore.groovy:
##########
@@ -0,0 +1,136 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.regression.suite
+
+suite("test_MoW_backup_restore", "p2") {
+
+ def syncer = getSyncer()
+ def repo = "__keep_on_local__"
+ def tableName = "demo_MoW"
+ sql """drop table if exists ${tableName}"""
+ sql """CREATE TABLE IF NOT EXISTS ${tableName}
+ ( `user_id` INT NOT NULL, `value` INT NOT NULL)
+ UNIQUE KEY(`user_id`)
+ DISTRIBUTED BY HASH(`user_id`)
+ BUCKETS 1
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1",
+ "disable_auto_compaction" = "true",
+ "enable_unique_key_merge_on_write" = "true");"""
+
+ // version1 (1,1)(2,2)
+ sql """insert into ${tableName} values(1,1),(2,2)"""
+ sql """backup snapshot ${context.dbName}.snapshot1 to ${repo} on (${tableName}) properties("type"="full")"""
+ while(checkSnapshotFinish()==false){
+ Thread.sleep(3000)
+ }
+ qt_3 """select * from ${tableName}"""
+
+ // version2 (1,10)(2,2)
+ sql """insert into ${tableName} values(1,10)"""
+ sql """backup snapshot ${context.dbName}.snapshot2 to ${repo} on (${tableName}) properties("type"="full")"""
+ while(checkSnapshotFinish()==false){
+ Thread.sleep(3000)
+ }
+ qt_4 """select * from ${tableName}"""
+
+ // version3 (1,100)(2,2)
+ sql """update ${tableName} set value = 100 where user_id = 1"""
+ sql """backup snapshot ${context.dbName}.snapshot3 to ${repo} on (${tableName}) properties("type"="full")"""
+ while(checkSnapshotFinish()==false){
+ Thread.sleep(3000)
+ }
+ qt_5 """select * from ${tableName}"""
+
+ // version4 (2,2)
+ sql """delete from ${tableName} where user_id = 1"""
+ sql """backup snapshot ${context.dbName}.snapshot4 to ${repo} on (${tableName}) properties("type"="full")"""
+ while(checkSnapshotFinish()==false){
+ Thread.sleep(3000)
+ }
+ qt_6 """select * from ${tableName}"""
+
+ // version1 (1,1)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot1", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_7 """select * from ${tableName}"""
+
+ // version2 (1,10)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot2", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_8 """select * from ${tableName}"""
+ // version3 (1,100)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot3", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_9 """select * from ${tableName}"""
+ // version4 (2,2)
+ assertTrue(syncer.getSnapshot("snapshot4", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_10 """select * from ${tableName}"""
+
+ sql """drop table if exists ${tableName}"""
+ sql """CREATE TABLE IF NOT EXISTS ${tableName}
+ ( `user_id` INT NOT NULL, `value` INT NOT NULL)
+ UNIQUE KEY(`user_id`)
+ DISTRIBUTED BY HASH(`user_id`)
+ BUCKETS 1
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1",
+ "disable_auto_compaction" = "true",
+ "enable_unique_key_merge_on_write" = "true");"""
+
+ // version1 (1,1)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot1", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_11 """select * from ${tableName}"""
+
+ // version2 (1,10)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot2", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_12 """select * from ${tableName}"""
+ // version3 (1,100)(2,2)
+ assertTrue(syncer.getSnapshot("snapshot3", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_13 """select * from ${tableName}"""
+ // version4 (2,2)
+ assertTrue(syncer.getSnapshot("snapshot4", "${tableName}"))
+ assertTrue(syncer.restoreSnapshot())
+ while (checkRestoreFinish() == false) {
+ Thread.sleep(3000)
+ }
+ qt_14 """select * from ${tableName}"""
+}
Review Comment:
it's required an extra line at the end of file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21223: [Enhancement](regression-test)Add regression test for MoW backup and restore
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21223:
URL: https://github.com/apache/doris/pull/21223#issuecomment-1621131577
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org