You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/11/12 08:44:47 UTC
[GitHub] [iotdb] irvine0109 opened a new pull request #4371: [IOTDB-1559] Refactor the IT framework
irvine0109 opened a new pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371
Currently, the following features can be achieved:
- The test cases can be reused. The same set of test cases can be shared with multiple test environments after being marked.
- Support a variety of test environments. Now the framework can support local standalone test environments, cluster test environments, and remote test environments. As the test environment and test cases are decoupled, new test environments can be easily implemented.
- Easy to run and expand. You can configure appropriate parameters through Maven commands or IDE to run tests based on various environments. And horizontal expansion is convenient to run in the GitHub Action test environment.
This is not the final version, and it needs to be improved in these aspects:
- Refactor more test cases. We refactor most of the IT cases, and there is still a small part of the IT cases.
- Implement more cluster environments. Currently, only the pseudo-cluster mode with five nodes is implemented in the cluster mode.
- The refactor of one-node writing and multi-node reading. At present, the same node reads and writes in the cluster environment. In the future, it can be refactored into one-node writing and multi-node reading. This is currently in progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44627477)
Coverage increased (+0.06%) to 67.496% when pulling **62f1ed9f47211ae9d36d8f1e3fb2b8619f2a2b55 on irvine0109:IOTDB-1559** into **3d48a88dc012165610cf7856e9308f1cf488fb97 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44474917)
Coverage increased (+0.008%) to 67.273% when pulling **7b879df2c3c423001c176ccacb52cdaf0b0c702b on irvine0109:IOTDB-1559** into **c4d73e89cc413e73be5fac1f5325ba2515c51860 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r754932031
##########
File path: integration/README.md
##########
@@ -0,0 +1,93 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+Integration Testing
+===================
+
+All integration tests are in this module.
+
+Now integration testing supports three modes, the Local Standalone mode, the Cluster mode and the Remote mode.
+
+Integration Testing with Local Standalone Mode
+-------------------
+
+Integration testing with local standalone mode can be run with both maven and IDEs like IntelliJ.
+
+If you want run all tests include UT and IT, the maven command is:
+```
+mvn clean verify
+```
+
+Optionally, you can also only run IT, the maven command is:
+```
+mvn clean verify -pl integration
+```
+
+And if you want to run IT in the IDE like IntelliJ, pay attention to the `fork mode` in `run options`:
Review comment:
If I want to run a test that is annotated with three modes.
What should I do to run LocalStandaloneTest, ClusterTest and RemoteTest separately?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44471845)
Coverage decreased (-0.02%) to 67.037% when pulling **1340739763f39aaf61cd45df92ce16dfef769103 on irvine0109:IOTDB-1559** into **c6744c53f5cb16dec9ef74a419ce6028a60e96a0 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44621363)
Coverage increased (+0.02%) to 67.369% when pulling **450ec9b0309ceaaa7655326f6db51b3db18c37fb on irvine0109:IOTDB-1559** into **28a78953714556be948b7e4b8b0a93f86190591d on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r754922292
##########
File path: integration/src/test/java/org/apache/iotdb/db/integration/IoTDBCreateStorageGroupIT.java
##########
@@ -40,24 +41,24 @@
* Notice that, all test begins with "IoTDB" is integration test. All test which will start the
* IoTDB server should be defined as integration test.
*/
+@Category({LocalStandaloneTest.class})
Review comment:
I wonder why do not add ClusterTest and RemoteTest...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44474396)
Coverage decreased (-0.002%) to 67.263% when pulling **7b879df2c3c423001c176ccacb52cdaf0b0c702b on irvine0109:IOTDB-1559** into **c4d73e89cc413e73be5fac1f5325ba2515c51860 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r754922292
##########
File path: integration/src/test/java/org/apache/iotdb/db/integration/IoTDBCreateStorageGroupIT.java
##########
@@ -40,24 +41,24 @@
* Notice that, all test begins with "IoTDB" is integration test. All test which will start the
* IoTDB server should be defined as integration test.
*/
+@Category({LocalStandaloneTest.class})
Review comment:
I wonder why do not add ClusterTest...
##########
File path: integration/README.md
##########
@@ -0,0 +1,93 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+Integration Testing
+===================
+
+All integration tests are in this module.
+
+Now integration testing supports three modes, the Local Standalone mode, the Cluster mode and the Remote mode.
+
+Integration Testing with Local Standalone Mode
+-------------------
+
+Integration testing with local standalone mode can be run with both maven and IDEs like IntelliJ.
+
+If you want run all tests include UT and IT, the maven command is:
+```
+mvn clean verify
+```
+
+Optionally, you can also only run IT, the maven command is:
+```
+mvn clean verify -pl integration
+```
+
+And if you want to run IT in the IDE like IntelliJ, pay attention to the `fork mode` in `run options`:
+you need to change `None` to `class` in `Fork mode`.
Review comment:
Run(Menu) -> Edit Configurations -> Modify options
Modify Run Configuration(IT file) -> Modify options
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r755083119
##########
File path: integration/src/test/java/org/apache/iotdb/db/integration/IoTDBCreateStorageGroupIT.java
##########
@@ -40,24 +41,24 @@
* Notice that, all test begins with "IoTDB" is integration test. All test which will start the
* IoTDB server should be defined as integration test.
*/
+@Category({LocalStandaloneTest.class})
Review comment:
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] irvine0109 commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
irvine0109 commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r755016791
##########
File path: integration/README.md
##########
@@ -0,0 +1,93 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+Integration Testing
+===================
+
+All integration tests are in this module.
+
+Now integration testing supports three modes, the Local Standalone mode, the Cluster mode and the Remote mode.
+
+Integration Testing with Local Standalone Mode
+-------------------
+
+Integration testing with local standalone mode can be run with both maven and IDEs like IntelliJ.
+
+If you want run all tests include UT and IT, the maven command is:
+```
+mvn clean verify
+```
+
+Optionally, you can also only run IT, the maven command is:
+```
+mvn clean verify -pl integration
+```
+
+And if you want to run IT in the IDE like IntelliJ, pay attention to the `fork mode` in `run options`:
+you need to change `None` to `class` in `Fork mode`.
Review comment:
Thanks for the reminder. I will add this later.
##########
File path: integration/src/test/java/org/apache/iotdb/db/integration/IoTDBCreateStorageGroupIT.java
##########
@@ -40,24 +41,24 @@
* Notice that, all test begins with "IoTDB" is integration test. All test which will start the
* IoTDB server should be defined as integration test.
*/
+@Category({LocalStandaloneTest.class})
Review comment:
> I wonder why do not add ClusterTest and RemoteTest...
> Good work! There are some class only marked by LocalStandaloneTest, are they could only run on local mode or we could add ClusterTest and RemoteTest also?
Yes, it can add. As the work of catching up with the master branch is continuous, I think all refactor work will be split into 2 temporary phases.
Phase 1,
- This PR should be reviewed.
- All tests should be annotated with the LocalStandaloneTest category.
- All these LocalStandaloneTest should be passed.
- No-brain tutorials should be done.
The milestone of phase 1 is this PR is merged.
Phase 2,
- All tests should be appropriately annotated with the ClusterTest category.
- All these ClusterTest should be passed.
- New IT should be followed by No-brain tutorials.
The milestone of phase 2 is all ITs are passed.
I think phase 1 is done this week is appropriate. And phase 2 is not so urgent.
The above is already mentioned in the #integration-test channel of slack in Chinese. I will sync the progress in slack.
> I guess all test should annotated by three test modes? Unless the function we do not support in cluster.
Yes, not all tests can add ClusterTest and RemoteTest.
Tests with local operations like restart or some descriptor's method invoking can't add for ClusterTest and RemoteTest and tests with config change can't add for RemoteTest.
> Some no brain tutorials are needed for others to work with the new IT framwork.
The no-brain tutorials are in progress. I will send the mail to [dev@iotdb.apache.org](mailto:dev@iotdb.apache.org) when finished. The mail will send tomorrow at the latest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44627181)
Coverage increased (+0.05%) to 67.483% when pulling **62f1ed9f47211ae9d36d8f1e3fb2b8619f2a2b55 on irvine0109:IOTDB-1559** into **3d48a88dc012165610cf7856e9308f1cf488fb97 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] irvine0109 commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
irvine0109 commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r755016852
##########
File path: integration/src/test/java/org/apache/iotdb/db/integration/IoTDBCreateStorageGroupIT.java
##########
@@ -40,24 +41,24 @@
* Notice that, all test begins with "IoTDB" is integration test. All test which will start the
* IoTDB server should be defined as integration test.
*/
+@Category({LocalStandaloneTest.class})
Review comment:
> I wonder why do not add ClusterTest and RemoteTest...
> Good work! There are some class only marked by LocalStandaloneTest, are they could only run on local mode or we could add ClusterTest and RemoteTest also?
Yes, it can add. As the work of catching up with the master branch is continuous, I think all refactor work will be split into 2 temporary phases.
Phase 1,
- This PR should be reviewed.
- All tests should be annotated with the LocalStandaloneTest category.
- All these LocalStandaloneTest should be passed.
- No-brain tutorials should be done.
The milestone of phase 1 is this PR is merged.
Phase 2,
- All tests should be appropriately annotated with the ClusterTest category.
- All these ClusterTest should be passed.
- New IT should be followed by No-brain tutorials.
The milestone of phase 2 is all ITs are passed.
I think phase 1 is done this week is appropriate. And phase 2 is not so urgent.
The above is already mentioned in the #integration-test channel of slack in Chinese. I will sync the progress in slack.
> I guess all test should annotated by three test modes? Unless the function we do not support in cluster.
Yes, not all tests can add ClusterTest and RemoteTest.
Tests with local operations like restart or some descriptor's method invoking can't add for ClusterTest and RemoteTest and tests with config change can't add for RemoteTest.
> Some no brain tutorials are needed for others to work with the new IT framwork.
The no-brain tutorials are in progress. I will send the mail to [dev@iotdb.apache.org](mailto:dev@iotdb.apache.org) when finished. The mail will send tomorrow at the latest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#issuecomment-976179720
[](https://coveralls.io/builds/44582765)
Coverage increased (+0.04%) to 67.353% when pulling **887def97569c23970d1f7ea3d192054fb7e4cc87 on irvine0109:IOTDB-1559** into **521f5626297e834049e8f8af94bac09ede3b8143 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] irvine0109 commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
irvine0109 commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r755016990
##########
File path: integration/README.md
##########
@@ -0,0 +1,93 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+Integration Testing
+===================
+
+All integration tests are in this module.
+
+Now integration testing supports three modes, the Local Standalone mode, the Cluster mode and the Remote mode.
+
+Integration Testing with Local Standalone Mode
+-------------------
+
+Integration testing with local standalone mode can be run with both maven and IDEs like IntelliJ.
+
+If you want run all tests include UT and IT, the maven command is:
+```
+mvn clean verify
+```
+
+Optionally, you can also only run IT, the maven command is:
+```
+mvn clean verify -pl integration
+```
+
+And if you want to run IT in the IDE like IntelliJ, pay attention to the `fork mode` in `run options`:
Review comment:
If you want to run any one of these three categories in the IDE like IntelliJ, like this:
And if you want to run in maven cmd, use a profile that is already defined in integration/pom.xml.
Profile id with "LocalStandalone", "Remote" and "Cluster" can work well with these three categories separately.
In integration/pom.xml, the <test.includedGroups> part of each profile choose the category to run.
```
<profile>
<id>LocalStandalone</id>
<properties>
<test.includedGroups>org.apache.iotdb.itbase.category.LocalStandaloneTest</test.includedGroups>
<test.excludedGroups/>
</properties>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #4371: [IOTDB-1559] Refactor the IT framework
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #4371:
URL: https://github.com/apache/iotdb/pull/4371#discussion_r758997646
##########
File path: .github/workflows/cluster.yml
##########
@@ -0,0 +1,53 @@
+name: Cluster Test
+
+on:
+ push:
+ branches:
+ - master
+ - 'rel/*'
+ - Vector
+ paths-ignore:
+ - 'docs/**'
+ pull_request:
+ branches:
+ - master
+ - 'rel/*'
+ - cluster_new
+ - Vector
Review comment:
```suggestion
branches:
- master
- 'rel/*'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- master
- 'rel/*'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org