You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by ka...@apache.org on 2021/07/15 12:15:51 UTC
[submarine] branch master updated: SUBMARINE-941. Improve documents
about operator and integration tests
This is an automated email from the ASF dual-hosted git repository.
kaihsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new adc083c SUBMARINE-941. Improve documents about operator and integration tests
adc083c is described below
commit adc083c8c13d6ac6d382d4fb333b5ca3bcdd2187
Author: Kai-Hsun Chen <b0...@ntu.edu.tw>
AuthorDate: Thu Jul 15 19:44:38 2021 +0800
SUBMARINE-941. Improve documents about operator and integration tests
### What is this PR for?
Improve documents about operator and integration tests
### What type of PR is it?
[Documentation]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-941
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: Kai-Hsun Chen <b0...@ntu.edu.tw>
Signed-off-by: Kai-Hsun Chen <ka...@apache.org>
Closes #682 from kevin85421/SUBMARINE-941 and squashes the following commits:
57c63558 [Kai-Hsun Chen] Fix bug
6dedc0d7 [Kai-Hsun Chen] SUBMARINE-941. Improve documents about submarine-operator and integration tests
---
website/docs/devDocs/Development.md | 106 ++++++++++++++---------------
website/docs/devDocs/IntegrationTest.md | 95 --------------------------
website/docs/devDocs/IntegrationTestE2E.md | 37 +++++++---
website/docs/devDocs/IntegrationTestK8s.md | 12 ++--
website/docs/gettingStarted/kind.md | 13 ++--
website/sidebars.js | 1 -
6 files changed, 90 insertions(+), 174 deletions(-)
diff --git a/website/docs/devDocs/Development.md b/website/docs/devDocs/Development.md
index 0934c1d..dfb82d0 100644
--- a/website/docs/devDocs/Development.md
+++ b/website/docs/devDocs/Development.md
@@ -17,30 +17,28 @@ title: Development Guide
limitations under the License.
-->
-## Overview
-
-From [Getting Started/Submarine Local Deployment](../gettingStarted/localDeployment.md), you already know that Submarine is installed and uninstalled by Helm. As you can see by `kubectl get pods`, there are nine major components in Submarine, including `notebook-controller`, `pytorch-operator`, `submarine-database`, `submarine-minio`, `submarine-mlflow`, `submarine-server`, `submarine-tensorboard`, `submarine-traefik`, and `tf-job-operator`. They are launched as pods in kubernetes from t [...]
-
-Some of the components are borrowed from other projects (kubeflow, traefik), including `notebook-controller`, `pytorch-operator`, `submarine-traefik` and `tf-job-operator`. The rest of them are built by ourselves, including `submarine-database`, `submarine-minio`, `submarine-mlflow`, `submarine-tensorboard`, and `submarine-server`.
-
-The purpose of the components are as the following:
-
-1. `tf-job-operator`: manage the operation of tensorflow jobs
-2. `pytorch-operator`: manage the operation of pytorch jobs
-3. `notebook-controller`: manage the operation of notebook instances
-4. `submarine-traefik`: manage the ingress service
-
-5. `submarine-database`: store metadata in mysql database
-6. `submarine-minio`: store machine learning artifacts in minio object storage
-7. `submarine-mlflow`: platform for managing the end-to-end machine learning lifecycle
-8. `submarine-tensorboard`: tool for providing the measurements and visualizations during ml workflow.
-9. `submarine-server`: handle api request, submit job to container orchestration, and connect with database.
-
-In this document, we focus on `submarine-server` and `submarine-database`. You can learn how to develop server, database, and workbench here.
-
+# Project Overview
+The document [Submarine Local Deployment](../gettingStarted/localDeployment.md) shows how to deploy the Submarine service to your Kubernetes cluster. The Submarine service consists mainly of nine components, and you can check them with the following command:
+
+```
+kubectl get pods -n ${your_namespace}
+```
+
+A brief introduction about these components:
+
+1. **tf-operator**: Enable users to run TensorFlow jobs distributedly
+2. **pytorch-operator**: Enable users to run PyTorch jobs distributedly
+3. **notebook-controller**: Jupyter Notebook controller
+4. **submarine-traefik**: Kubernetes Ingress controller
+5. **submarine-database**: A MySQL database to store metadata
+6. **submarine-minio**: An object store for machine learning artifacts
+7. **submarine-mlflow**: A platform for model management
+8. **submarine-tensorboard**: A visualization tool for distributed training experiments
+9. **submarine-server**: Handle API requests, and submit distributed training experiments to Kubernetes.
+
+# Submarine Development
## Video
-
-From [This Video](https://youtu.be/32Na2k6Alv4), you will know how to deal with the configuration of Submarine and be able to contribute to it via Github.
+* From this [Video](https://youtu.be/32Na2k6Alv4), you will know how to deal with the configuration of Submarine and be able to contribute to it via Github.
## Develop server
@@ -69,9 +67,7 @@ Checkstyle plugin may help to detect violations directly from the IDE.
For each class, there is a corresponding testClass. For example, `SubmarineServerTest` is used for testing `SubmarineServer`. Whenever you add a funtion in classes, you must write a unit test to test it.
-- Integration Test
-
- See [IntegrationTest.md](./IntegrationTest.md)
+- Integration Test: [IntegrationTestK8s.md](./IntegrationTestK8s.md)
### Build from source
@@ -129,6 +125,8 @@ Checkstyle plugin may help to detect violations directly from the IDE.
1. The request sent to `http://localhost:4200` will be redirected to `http://localhost:32080`.
2. Open `http://localhost:4200` in browser to see the real-time change of workbench.
+4. Frontend E2E test: [IntegrationTestE2E.md](./IntegrationTestE2E.md)
+
## Develop database
1. Build the docker image
@@ -159,56 +157,56 @@ Checkstyle plugin may help to detect violations directly from the IDE.
1. Start the minikube cluster
- ```bash
- minikube start --vm-driver=docker --kubernetes-version v1.15.11
- ```
+ ```bash
+ minikube start --vm-driver=docker --kubernetes-version v1.15.11
+ ```
2. Install the dependencies
- ```bash
- cd submarine-cloud-v2/
- go mod vendor
- ```
+ ```bash
+ cd submarine-cloud-v2/
+ go mod vendor
+ ```
3. Run the operator out-of-cluster
- ```bash
- make
- ./submarine-operator
- ```
+ ```bash
+ make
+ ./submarine-operator
+ ```
4. Deploy a Submarine
- ```bash
- kubectl apply -f artifacts/examples/crd.yaml
- kubectl create ns submarine-user-test
- kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml
- ```
+ ```bash
+ kubectl apply -f artifacts/examples/crd.yaml
+ kubectl create ns submarine-user-test
+ kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml
+ ```
5. Exposing service
- ```bash
- # Method1 -- use minikube ip
- minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
+ ```bash
+ # Method1 -- use minikube ip
+ minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
- # Method2 -- use port-forwarding
- kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik 32080:80
- ```
+ # Method2 -- use port-forwarding
+ kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik 32080:80
+ ```
6. View workbench
- If you use method 1 in step 5, please go to `http://{minikube ip}:32080`, ex: http://192.168.49.2:32080
+ If you use method 1 in step 5, please go to `http://{minikube ip}:32080`, ex: http://192.168.49.2:32080
- If you use method 2 in step 5, please go to http://127.0.0.1:32080
+ If you use method 2 in step 5, please go to http://127.0.0.1:32080
7. Delete submarine
- ```bash
- kubectl delete submarine example-submarine -n submarine-user-test
- ```
+ ```bash
+ kubectl delete submarine example-submarine -n submarine-user-test
+ ```
8. Stop the operator
- Press ctrl+c to stop the operator.
+ Press ctrl+c to stop the operator.
For other details, please check out the [README](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/README.md) and [Developer Guide](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/docs/developer-guide.md) on GitHub.
diff --git a/website/docs/devDocs/IntegrationTest.md b/website/docs/devDocs/IntegrationTest.md
deleted file mode 100644
index dd94141..0000000
--- a/website/docs/devDocs/IntegrationTest.md
+++ /dev/null
@@ -1,95 +0,0 @@
----
-title: How to Run Integration Test
----
-
-<!---
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-## Introduction
-1. Now, Apache Submarine supports two kinds of integration test: `test-e2e` and `test-k8s`. These two modules can be found in the [submarine/submarine-test](https://github.com/apache/submarine/tree/master/submarine-test) directory.
-
-2. Currently, there are some differences between `test-e2e` and `test-k8s` in operation mode. To elaborate, `test-e2e` needs to deploy Apache Submarine locally, while `test-k8s` deploys Apache Submarine via k8s.
-
-3. These two test modules can be applied to different test scenarios. (In the future, these two test modules may be combined or adjusted)
-
-## E2E test
-
-### E2E tests can be executed both locally and in Travis (For workbench developer)
-* Run E2E tests locally:
- * Step1: Follow [HowToRun.md](https://github.com/apache/submarine/blob/master/website/docs/adminDocs/yarn/workbench/HowToRun.md) to launch the submarine-server and database.
- * Step2: Run workbench (Angular version) locally
- ```
- cd submarine/submarine-workbench/workbench-web
- npm start
- // Check 127.0.0.1:4200
- ```
- * Step3: Modify the port from 8080 to 4200
- * [WebDriverManager.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/WebDriverManager.java): `url = "http://localhost:8080";` --> `url = "http://localhost:4200";`
- * [Your Unit test case](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration): `8080` --> `4200`
- * Step4: Comment the `headless` option
- * [ChromeWebDriverProvider.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/ChromeWebDriverProvider.java): `chromeOptions.addArguments("--headless");` --> `//chromeOptions.addArguments("--headless");`
- * With the `headless` option, the selenium will be executed in background.
- * Step5: Run E2E test cases (Please check the following section **Run the existing tests**)
-* Run E2E tests in Travis:
- * Step1: Make sure that the port must be 8080 rather than in [WebDriverManager.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/WebDriverManager.java) and [all test cases](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration).
- * Step2: Make sure that the `headless` option is not commented in [ChromeWebDriverProvider.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/ChromeWebDriverProvider.java).
- * Step3: If you push the commit to Github, the Travis CI will execute automatically and you can check it in `https://travis-ci.com/${your_github_account}/${your_repo_name}`.
-### Run the existing tests.
-##### Move to the working directory.
-```
-cd submarine/submarine-test/test-e2e
-```
-##### Compile & Run.
-
-> Following command will compile all files and run **all** files ending with "IT" in the [directory](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration).
-* For linux
- ```
- mvn verify
- ```
-* For MacOS
-```
-mvn clean install -U
-```
-> Run a specific testcase
-```
-mvn -Dtest=${your_test_case_file_name} test //ex: mvn -Dtest=loginIT test
-```
-
-##### Result
-If all of the function under test are succeeded, it will show.
-```
-BUILD SUCCESS
-```
-Otherwise, it will show.
-```
-BUILD FAILURE
-```
-
-### Add your own integration test
-1. Create a new file ending with "IT" under "submarine/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/".
-2. Your public class is recommended to extend AbstractSubmarineIT. The class AbstractSubmarineIT contains some commonly used functions.
-```java
- WebElement pollingWait(final By locator, final long timeWait); // Find element on the website.
- void clickAndWait(final By locator); // Click element and wait for 1 second.
- void sleep(long millis, boolean logOutput); // Let system sleep a period of time.
-```
-3. There are also some commonly used functions except in AbstractSubmarineIT.java.
-```java
- // In WebDriverManager.java:
- public static WebDriver getWebDriver(); // This return a firefox webdriver which has been set to your workbench website.
-```
-4. Add [JUnit](https://junit.org/junit5/docs/current/user-guide/) annotation before your testing function, e.g., @Beforeclass, @Test, and @AfterClass. You can refer to [loginIT.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/loginIT.java).
-5. Use command mentioned above to compile and run to test whether it works as your anticipation.
-
-
diff --git a/website/docs/devDocs/IntegrationTestE2E.md b/website/docs/devDocs/IntegrationTestE2E.md
index adc7c9a..46317dd 100644
--- a/website/docs/devDocs/IntegrationTestE2E.md
+++ b/website/docs/devDocs/IntegrationTestE2E.md
@@ -1,5 +1,5 @@
---
-title: How to Run integration E2E Test
+title: How to Run Frontend Integration Test
---
<!---
@@ -15,10 +15,9 @@ title: How to Run integration E2E Test
-->
## Introduction
+* The test cases under the directory [test-e2e](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration) are integration tests to ensure the correctness of the Submarine Workbench.
-* It checks the components in the website works correctly.
-
-* You can run the test-e2e either locally or on GitHub Actions.
+* These test cases can be run either locally or on GitHub Actions.
## Run E2E test locally
@@ -26,16 +25,16 @@ title: How to Run integration E2E Test
2. Forward port
- ```bash
- kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80
- ```
+ ```bash
+ kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80
+ ```
3. Modify run_frontend_e2e.sh
You need to modify the port and the URL in this script to where you run the workbench on.
> Example:
- > If you just finished developing the workbench appearance and the workbench is running on localhost:4200, then you should modify the WORKBENCH_PORT to 4200
+ > If your Submarine workbench is running on 127.0.0.1:4200, you should modify the **WORKBENCH_PORT** to 4200.
```bash
# at submarine-test/test_e2e/run_frontend_e2e.sh
@@ -49,7 +48,7 @@ title: How to Run integration E2E Test
...
```
-4. Run run_frontend_e2e.sh
+4. Run run_frontend_e2e.sh (Run a specific test case)
This script will check whether the port can be accessed or not, and run the test case.
```bash
@@ -58,6 +57,24 @@ title: How to Run integration E2E Test
# TESTCASE is the IT you want to run, ex: loginIT, experimentIT...
```
+5. Run all test cases
+* Following commands will compile all files and run all files ending with "IT" in the [directory](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration).
+ ```bash
+ # Make sure the Submarine workbench is running on 127.0.0.1:8080
+ cd submarine/submarine-test/test-e2e
+ # Method 1:
+ mvn verify
+
+ # Method 2:
+ mvn clean install -U
+ ```
+
## Run E2E test in GitHub Actions
-Each time a code is submitted, GitHub Actions is automatically triggered for testing.
\ No newline at end of file
+* Each time a commit is pushed, GitHub Actions will be triggered automatically.
+
+## Add a new frontend E2E test case
+* **WARNING**
+ * You **MUST** read the [document](https://www.selenium.dev/documentation/en/webdriver/waits/) carefully, and understand the difference between **explicit wait**, **implicit wait**, and **fluent wait**.
+ * **Do not mix implicit and explicit waits.** Doing so can cause unpredictable wait times.
+* We define many useful functions in [AbstractSubmarineIT.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java).
\ No newline at end of file
diff --git a/website/docs/devDocs/IntegrationTestK8s.md b/website/docs/devDocs/IntegrationTestK8s.md
index 75c2802..8f98e3f 100644
--- a/website/docs/devDocs/IntegrationTestK8s.md
+++ b/website/docs/devDocs/IntegrationTestK8s.md
@@ -18,13 +18,12 @@ title: How to Run Integration K8s Test
## Introduction
-* It checks the API on each page works correctly.
+* The test cases under the directory `test-k8s` are integration tests to ensure the correctness of the Submarine RESTful API.
-* You can run the test-k8s either locally or on GitHub Actions.
- * Before running the test-k8s, the minikube (KinD) cluster must be created.
+* You can run these tests either locally or on GitHub Actions.
+ * Before running the tests, the minikube (KinD) cluster must be created.
* Then, compile and package the submarine project in `submarine-dist` directory for building a docker image.
- * Otherwise, the 8080 port in submarine-traefik should be forward.
- * Finally, the test case under the `test-k8s` directory is ready to run.
+ * In addition, the 8080 port in submarine-traefik should be forwarded.
## Run k8s test locally
@@ -45,5 +44,4 @@ title: How to Run Integration K8s Test
```
## Run k8s test in GitHub Actions
-
-Each time a code is submitted, GitHub Actions is automatically triggered for testing.
+* Each time a code is submitted, GitHub Actions is triggered automatically.
diff --git a/website/docs/gettingStarted/kind.md b/website/docs/gettingStarted/kind.md
index 8fbb18e..5c18432 100644
--- a/website/docs/gettingStarted/kind.md
+++ b/website/docs/gettingStarted/kind.md
@@ -21,10 +21,10 @@ specific language governing permissions and limitations
under the License.
-->
-## Create K8s cluster
-We recommend using [`KinD`](https://kind.sigs.k8s.io/) to setup a Kubernetes cluster on a local machine.
+## Create Kubernetes cluster with KinD
+We recommend users developing Submarine with minikube. However, [`KinD`](https://kind.sigs.k8s.io/) is also an option to setup a Kubernetes cluster on your local machine.
-Running the following command, and specify the KinD version and Kubernetes version [`here`](../devDocs/Dependencies).
+Run the following command, and specify the KinD version and Kubernetes version [`here`](../devDocs/Dependencies).
```bash
# Download the specific version of KinD (must >= v0.6.0)
export KIND_VERSION=v0.11.1
@@ -41,20 +41,20 @@ kind create cluster --image kindest/node:${KUBE_VERSION}
## Kubernetes Dashboard (optional)
### Deploy
-To deploy Dashboard, execute following command:
+To deploy Dashboard, execute the following command:
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
```
### Create RBAC
-Ensure to grant the cluster access permission of dashboard, run the following command:
+Run the following commands to grant the cluster access permission of dashboard:
```
kubectl create serviceaccount dashboard-admin-sa
kubectl create clusterrolebinding dashboard-admin-sa --clusterrole=cluster-admin --serviceaccount=default:dashboard-admin-sa
```
### Get access token (optional)
-If you want to use the token to login the dashboard, run the following command to get key:
+If you want to use the token to login the dashboard, run the following commands to get key:
```
kubectl get secrets
# select the right dashboard-admin-sa-token to describe the secret
@@ -62,7 +62,6 @@ kubectl describe secret dashboard-admin-sa-token-6nhkx
```
### Start dashboard service
-To start the dashboard service, we can run the following command:
```
kubectl proxy
```
diff --git a/website/sidebars.js b/website/sidebars.js
index cf4693b..80fd24d 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -66,7 +66,6 @@ module.exports = {
"devDocs/Dependencies",
"devDocs/BuildFromCode",
"devDocs/Development",
- "devDocs/IntegrationTest",
"devDocs/IntegrationTestK8s",
"devDocs/IntegrationTestE2E",
],
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org