You are viewing a plain text version of this content. The canonical link for it is here.
Posted to submarine-dev@hadoop.apache.org by zh...@apache.org on 2019/10/10 12:16:52 UTC
[hadoop-submarine] branch master updated: SUBMARINE-217. Add
Submarine Workbench introduction document
This is an automated email from the ASF dual-hosted git repository.
zhouquan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 2736913 SUBMARINE-217. Add Submarine Workbench introduction document
2736913 is described below
commit 273691339725351fee9a7887a200fd2ea4928082
Author: Xun Liu <li...@apache.org>
AuthorDate: Tue Oct 8 08:57:00 2019 +0800
SUBMARINE-217. Add Submarine Workbench introduction document
### What is this PR for?
Submarine workbench introduction document mainly contains following parts:
1. workbench homepage
2. Project create/list/delete/modify manager
### What type of PR is it?
[Documentation]
### Todos
* [ ] - Interpreter create/list/delete/modify manager
* [ ] - Job create/list/delete/modify manager
* [ ] - Data create/list/delete/modify manager
* [ ] - Model create/list/delete/modify manager
* [ ] - System user/role/team/organization manager
### What is the Jira issue?
* https://issues.apache.org/jira/browse/SUBMARINE-217
### How should this be tested?
* [CI Pass](https://travis-ci.org/liuxunorg/hadoop-submarine/builds/594576189)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Xun Liu <li...@apache.org>
Closes #32 from liuxunorg/SUBMARINE-217 and squashes the following commits:
40cb5d5 [Xun Liu] Fixed some typo
d00bc10 [Xun Liu] Fixed some typo
1286b8e [Xun Liu] SUBMARINE-217. Add Submarine Workbench introduction document
---
docs/workbench/{README.md => HowToRun.md} | 10 +-
docs/workbench/README.md | 225 +++++++++++++++------
docs/workbench/README.zh-CN.md | 176 ++++++++++++++++
docs/workbench/assets/workspace-project-step1.png | Bin 0 -> 215256 bytes
.../assets/workspace-project-step2-blank.png | Bin 0 -> 140145 bytes
.../assets/workspace-project-step2-git.png | Bin 0 -> 212974 bytes
.../assets/workspace-project-step2-template.png | Bin 0 -> 159251 bytes
.../assets/workspace-project-step2-upload.png | Bin 0 -> 138045 bytes
docs/workbench/assets/workspace-project-step3.png | Bin 0 -> 181445 bytes
docs/workbench/assets/workspace-project.png | Bin 0 -> 312940 bytes
10 files changed, 342 insertions(+), 69 deletions(-)
diff --git a/docs/workbench/README.md b/docs/workbench/HowToRun.md
similarity index 89%
copy from docs/workbench/README.md
copy to docs/workbench/HowToRun.md
index 7cc3baa..618dedc 100644
--- a/docs/workbench/README.md
+++ b/docs/workbench/HowToRun.md
@@ -12,11 +12,7 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-# Submarine Workbench
-
-Submarine Workbench is an UI designed for data scientists.
-Data scientists can interact with Submarine Workbench UI to access notebooks,
-submit/manage jobs, manage models, create model training workflows, access dataset, etc.
+# How To Run Submarine Workbench
## Run Workbench
@@ -42,7 +38,7 @@ cd submarine
| WORKBENCH_JAVA_OPTS | Set the JAVA OPTS parameter when the Workbench process starts. If you need to debug the Workbench process, you can set it to `-agentlib:jdwp=transport=dt_socket, server=y,suspend=n,address=5005` |
| WORKBENCH_MEM | Set the java memory parameter when the Workbench process starts. |
| MYSQL_JAR_URL | The customized URL to download mysql jdbc jar. |
-| MYSQL_VERSION | The version of mysql jdbc jar would be downloaded. The default value is 5.1.39. It's used to generate the default value of MYSQL_JDBC_URL|
+| MYSQL_VERSION | The version of mysql jdbc jar to downloaded. The default value is 5.1.39. It's used to generate the default value of MYSQL_JDBC_URL |
## submarine-site.xml
@@ -68,7 +64,7 @@ cd submarine
## Compile
-[Build From Code Guide](./development/BuildFromCode.md)
+[Build From Code Guide](../development/BuildFromCode.md)
```$xslt
cd submarine/submarine-dist/target/submarine-dist-<version>/submarine-dist-<version>/
diff --git a/docs/workbench/README.md b/docs/workbench/README.md
index 7cc3baa..274413a 100644
--- a/docs/workbench/README.md
+++ b/docs/workbench/README.md
@@ -12,65 +12,166 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-# Submarine Workbench
-
-Submarine Workbench is an UI designed for data scientists.
-Data scientists can interact with Submarine Workbench UI to access notebooks,
-submit/manage jobs, manage models, create model training workflows, access dataset, etc.
-
-## Run Workbench
-
-```$xslt
-cd submarine
-./bin/workbench-daemon.sh [start|stop|restart]
-```
-To start workbench server, you need to download mysql jdbc jar and put it in the
-path of workbench/lib for the first time. Or you can add parameter, getMysqlJar,
-to get mysql jar automatically.
-```$xslt
-cd submarine
-./bin/workbench-daemon.sh start getMysqlJar
-```
-
-## submarine-env.sh
-
-`submarine-env.sh` is automatically executed each time the `workbench.sh` script is executed, so we can set the `workbench.sh` script and the environment variables in the `WorkbenchServer` process via `submarine-env.sh`.
-
-| Name | Variable |
-| ------------------- | ------------------------------------------------------------ |
-| JAVA_HOME | Set your java home path, default is `java`. |
-| WORKBENCH_JAVA_OPTS | Set the JAVA OPTS parameter when the Workbench process starts. If you need to debug the Workbench process, you can set it to `-agentlib:jdwp=transport=dt_socket, server=y,suspend=n,address=5005` |
-| WORKBENCH_MEM | Set the java memory parameter when the Workbench process starts. |
-| MYSQL_JAR_URL | The customized URL to download mysql jdbc jar. |
-| MYSQL_VERSION | The version of mysql jdbc jar would be downloaded. The default value is 5.1.39. It's used to generate the default value of MYSQL_JDBC_URL|
-
-## submarine-site.xml
-
-`submarine-site.xml` is the configuration file for the entire `Submarine` system to run.
-
-| Name | Variable |
-| ---------------------------------- | ------------------------------------------------------------ |
-| workbench.server.addr | workbench server address, default is `0.0.0.0` |
-| workbench.server.port | workbench server port, default `8080` |
-| workbench.ssl | Should SSL be used by the workbench servers?, default `false` |
-| workbench.server.ssl.port | Server ssl port. (used when ssl property is set to true), default `8483` |
-| workbench.ssl.client.auth | Should client authentication be used for SSL connections? |
-| workbench.ssl.keystore.path | Path to keystore relative to submarine configuration directory |
-| workbench.ssl.keystore.type | The format of the given keystore (e.g. JKS or PKCS12) |
-| workbench.ssl.keystore.password | Keystore password. Can be obfuscated by the Jetty Password tool |
-| workbench.ssl.key.manager.password | Key Manager password. Defaults to keystore password. Can be obfuscated. |
-| workbench.ssl.truststore.path | Path to truststore relative to submarine configuration directory. Defaults to the keystore path |
-| workbench.ssl.truststore.type | The format of the given truststore (e.g. JKS or PKCS12). Defaults to the same type as the keystore type |
-| workbench.ssl.truststore.password | Truststore password. Can be obfuscated by the Jetty Password tool. Defaults to the keystore password |
-| workbench.web.war | Submarine workbench web war file path. |
-
-
-
-## Compile
-
-[Build From Code Guide](./development/BuildFromCode.md)
-
-```$xslt
-cd submarine/submarine-dist/target/submarine-dist-<version>/submarine-dist-<version>/
-./bin/workbench-daemon.sh [start|stop|restart]
-```
+
+English | [简体中文](./README.zh-CN.md)
+
+# Submarine Workbench Introduction
+
+Submarine Workbench is a WEB system for data scientists.
+Data scientists can interactively access notebooks, submit/manage jobs, manage models, create model training workflows, access data sets, and more through Submarine Workbench.
+
+## Register
+
+Everyone who needs to use Submarine for machine learning algorithm development can log in to Submarine Workbench's WEB homepage. On the homepage, click the registration link, fill in the user name, email address and password to register the user. At this time, the user status is `waiting for approval` status.
+
+After receiving the registration request from the user in Submarine Workbench, the administrator sets the operation authority according to the user's needs, sets the user's organization and allocates resources, and sets the user status to `pass the audit`. The user can log in to the Submarine Workbench. Different users have different permission.
+
+## Login
+
+Each Submarine user logs in to the Home page of Submarine Workbench by entering their username and password on the Login page.
+
+## Home
+
+In the Submarine Workbench Home page, the top level shows the user's resource usage and task execution through four charts.
+
+In the `Quick Start` list, the most commonly used feature links in the Workbench are displayed so that users can work quickly.
+
+In the `Open Recent` list, there are nine items that the user has used recently, so you can work quickly.
+
+At `What's New?` In the list, some of the latest features and project information released by Submarine are displayed to help you understand the latest developments in the Subamrine project.
+
+## Workspace
+
+Workspace consists primarily of five tab pages, with the total number of items in each tab page's title.
+
+### Project
+
+In the Project page, all the projects created by the user themselves are displayed as cards.
+
+
+
+Each Project card consists of the following sections:
+
+1. **Project Type**：Submarine currently supports six types of machine learning algorithm frameworks and development languages: `Notebook`, `Python`, `R`, `Scala`, `Tensorflow`, and `PyTorch`, which are identified by corresponding icons in the project card.
+2. **Project Tags**：Users can tag each Project with different tags for easy searching and management.
+3. **Github/Gitlab integrated**：Submarine Workbench is system integrated with `Github`/`Gitlab`, and each Project can perform `Watch`, `Star`, `Fork`, and `Comment `operations in Workbench.
+ + **Watch**：[TODO]
+ + **Star**：[TODO]
+ + **Fork**：[TODO]
+ + **Comment**：Users can comment on the project.
+4. **Edit**：Users can open projects in **Notebook** and perform algorithm development by double-clicking on the project or by clicking the **Edit** button.
+5. **Download**：The user downloads the project package locally by clicking the **Download** button.
+6. **Setting**：Edit project information such as project name, profile, visibility level and permissions.
+7. **Delete**：Delete the project and all included files.
+
+#### Add New Project
+
+Clicking the **Add New Project** button on the project page will display the guide page for creating the project, and you can create a new project in just three steps.
+
+**Step 1**: Fill in the project name and project description in the **Base Information** step.
+
+
+
++ **Visibility**: Set the visibility level of the item externally
+
+ + **Private**: (Default) Set to private project, and all the files included in the project are not publicly displayed. but the execution result of the project can be individually set and exposed in Notebook, so that others can view the visual report of the project.
+ + **Team**: Set to team project, select the team name in the team selection box, and other members of the team can access the project according to the set permissions.
+ + **Public**: Set to public project, all users in Workbench can view this project through search.
++ **Permission**: Set the external access rights of the project. The permission setting interface will appear only when the **Visibility** of the project is set to **Team** or **Public**.
+
+ + **Can View**
+
+ When the project's **Visibility** is set to **Team**, other members of the team can **only view** the files for this project.
+
+ When the project's **Visibility** is set to **Public**, other members of the Workbench can **only view** the files for this project.
+
+ + **Can Edit**
+
+ When the project's **Visibility** is set to **Team**, other members of the team can **view** and **edit** the files for this project.
+
+ When the project's **Visibility** is set to **Public**, other members of the Workbench can **view** and **edit** the files for this project.
+
+ + **Can Execute**
+
+ When the project's **Visibility** is set to **Team**, other members of the team can **view**, **edit**, and **execute** the project's files.
+
+ When the project's **Visibility** is set to **Public**, other members of the Workbench can **view**, **edit**, and **execute** the project's files.
+
+**Step 2**: In the Initial Project step, Workbench provides four ways to initialize the project.
+
++ **Template**: Workbench Project templates with several different development languages and algorithm frameworks are built in. You can choose any template to initialize your project and you can execute it directly in Notebook without any modification. It is especially suitable for novices to experience quickly.
+
+ 
+
++ **Blank**：Create a blank project, and later we can manually add the project's file in Notebook
+
+ 
+
++ **Upload**: Initialize your project by uploading a file in notebook format that is compatible with the **Jupyter Notebook** and **Zeppelin Notebook** file formats.
+
+ 
+
++ **Git Repo**: Fork a file in the repository to initialize the project in your **Github**/**Gitlab** account.
+
+ 
+
+**Step 3**：Preview the included files in the project
+
+
+
++ **Save**: Save the project to Workspace.
++ **Open In Notebook**: Save the project to Workspace and open the project with Notebook.
+
+### Release
+
+[TODO]
+
+### Training
+
+[TODO]
+
+### Team
+
+[TODO]
+
+### Shared
+
+[TODO]
+
+## Interpreters
+
+[TODO]
+
+## Job
+
+[TODO]
+
+## Data
+
+[TODO]
+
+## Model
+
+[TODO]
+
+## Manager
+
+### User
+
+[TODO]
+
+### Team
+
+[TODO]
+
+### Data Dict
+
+[TODO]
+
+### Department
+
+[TODO]
+
+## How to run workbench
+
+[How To Run Submarine Workbench Guide](./HowToRun.md)
\ No newline at end of file
diff --git a/docs/workbench/README.zh-CN.md b/docs/workbench/README.zh-CN.md
new file mode 100644
index 0000000..0d256a4
--- /dev/null
+++ b/docs/workbench/README.zh-CN.md
@@ -0,0 +1,176 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+[English](./README.md) | 简体中文
+
+# Submarine Workbench Introduction
+
+`Submarine Workbench` 是为数据科学家设计的 WEB 系统。数据科学家可以通过 `Submarine Workbench` 进行交互式的访问 `Notebook`，提交/管理 Job，管理模型，创建模型训练工作流，访问数据集等。
+
+## Register
+
+每个需要使用 Submarine 进行机器学习算法开发的用户，都可以登录 `Submarine Workbench` 的 WEB 首页，在首页上，点击注册链接，填写用户名、注册邮箱和密码就可以完成注册，但此时用户状态为 `等待审核` 状态。
+
+管理员在 `Submarine Workbench` 中接收到用户的注册请求后，设置用户的操作权限，所属机构部门和分配资源，设置用户状态为 `审核通过` 后，用户才可以登录 Submarine Workbench。
+
+## Login
+
+每个 Submarine 的用户在 `Login` 页面中输入用户名和密码，登录到 `Submarine Workbench` 的首页 `Home`。
+
+## Home
+
+在 `Submarine Workbench` 的 `Home` 首页中，顶层通过四个图表显示了用户的资源的使用情况和任务执行的情况。
+
+在 `Quick Start` 列表中，显示了 Workbench 中最常使用的功能链接，方便用户可以快速的进行工作。
+
+在 `Open Recent` 列表中，显示了用户最近使用过的九个项目，方便你快速的进行工作。
+
+在 `What‘s New？` 列表中，显示了 Submarine 最新发布的一些功能特性和项目信息，方便你了解 Subamrine 项目的最新进展。
+
+## Workspace
+
+Workspace 主要有五个 Tab 页组成，每个 Tab 页的标题中显示了各自项目的总数。
+
+### Project
+
+在 Project 页面中，以卡片的方式显示了用户自己创建的所有 Project。
+
+
+
+每个 Project 卡片由以下部分内容组成：
+
+1. **Project 类型**：目前 Submarine 支持 `Notebook`、`Python`、`R`、`Scala`、`Tensorflow` 和 `PyTorch` 这六种类型的机器学习算法框架和开发语言，在项目卡片中以对应的图标进行标识。
+2. **Project Tags**：用户可以为每个 Project 打上不同的 `Tag` 标签，方便查找和管理。
+3. **Github/Gitlab 集成**：Submarine Workbench 与 `Github`/`Gitlab` 进行了系统集成，每个 Project 都可以在 Workbench 中进行 `Watch`、`Star`、`Frok` 和 `Comment` 操作。
+ + **Watch**：[TODO]
+ + **Star**：[TODO]
+ + **Frok**：[TODO]
+ + **Comment**：用户可以在项目中进行评论
+4. **Edit**：用户通过双击项目或者点击 `Edit` 按钮，可以在 `Notebook` 中打开项目，进行算法开发等操作。
+5. **Download**：用户通过点击 `Download` 按钮，将项目打包下载到本地。
+6. **Setting**：编辑项目信息，例如项目的名字，简介，分享级别和权限。
+7. **Delete**：删除项目中所有包含的文件。
+
+#### Add New Project
+
+在项目页面中点击 `Add New Project` 按钮，将会显示出创建项目的引导页面，只需要三个步骤就可以创建一个新的项目。
+
+第一步：在 **Base Information** 步骤中填写项目名称、项目简介。
+
+
+
++ **Visibility**: 设置项目对外的可见级别
+
+ + **Private**: （默认）设置为私有项目，不对外公开项目中包含的所有文件，但是可以在 **Notebook** 中将项目的执行结果单独设置公开，方便其他人查看项目的可视化报告。
+ + **Team**: 设置为团队项目，在团队选择框中选择团队的名称，团队的其他成员可以根据设置的权限访问这个项目。
+ + **Public**: 设置为公开项目，**Workbench** 中的所有用户都可以通过搜索查看到这个项目。
++ **Permission**: 设置项目对外的访问权限，只有将项目的 **Visibility** 设置为 **Team** 或 **Public** 的时候，才会出现权限设置界面。
+
+ + **Can View**
+
+ 当项目的 **Visibility** 设置为 **Team** 时，团队中其他成员都只能**查看**这个项目的文件。
+
+ 当项目的 Visibility 设置为 **Public** 时，**Workbench** 中其他成员都只能**查看**这个项目的文件。
+
+ + **Can Edit**
+
+ 当项目的 **Visibility** 设置为 **Team** 时，团队中其他成员都可以**查看**、**编辑**这个项目的文件。
+
+ 当项目的 **Visibility** 设置为 **Public** 时，**Workbench** 中其他成员都可以**查看**、**编辑**这个项目的文件。
+
+ + **Can Execute**
+
+ 当项目的 **Visibility** 设置为 **Team** 时，团队中其他成员都可以**查看**、**编辑**、**执行**这个项目的文件。
+
+ 当项目的 **Visibility** 设置为 **Public** 时，**Workbench** 中其他成员都可以**查看**、**编辑**、**执行**这个项目的文件。
+
+第二步：在 **Initial Project** 步骤中，**Workbench** 提供了四种项目初始化的方式
+
++ **Template**: **Workbench** 内置了几种不同开发语言和算法框架的项目模版，你可以选择任何一种模版初始化你的项目，无需做任何修改就可以直接在 **Notebook** 中执行，特别适合新手进行快速的体验。
+
+ 
+
++ **Blank**：创建一个空白的项目，稍后，我们可以通过在 **Notebook** 中手工添加项目的文件
+
+ 
+
++ **Upload**: 通过上传 **notebook** 格式的文件来初始化你的项目，**notebook** 格式兼容 **Jupyter Notebook** 和 **Zeppelin Notebook** 文件格式。
+
+ 
+
++ **Git Repo**: 在你的 **Github**/**Gitlab** 账号中 **Frok** 一个仓库中的文件内容来初始化项目。
+
+ 
+
+第三步：预览项目中的所包含的文件
+
+
+
++ **Save**: 将项目保存到 Workspace 中。
++ **Open In Notebook**: 将项目保存到 **Workspace** 中，并用 **Notebook** 打开项目。
+
+### Release
+
+[TODO]
+
+### Training
+
+[TODO]
+
+### Team
+
+[TODO]
+
+### Shared
+
+[TODO]
+
+## Interpreters
+
+[TODO]
+
+## Job
+
+[TODO]
+
+## Data
+
+[TODO]
+
+## Model
+
+[TODO]
+
+## Manager
+
+### User
+
+[TODO]
+
+### Team
+
+[TODO]
+
+### Data Dict
+
+[TODO]
+
+### Department
+
+[TODO]
+
+## How to run workbench
+
+[How To Run Submarine Workbench Guide](./HowToRun.md)
\ No newline at end of file
diff --git a/docs/workbench/assets/workspace-project-step1.png b/docs/workbench/assets/workspace-project-step1.png
new file mode 100644
index 0000000..c090e7c
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step1.png differ
diff --git a/docs/workbench/assets/workspace-project-step2-blank.png b/docs/workbench/assets/workspace-project-step2-blank.png
new file mode 100644
index 0000000..e6a0954
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step2-blank.png differ
diff --git a/docs/workbench/assets/workspace-project-step2-git.png b/docs/workbench/assets/workspace-project-step2-git.png
new file mode 100644
index 0000000..cf4730e
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step2-git.png differ
diff --git a/docs/workbench/assets/workspace-project-step2-template.png b/docs/workbench/assets/workspace-project-step2-template.png
new file mode 100644
index 0000000..f28d4cf
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step2-template.png differ
diff --git a/docs/workbench/assets/workspace-project-step2-upload.png b/docs/workbench/assets/workspace-project-step2-upload.png
new file mode 100644
index 0000000..c70bad1
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step2-upload.png differ
diff --git a/docs/workbench/assets/workspace-project-step3.png b/docs/workbench/assets/workspace-project-step3.png
new file mode 100644
index 0000000..f7be384
Binary files /dev/null and b/docs/workbench/assets/workspace-project-step3.png differ
diff --git a/docs/workbench/assets/workspace-project.png b/docs/workbench/assets/workspace-project.png
new file mode 100644
index 0000000..1e8cefa
Binary files /dev/null and b/docs/workbench/assets/workspace-project.png differ