You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2020/06/01 01:24:04 UTC
[GitHub] [submarine] jiwq commented on a change in pull request #303: SUBMARINE-519. [WIP] refactor of submarine documentation
jiwq commented on a change in pull request #303:
URL: https://github.com/apache/submarine/pull/303#discussion_r433003569
##########
File path: docs/user-guide-home.md
##########
@@ -0,0 +1,34 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# User Document of Submarine
+
+This is index of user-document of Submarine.
+
+## Build From Code
+
+[Build From Code Guide](./development/BuildFromCode.md)
+
+FIXME: Where's build guide for K8s? How can we make it clear?
+
+## Use Submarine on YARN
+
+Please refer to [Submarine On YARN](userdocs/yarn/README.md)
+
+## Use Submarine on K8s
+
+Please refer to [Submarine On K8s]()
+
+## Submarine Server
+[Submarine Server Guide](project/github/submarine/docs/userdocs/k8s/run-tensorflow-on-k8s.md)
Review comment:
The path of the doc cannot found in Github, so we should fix it.
##########
File path: docs/user-guide-home.md
##########
@@ -0,0 +1,34 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# User Document of Submarine
+
+This is index of user-document of Submarine.
+
+## Build From Code
+
+[Build From Code Guide](./development/BuildFromCode.md)
+
+FIXME: Where's build guide for K8s? How can we make it clear?
+
+## Use Submarine on YARN
+
+Please refer to [Submarine On YARN](userdocs/yarn/README.md)
+
+## Use Submarine on K8s
+
+Please refer to [Submarine On K8s]()
Review comment:
I think we should link the doc like `YARN` even it not ready now.
```suggestion
Please refer to [Submarine On K8s](userdocs/k8s/)
```
##########
File path: docs/userdocs/k8s/deploy-submarine.md
##########
@@ -0,0 +1,70 @@
+
+
+# Deploy Submarine On K8s
+
+## Deploy Submarine using Helm Chart (Recommended)
+
+### Create images
+submarine server
+```bash
+./dev-support/docker-images/submarine/build.sh
+```
+
+submarine database
+```bash
+./dev-support/docker-images/database/build.sh
+```
+
+### install helm
+For more info see https://helm.sh/docs/intro/install/
+
+### Deploy Submarine Server, mysql
+You can modify some settings in ./helm-charts/submarine/values.yaml
+```bash
+helm install submarine ./helm-charts/submarine
+```
+
+### Delete deployment
+```bash
+helm delete submarine
+```
+
+FIXME: TF Operator / PyTorch Operator can be deployed by Helm Chart, correct?
+
+## Deploy Submarine Manually
+
+### Get package
+You can dowload submarine from releases page or build from source.
+
+### Configuration
+`KUBE_CONFIG` environment need to be specified when deploy Submarine server.
+
+### Start Submarine Server
+Running the submarine server, executing the following command:
+```
+# if build from source. You need to run this under the target dir like submarine-dist/target/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/
+./bin/submarine-daemon.sh start getMysqlJar
+```
+
+The REST API URL is: `http://127.0.0.1:8080/api/v1/jobs`
+
+### Deploy Tensorflow Operator
+For more info see [deploy tensorflow operator](./ml-frameworks/tensorflow.md).
+
+### Deploy PyTorch Operator
+```bash
+cd <submarine_code_path_root>/dev-support/k8s/pytorchjob
+./deploy-pytorch-operator.sh
+
+```
+
+## Post Deploy
+
+### Setup port-forward {host port}:{container port}
+
+FIXME: Why we need this? Need elaborate
Review comment:
I think it only used in manual deployment to export the submarine service outside the K8s cluster, and we should support to automatically operate in helm charts.
##########
File path: docs/userdocs/k8s/deploy-submarine.md
##########
@@ -0,0 +1,70 @@
+
+
+# Deploy Submarine On K8s
+
+## Deploy Submarine using Helm Chart (Recommended)
+
+### Create images
+submarine server
+```bash
+./dev-support/docker-images/submarine/build.sh
+```
+
+submarine database
+```bash
+./dev-support/docker-images/database/build.sh
+```
+
+### install helm
+For more info see https://helm.sh/docs/intro/install/
+
+### Deploy Submarine Server, mysql
+You can modify some settings in ./helm-charts/submarine/values.yaml
+```bash
+helm install submarine ./helm-charts/submarine
+```
+
+### Delete deployment
+```bash
+helm delete submarine
+```
+
+FIXME: TF Operator / PyTorch Operator can be deployed by Helm Chart, correct?
Review comment:
Yes, it can be deployed automatically when user use helm install submarine.
##########
File path: docs/userdocs/k8s/deploy-submarine.md
##########
@@ -0,0 +1,70 @@
+
Review comment:
`Apache License` missing.
##########
File path: docs/userdocs/k8s/deploy-submarine.md
##########
@@ -0,0 +1,70 @@
+
+
+# Deploy Submarine On K8s
+
+## Deploy Submarine using Helm Chart (Recommended)
+
+### Create images
+submarine server
+```bash
+./dev-support/docker-images/submarine/build.sh
+```
+
+submarine database
+```bash
+./dev-support/docker-images/database/build.sh
+```
+
+### install helm
+For more info see https://helm.sh/docs/intro/install/
+
+### Deploy Submarine Server, mysql
+You can modify some settings in ./helm-charts/submarine/values.yaml
+```bash
+helm install submarine ./helm-charts/submarine
+```
+
+### Delete deployment
+```bash
+helm delete submarine
+```
+
+FIXME: TF Operator / PyTorch Operator can be deployed by Helm Chart, correct?
+
+## Deploy Submarine Manually
Review comment:
In the pre-version deploy by manual it aims to standalone and supports more K8s cluster in one submarine server. But in here, my understanding is manually deploy submarine in K8s with in-cluster mode.
##########
File path: docs/userdocs/k8s/deploy-submarine.md
##########
@@ -0,0 +1,70 @@
+
+
+# Deploy Submarine On K8s
+
+## Deploy Submarine using Helm Chart (Recommended)
+
+### Create images
+submarine server
+```bash
+./dev-support/docker-images/submarine/build.sh
+```
+
+submarine database
+```bash
+./dev-support/docker-images/database/build.sh
+```
Review comment:
If the user select to build docker image by manual, he/her should to modify the `values.yaml` and specify the correctly image in helm charts.
##########
File path: docs/userdocs/k8s/README.md
##########
@@ -0,0 +1,41 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Running Submarine on K8s
+
+Submarine for K8s supports standalone distributed TensorFlow and PyTorch.
+
+Submarine can run on K8s >= (FIXME, version), supports features like GPU isolation.
+
+## Submarine on K8s guide
+
+### Prepare K8s and deploy Submarine Service
+
+[Setup Kubernetes](setup-kubernetes.md): Submarine can be deployed on any K8s environment if version matches. If you don't have a running K8s, you can follow the steps to set up a K8s using [kind, Kubernetes-in-Docker](https://kind.sigs.k8s.io/) for testing purpose.
+
+After you have an up-and-running K8s, you can follow [Deploy Submarine Services on K8s](deploy-submarine.md) guide to deploy Submarine services on K8s using Helmchart in minutes (FIXME: is it true?).
+
+### Use Submarine
+
+#### Model training (experiment) on K8s
+
+- [Run model training using Tensorflow](run-tensorflow-on-k8s.md)
+- [Run model training using PyTorch](FIXME, add one).
Review comment:
I found the PyTorch training is also in `run-tensorflow-on-k8s.md`, so we can split it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org