You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by li...@apache.org on 2020/09/18 01:46:29 UTC
[submarine] branch master updated: SUBMARINE-622. Experiment
Template API docs
This is an automated email from the ASF dual-hosted git repository.
liuxun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new eef782d SUBMARINE-622. Experiment Template API docs
eef782d is described below
commit eef782de8387664b26e608975d5c9fe62176df34
Author: JohnTing <jo...@gmail.com>
AuthorDate: Tue Sep 15 00:09:31 2020 +0800
SUBMARINE-622. Experiment Template API docs
### What is this PR for?
Write a Experiment Template API docs to docs/userdocs/k8s/api/experiment-template.md
and docs/userdocs/k8s/run-experiment-template.md
### What type of PR is it?
Improvement
### Todos
* [x] - experiment-template.md
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-622
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: JohnTing <jo...@gmail.com>
Closes #399 from JohnTing/SUBMARINE-622 and squashes the following commits:
a26bbed [JohnTing] Update docs/userdocs/k8s/api/experiment-template.md
0370aef [JohnTing] Update docs/userdocs/k8s/api/experiment-template.md
b8b7ec1 [JohnTing] Update docs/userdocs/k8s/api/experiment-template.md
7de55da [JohnTing] Update docs/userdocs/k8s/api/experiment-template.md
403e44e [JohnTing] Update docs/userdocs/k8s/api/experiment-template.md
21cb8dd [JohnTing] change
61703fa [JohnTing] add experiment template docs
---
.../experiment-template.md} | 126 +++++++++++++++------
docs/userdocs/k8s/run-experiment-template.md | 43 +++++++
2 files changed, 135 insertions(+), 34 deletions(-)
diff --git a/docs/userdocs/k8s/use-experiment-template.md b/docs/userdocs/k8s/api/experiment-template.md
similarity index 50%
rename from docs/userdocs/k8s/use-experiment-template.md
rename to docs/userdocs/k8s/api/experiment-template.md
index 732b29a..40afe03 100644
--- a/docs/userdocs/k8s/use-experiment-template.md
+++ b/docs/userdocs/k8s/api/experiment-template.md
@@ -17,34 +17,60 @@ specific language governing permissions and limitations
under the License.
-->
-# Use Experiment Template Guide
+# Experiment Template API Reference
-The {{name}} variable in "experimentSpec" will be replace by the parameters value.
+> Note: The Experiment API is in the alpha stage which is subjected to incompatible changes in
+> future releases.
-JSON Format example:
+
+Developers can register a parameterized experiment as an experiment template,
+For example, change --learning_rate=0.1 to --learning_rate={{training.learning_rate}}, and add the following in the parameters
```json
{
- "name": "tf-mnist-test",
+ "name": "training.learning_rate",
+ "value": 0.1,
+ "required": true,
+ "description": "This is learning_rate of training."
+}
+```
+So users can use existing experiment templates and adjust the default value to create experiments.
+After the user submits the experiment template, the submarine server finds the corresponding template based on the name. And the template handler converts input parameters to an actual experiment, such as a distributed TF experiment.
+
+
+
+## Create experiment template
+`POST /api/v1/template`
+
+**Example Request**
+```sh
+curl -X POST -H "Content-Type: application/json" -d '
+{
+ "name": "my-tf-mnist-template",
"author": "author",
"description": "This is a template to run tf-mnist",
- "parameters": [
- {
+ "parameters": [{
"name": "training.learning_rate",
"value": 0.1,
"required": true,
- "description": " mnist learning_rate "
+ "description": "This is learning_rate of training."
},
{
"name": "training.batch_size",
"value": 150,
- "required": false,
- "description": "This is batch size of training"
+ "required": true,
+ "description": "This is batch_size of training."
+ },
+ {
+ "name": "experiment.name",
+ "value": "tf-mnist1",
+ "required": true,
+ "description": "the name of experiment."
}
],
"experimentSpec": {
"meta": {
"cmd": "python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate={{training.learning_rate}} --batch_size={{training.batch_size}}",
- "name": "tf-mnist-template-test",
+ "name": "{{experiment.name}}",
"envVars": {
"ENV1": "ENV1"
},
@@ -66,33 +92,58 @@ JSON Format example:
}
}
}
+' http://127.0.0.1:8080/api/v1/template
+```
+
+
+### List experiment template
+`GET /api/v1/template`
+
+**Example Request:**
+```sh
+curl -X GET http://127.0.0.1:8080/api/v1/template
```
-### Register experiment template
+### Get experiment template
+`GET /api/v1/template/{name}`
+
+**Example Request:**
```sh
-curl -X POST -H "Content-Type: application/json" -d '
+curl -X GET http://127.0.0.1:8080/api/v1/template/my-tf-mnist-template
+```
+
+
+### Patch template
+`PATCH /api/v1/template/{name}`
+```sh
+curl -X PATCH -H "Content-Type: application/json" -d '
{
- "name": "tf-mnist-test",
- "author": "author",
+ "name": "my-tf-mnist-template",
+ "author": "author-new",
"description": "This is a template to run tf-mnist",
- "parameters": [
- {
+ "parameters": [{
"name": "training.learning_rate",
"value": 0.1,
"required": true,
- "description": " mnist learning_rate "
+ "description": "This is learning_rate of training."
},
{
"name": "training.batch_size",
"value": 150,
- "required": false,
- "description": "This is batch size of training"
+ "required": true,
+ "description": "This is batch_size of training."
+ },
+ {
+ "name": "experiment.name",
+ "value": "tf-mnist1",
+ "required": true,
+ "description": "the name of experiment."
}
],
"experimentSpec": {
"meta": {
"cmd": "python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate={{training.learning_rate}} --batch_size={{training.batch_size}}",
- "name": "tf-mnist-template-test",
+ "name": "{{experiment.name}}",
"envVars": {
"ENV1": "ENV1"
},
@@ -114,29 +165,36 @@ curl -X POST -H "Content-Type: application/json" -d '
}
}
}
-' http://127.0.0.1:8080/api/v1/template
+' http://127.0.0.1:8080/api/v1/template/my-tf-mnist-template
```
-JSON Format example:
-```json
-{
- "name": "tf-mnist-test",
- "params": {
- "training.learning_rate":"0.01",
- "training.batch_size":"150"
- }
-}
+> "description", "parameters", "experimentSpec", "author" etc can be updated using this API.
+"name" of experiment template is not supported.
+
+
+
+### Delete template
+`GET /api/v1/template/{name}`
+
+**Example Request:**
+```sh
+curl -X DELETE http://127.0.0.1:8080/api/v1/template/my-tf-mnist-template
```
-### Submit experiment template
+
+### Use template to create a experiment
+`POST /api/v1/experiment/{template_name}`
+
+**Example Request:**
```sh
curl -X POST -H "Content-Type: application/json" -d '
{
- "name": "tf-mnist-test",
+ "name": "tf-mnist",
"params": {
"training.learning_rate":"0.01",
- "training.batch_size":"150"
+ "training.batch_size":"150",
+ "experiment.name":"newExperiment"
}
}
-' http://127.0.0.1:8080/api/v1/template/submit
+' http://127.0.0.1:8080/api/v1/experiment/my-tf-mnist-template
```
diff --git a/docs/userdocs/k8s/run-experiment-template.md b/docs/userdocs/k8s/run-experiment-template.md
new file mode 100644
index 0000000..dcae977
--- /dev/null
+++ b/docs/userdocs/k8s/run-experiment-template.md
@@ -0,0 +1,43 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Experiment Template API Reference
+
+## Experiment Template Spec
+The experiment is represented in [JSON](https://www.json.org) or [YAML](https://yaml.org) format.
+
+
+### Use existing experiment template to create a experiment
+`POST /api/v1/environment/{name}`
+
+**Example Request:**
+```sh
+curl -X POST -H "Content-Type: application/json" -d '
+{
+ "name": "tf-mnist",
+ "params": {
+ "training.learning_rate":"0.01",
+ "training.batch_size":"150",
+ "experiment.name":"newExperiment"
+ }
+}
+' http://127.0.0.1:8080/api/v1/experiment/my-tf-mnist-template
+```
+
+Register experiment template and more info see [Experiment Template API Reference](api/experiment-template.md).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org