You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by zt...@apache.org on 2020/05/25 08:48:59 UTC
[submarine] branch master updated: SUBMARINE-506. [SDK] Generate
experiment Python SDK
This is an automated email from the ASF dual-hosted git repository.
ztang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 931e147 SUBMARINE-506. [SDK] Generate experiment Python SDK
931e147 is described below
commit 931e147c98d8796fad83e3becd7324672ced41e6
Author: pingsutw <pi...@gmail.com>
AuthorDate: Sat May 23 07:29:54 2020 +0800
SUBMARINE-506. [SDK] Generate experiment Python SDK
### What is this PR for?
Generate Experiment Python SDK
- Generated Experiment Python SDK guide [README.md](https://github.com/pingsutw/hadoop-submarine/blob/SUBMARINE-506/submarine-sdk/pysubmarine/submarine/job/README.md)
- Created a sample for the SDK usage. [submarine_job_sdk.ipynb](https://github.com/pingsutw/hadoop-submarine/blob/SUBMARINE-506/submarine-sdk/pysubmarine/example/submarine_job_sdk.ipynb)
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-506
### How should this be tested?
https://github.com/pingsutw/hadoop-submarine/actions/runs/111931885
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: pingsutw <pi...@gmail.com>
Closes #296 from pingsutw/SUBMARINE-506 and squashes the following commits:
df2391e [pingsutw] update
5a7d524 [pingsutw] update doc
5d76b94 [pingsutw] update
cd32832 [pingsutw] Add openapi.yaml
82ef19d [pingsutw] SUBMARINE-506. [SDK] Generate experiment Python SDK
---
docs/submarine-sdk/pysubmarine/README.md | 31 +-
docs/submarine-sdk/pysubmarine/generate_api.md | 31 +
docs/submarine-sdk/pysubmarine/openapi.yaml | 244 +++++++
docs/submarine-sdk/pysubmarine/tracking.md | 3 +
pom.xml | 1 +
submarine-sdk/pysubmarine/example/client/README.md | 36 -
.../pysubmarine/example/client/mnist.json | 31 -
.../pysubmarine/example/submarine_job_sdk.ipynb | 608 ++++++++++++++++
submarine-sdk/pysubmarine/github-actions/lint.sh | 2 +-
.../github-actions/test-requirements.txt | 8 +-
submarine-sdk/pysubmarine/setup.py | 8 +-
submarine-sdk/pysubmarine/submarine/__init__.py | 17 +-
.../pysubmarine/submarine/job/__init__.py | 26 +-
.../submarine/job/{ => api}/__init__.py | 7 +-
.../pysubmarine/submarine/job/api/jobs_api.py | 786 +++++++++++++++++++++
.../pysubmarine/submarine/job/api_client.py | 643 +++++++++++++++++
.../pysubmarine/submarine/job/configuration.py | 259 +++++++
.../pysubmarine/submarine/job/models/__init__.py | 35 +
.../submarine/job/models/job_library_spec.py | 230 ++++++
.../pysubmarine/submarine/job/models/job_spec.py | 230 ++++++
.../submarine/job/models/job_task_spec.py | 334 +++++++++
.../submarine/job/models/json_response.py | 204 ++++++
submarine-sdk/pysubmarine/submarine/job/rest.py | 337 +++++++++
.../submarine/job/submarine_job_client.py | 52 --
.../pysubmarine/tests/client/test_client.py | 56 --
25 files changed, 4024 insertions(+), 195 deletions(-)
diff --git a/docs/submarine-sdk/pysubmarine/README.md b/docs/submarine-sdk/pysubmarine/README.md
index 59fe9f6..ac872a0 100644
--- a/docs/submarine-sdk/pysubmarine/README.md
+++ b/docs/submarine-sdk/pysubmarine/README.md
@@ -10,25 +10,26 @@
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--->
+-->
# PySubmarine
-PySubmarine helps developers use submarine's internal data caching,
-data exchange, and task tracking capabilities to more efficiently improve the
-development and execution of machine learning productivity.
+PySubmarine is aiming to ease the ML engineer's life by providing a set of libraries.
+
+It includes a high-level out-of-box ML library like deepFM, FM, etc.
+low-level library to interact with submarine like creating experiment,
+tracking experiment metrics, parameters.
+
## Package setup
-- Clone repo
+- Clone repository
```bash
git clone https://github.com/apache/submarine.git
cd submarine/submarine-sdk/pysubmarine
```
-
- Install pip package
```bash
pip install .
```
-
- Run tests
```bash
pytest --cov=submarine -vs
@@ -36,10 +37,18 @@ pytest --cov=submarine -vs
- Run checkstyle
```bash
-pylint --msg-template="{path} ({line},{column}): \
-[{msg_id} {symbol}] {msg}" --rcfile=pylintrc -- submarine tests
+./submarine-sdk/pysubmarine/github-actions/lint.sh
```
+## How to generate REST SDK from swagger
+- [Generate REST SDK from swagger](./generate_api.md)
+
+## Easy-to-use model trainers
+- [FM](../../../submarine-sdk/pysubmarine/example/deepfm)
+- [DeepFM](../../../submarine-sdk/pysubmarine/example/fm)
+
+## Submarine experiment management
+Makes it easy to run distributed or non-distributed TensorFlow, PyTorch experiments on Kubernetes.
+- [mnist example](../../../submarine-sdk/pysubmarine/example/submarine_job_sdk.ipynb)
## PySubmarine API Reference
-### Tracking
-- [Tracking](tracking.md)
\ No newline at end of file
+- [Tracking](tracking.md)
diff --git a/docs/submarine-sdk/pysubmarine/generate_api.md b/docs/submarine-sdk/pysubmarine/generate_api.md
new file mode 100644
index 0000000..c331a42
--- /dev/null
+++ b/docs/submarine-sdk/pysubmarine/generate_api.md
@@ -0,0 +1,31 @@
+<!---
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# How to generate Experiment API
+
+### Build Submarine project
+- git clone git@github.com:apache/submarine.git
+- mvn clean package -DskipTests
+### Start Submarine server
+- cd submarine-dist/target/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/submarine-dist-0.4.0-SNAPSHOT-hadoop-2.9/
+- ./bin/submarine-daemon.sh start getMysqlJar
+### Generate experiment API
+- open localhost:8080/v1/openapi.json
+- copy `openapi.json` file to [swagger editor](https://editor.swagger.io/)
+- click *generate client* -> *python* to generate experiment API archive
+### Add experiment API to pysubmarine
+- mv `./python-client-generated/swagger-client/*` to `pysubmarine/submarine/job`
+- rename all `swagger_client` in `submarine/job/*.py` to `submarine.job`.
+ - e.g. `from swagger_client.models.job_task_spec import JobTaskSpec` -> `from submarine.job.models.job_task_spec import JobTaskSpec`
+- import experiment API in [\_\_init__.py](../__init__.py)
diff --git a/docs/submarine-sdk/pysubmarine/openapi.yaml b/docs/submarine-sdk/pysubmarine/openapi.yaml
new file mode 100644
index 0000000..d9264eb
--- /dev/null
+++ b/docs/submarine-sdk/pysubmarine/openapi.yaml
@@ -0,0 +1,244 @@
+openapi: 3.0.1
+info:
+ title: Submarine Experiment API
+ description: 'The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/'
+ termsOfService: 'http://swagger.io/terms/'
+ contact:
+ email: submarine-dev@submarine.apache.org
+ license:
+ name: Apache 2.0
+ url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
+ version: 0.4.0-SNAPSHOT
+servers:
+ - url: /api
+paths:
+ /v1/jobs:
+ get:
+ tags:
+ - jobs
+ summary: List jobs
+ operationId: listJob
+ parameters:
+ - name: status
+ in: query
+ schema:
+ type: string
+ responses:
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ post:
+ tags:
+ - jobs
+ summary: Create a job
+ operationId: createJob
+ requestBody:
+ content:
+ application/yaml:
+ schema:
+ $ref: '#/components/schemas/JobSpec'
+ application/json:
+ schema:
+ $ref: '#/components/schemas/JobSpec'
+ responses:
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ '/v1/jobs/{id}':
+ get:
+ tags:
+ - jobs
+ summary: Find job by id
+ operationId: getJob
+ parameters:
+ - name: id
+ in: path
+ required: true
+ schema:
+ type: string
+ responses:
+ '404':
+ description: Job not found
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ delete:
+ tags:
+ - jobs
+ summary: Delete the job
+ operationId: deleteJob
+ parameters:
+ - name: id
+ in: path
+ required: true
+ schema:
+ type: string
+ responses:
+ '404':
+ description: Job not found
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ patch:
+ tags:
+ - jobs
+ summary: Update the job in the submarine server with job spec
+ operationId: patchJob
+ parameters:
+ - name: id
+ in: path
+ required: true
+ schema:
+ type: string
+ requestBody:
+ content:
+ application/yaml:
+ schema:
+ $ref: '#/components/schemas/JobSpec'
+ application/json:
+ schema:
+ $ref: '#/components/schemas/JobSpec'
+ responses:
+ '404':
+ description: Job not found
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ /v1/jobs/logs:
+ get:
+ tags:
+ - jobs
+ summary: Log jobs
+ operationId: listLog
+ parameters:
+ - name: status
+ in: query
+ schema:
+ type: string
+ responses:
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ '/v1/jobs/logs/{id}':
+ get:
+ tags:
+ - jobs
+ summary: Log job by id
+ operationId: getLog
+ parameters:
+ - name: id
+ in: path
+ required: true
+ schema:
+ type: string
+ responses:
+ '404':
+ description: Job not found
+ default:
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ $ref: '#/components/schemas/JsonResponse'
+ /v1/jobs/ping:
+ get:
+ tags:
+ - jobs
+ summary: Ping submarine server
+ description: Return the Pong message for test the connectivity
+ operationId: ping
+ responses:
+ '200':
+ description: successful operation
+ content:
+ application/json; charset=utf-8:
+ schema:
+ type: string
+components:
+ schemas:
+ JsonResponse:
+ type: object
+ properties:
+ code:
+ type: integer
+ format: int32
+ success:
+ type: boolean
+ result:
+ type: object
+ attributes:
+ type: object
+ additionalProperties:
+ type: object
+ JobLibrarySpec:
+ type: object
+ properties:
+ name:
+ type: string
+ version:
+ type: string
+ image:
+ type: string
+ cmd:
+ type: string
+ envVars:
+ type: object
+ additionalProperties:
+ type: string
+ JobSpec:
+ type: object
+ properties:
+ name:
+ type: string
+ namespace:
+ type: string
+ librarySpec:
+ $ref: '#/components/schemas/JobLibrarySpec'
+ taskSpecs:
+ type: object
+ additionalProperties:
+ $ref: '#/components/schemas/JobTaskSpec'
+ projects:
+ type: string
+ JobTaskSpec:
+ type: object
+ properties:
+ name:
+ type: string
+ image:
+ type: string
+ cmd:
+ type: string
+ envVars:
+ type: object
+ additionalProperties:
+ type: string
+ resources:
+ type: string
+ replicas:
+ type: integer
+ format: int32
+ cpu:
+ type: string
+ gpu:
+ type: string
+ memory:
+ type: string
diff --git a/docs/submarine-sdk/pysubmarine/tracking.md b/docs/submarine-sdk/pysubmarine/tracking.md
index 09a5773..3697b93 100644
--- a/docs/submarine-sdk/pysubmarine/tracking.md
+++ b/docs/submarine-sdk/pysubmarine/tracking.md
@@ -13,6 +13,9 @@
-->
# pysubmarine-tracking
+It helps developers use submarine's internal data caching,
+data exchange, and task tracking capabilities to more efficiently improve the
+development and execution of machine learning productivity
- Allow data scientist to track distributed ML job
- Support store ML parameters and metrics in Submarine-server
- [TODO] Support store ML job output (e.g. csv,images)
diff --git a/pom.xml b/pom.xml
index f7aa8ed..7d135ca 100644
--- a/pom.xml
+++ b/pom.xml
@@ -565,6 +565,7 @@
<exclude>**/go.mod</exclude>
<exclude>**/go.sum</exclude>
<exclude>**/workbench-web/**</exclude>
+ <exclude>**/*.ipynb</exclude>
</excludes>
</configuration>
</plugin>
diff --git a/submarine-sdk/pysubmarine/example/client/README.md b/submarine-sdk/pysubmarine/example/client/README.md
deleted file mode 100644
index bdd5d7b..0000000
--- a/submarine-sdk/pysubmarine/example/client/README.md
+++ /dev/null
@@ -1,36 +0,0 @@
-<!---
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-# Submarine Client API Example
-This example shows how you could use Submarine Client API to
-manage submarine tasks
-
-## Prerequisites
-- [Deploy Submarine Server on Kubernetes](https://github.com/apache/submarine/blob/master/docs/submarine-server/setup-kubernetes.md)
-- [Deploy Tensorflow Operator on Kubernetes](https://github.com/apache/submarine/blob/master/docs/submarine-server/ml-frameworks/tensorflow.md)
-
-#### Submit Job
-1. Create a job description for submarine client. e.g.[mnist.json](./mnist.json)
-
-2. Create Submarine job client
-```python
-from submarine.job import SubmarineJobClient
-client = SubmarineJobClient('localhost', 8080)
-```
-3. Submit job
-```python
-response = client.submit_job('mnist.json')
-```
-#### Delete job
-```python
-response = client.delete_job('job_1586791302310_0005')
-```
diff --git a/submarine-sdk/pysubmarine/example/client/mnist.json b/submarine-sdk/pysubmarine/example/client/mnist.json
deleted file mode 100644
index d84c966..0000000
--- a/submarine-sdk/pysubmarine/example/client/mnist.json
+++ /dev/null
@@ -1,31 +0,0 @@
-{
- "name": "mnist",
- "librarySpec": {
- "name": "TensorFlow",
- "version": "2.1.0",
- "image": "gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0",
- "cmd": "python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate=0.01 --batch_size=150",
- "envVars": {
- "ENV_1": "ENV1"
- }
- },
- "submitterSpec": {
- "type": "k8s",
- "configPath": null,
- "namespace": "submarine",
- "kind": "TFJob",
- "apiVersion": "kubeflow.org/v1"
- },
- "taskSpecs": {
- "Ps": {
- "name": "tensorflow",
- "replicas": 1,
- "resources": "cpu=1,memory=1024M"
- },
- "Worker": {
- "name": "tensorflow",
- "replicas": 1,
- "resources": "cpu=1,memory=1024M"
- }
- }
-}
diff --git a/submarine-sdk/pysubmarine/example/submarine_job_sdk.ipynb b/submarine-sdk/pysubmarine/example/submarine_job_sdk.ipynb
new file mode 100644
index 0000000..9ff9550
--- /dev/null
+++ b/submarine-sdk/pysubmarine/example/submarine_job_sdk.ipynb
@@ -0,0 +1,608 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Sample for Submarine Experiment SDK\n",
+ "\n",
+ "The notebook shows how to use Subamrine Experiment SDK to create, get, list, log, delete Submarine Experiment."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "pycharm": {
+ "is_executing": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "from __future__ import print_function\n",
+ "import submarine\n",
+ "\n",
+ "from submarine import JobLibrarySpec\n",
+ "from submarine import JobTaskSpec\n",
+ "from submarine import JobSpec"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Create Submarine Client"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "pycharm": {
+ "is_executing": false,
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "configuration = submarine.Configuration()\n",
+ "configuration.host = 'http://192.168.103.9:8080/api'\n",
+ "api_client = submarine.ApiClient(configuration=configuration)\n",
+ "submarine_client = submarine.JobsApi(api_client=api_client)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "pycharm": {
+ "name": "#%% md\n"
+ }
+ },
+ "source": [
+ "### Define TFJob¶\n",
+ "Define Submarine spec¶\n",
+ "The demo only creates a worker of TFJob to run mnist sample."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {
+ "pycharm": {
+ "is_executing": false,
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [],
+ "source": [
+ "jobLibrarySpec = JobLibrarySpec(name='tensorflow',\n",
+ " version='2.1.0',\n",
+ " image='gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0',\n",
+ " cmd='python /var/tf_mnist/mnist_with_summaries.py'\n",
+ " ' --log_dir=/train/log --learning_rate=0.01'\n",
+ " ' --batch_size=150',\n",
+ " env_vars={'ENV1': 'ENV1'})\n",
+ "\n",
+ "worker = JobTaskSpec(name='tensorlfow',\n",
+ " image=None,\n",
+ " cmd=None,\n",
+ " env_vars=None,\n",
+ " resources='cpu=4,memory=2048M',\n",
+ " replicas=1)\n",
+ "\n",
+ "body = JobSpec(name='mnist',\n",
+ " namespace='submarine',\n",
+ " library_spec=jobLibrarySpec,\n",
+ " task_specs={\"Worker\": worker})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Create experiment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {
+ "pycharm": {
+ "name": "#%%\n"
+ },
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'attributes': {},\n",
+ " 'code': 200,\n",
+ " 'result': {'acceptedTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'createdTime': None,\n",
+ " 'finishedTime': None,\n",
+ " 'jobId': 'job_1590056548552_0001',\n",
+ " 'name': 'mnist',\n",
+ " 'runningTime': None,\n",
+ " 'spec': {'librarySpec': {'cmd': 'python '\n",
+ " '/var/tf_mnist/mnist_with_summaries.py '\n",
+ " '--log_dir=/train/log '\n",
+ " '--learning_rate=0.01 '\n",
+ " '--batch_size=150',\n",
+ " 'envVars': {'ENV1': 'ENV1'},\n",
+ " 'image': 'gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0',\n",
+ " 'name': 'tensorflow',\n",
+ " 'version': '2.1.0'},\n",
+ " 'name': 'mnist',\n",
+ " 'namespace': 'submarine',\n",
+ " 'projects': None,\n",
+ " 'taskSpecs': {'Worker': {'cmd': None,\n",
+ " 'envVars': None,\n",
+ " 'image': None,\n",
+ " 'name': 'tensorlfow',\n",
+ " 'replicas': 1,\n",
+ " 'resourceMap': {'cpu': '4',\n",
+ " 'memory': '2048M'},\n",
+ " 'resources': 'cpu=4,memory=2048M'}}},\n",
+ " 'status': 'Accepted',\n",
+ " 'uid': '10b226b9-3710-40e9-9f13-9322e03dcb8d'},\n",
+ " 'success': True}"
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "submarine_client.create_job(body=body)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Get the created experiment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "pycharm": {
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'attributes': {},\n",
+ " 'code': 200,\n",
+ " 'result': {'acceptedTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'createdTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'finishedTime': None,\n",
+ " 'jobId': 'job_1590056548552_0001',\n",
+ " 'name': 'mnist',\n",
+ " 'runningTime': '2020-05-21T18:23:11.000+08:00',\n",
+ " 'spec': {'librarySpec': {'cmd': 'python '\n",
+ " '/var/tf_mnist/mnist_with_summaries.py '\n",
+ " '--log_dir=/train/log '\n",
+ " '--learning_rate=0.01 '\n",
+ " '--batch_size=150',\n",
+ " 'envVars': {'ENV1': 'ENV1'},\n",
+ " 'image': 'gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0',\n",
+ " 'name': 'tensorflow',\n",
+ " 'version': '2.1.0'},\n",
+ " 'name': 'mnist',\n",
+ " 'namespace': 'submarine',\n",
+ " 'projects': None,\n",
+ " 'taskSpecs': {'Worker': {'cmd': None,\n",
+ " 'envVars': None,\n",
+ " 'image': None,\n",
+ " 'name': 'tensorlfow',\n",
+ " 'replicas': 1,\n",
+ " 'resourceMap': {'cpu': '4',\n",
+ " 'memory': '2048M'},\n",
+ " 'resources': 'cpu=4,memory=2048M'}}},\n",
+ " 'status': 'Running',\n",
+ " 'uid': '10b226b9-3710-40e9-9f13-9322e03dcb8d'},\n",
+ " 'success': True}"
+ ]
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "id = 'job_1590056548552_0001'\n",
+ "submarine_client.get_job(id)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### List all running experiments"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {
+ "pycharm": {
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'attributes': {},\n",
+ " 'code': 200,\n",
+ " 'result': [{'acceptedTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'createdTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'finishedTime': None,\n",
+ " 'jobId': 'job_1590056548552_0001',\n",
+ " 'name': 'mnist',\n",
+ " 'runningTime': '2020-05-21T18:23:11.000+08:00',\n",
+ " 'spec': {'librarySpec': {'cmd': 'python '\n",
+ " '/var/tf_mnist/mnist_with_summaries.py '\n",
+ " '--log_dir=/train/log '\n",
+ " '--learning_rate=0.01 '\n",
+ " '--batch_size=150',\n",
+ " 'envVars': {'ENV1': 'ENV1'},\n",
+ " 'image': 'gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0',\n",
+ " 'name': 'tensorflow',\n",
+ " 'version': '2.1.0'},\n",
+ " 'name': 'mnist',\n",
+ " 'namespace': 'submarine',\n",
+ " 'projects': None,\n",
+ " 'taskSpecs': {'Worker': {'cmd': None,\n",
+ " 'envVars': None,\n",
+ " 'image': None,\n",
+ " 'name': 'tensorlfow',\n",
+ " 'replicas': 1,\n",
+ " 'resourceMap': {'cpu': '4',\n",
+ " 'memory': '2048M'},\n",
+ " 'resources': 'cpu=4,memory=2048M'}}},\n",
+ " 'status': 'Running',\n",
+ " 'uid': '10b226b9-3710-40e9-9f13-9322e03dcb8d'}],\n",
+ " 'success': True}"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "status = 'running'\n",
+ "submarine_client.list_job(status=status)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Get specific experiment training log "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {
+ "pycharm": {
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'attributes': {},\n",
+ " 'code': 200,\n",
+ " 'result': {'jobId': 'job_1590056548552_0001',\n",
+ " 'logContent': [{'podLog': 'WARNING:tensorflow:From '\n",
+ " '/var/tf_mnist/mnist_with_summaries.py:39: '\n",
+ " 'read_data_sets (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.mnist) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please use alternatives such as '\n",
+ " 'official/mnist/dataset.py from '\n",
+ " 'tensorflow/models.\\n'\n",
+ " 'WARNING:tensorflow:From '\n",
+ " '/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: '\n",
+ " 'maybe_download (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.base) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please write your own downloading '\n",
+ " 'logic.\\n'\n",
+ " 'WARNING:tensorflow:From '\n",
+ " '/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:252: '\n",
+ " 'wrapped_fn (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.base) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please use urllib or similar directly.\\n'\n",
+ " 'WARNING:tensorflow:From '\n",
+ " '/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: '\n",
+ " 'extract_images (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.mnist) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please use tf.data to implement this '\n",
+ " 'functionality.\\n'\n",
+ " 'WARNING:tensorflow:From '\n",
+ " '/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: '\n",
+ " 'extract_labels (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.mnist) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please use tf.data to implement this '\n",
+ " 'functionality.\\n'\n",
+ " 'WARNING:tensorflow:From '\n",
+ " '/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: '\n",
+ " '__init__ (from '\n",
+ " 'tensorflow.contrib.learn.python.learn.datasets.mnist) '\n",
+ " 'is deprecated and will be removed in a '\n",
+ " 'future version.\\n'\n",
+ " 'Instructions for updating:\\n'\n",
+ " 'Please use alternatives such as '\n",
+ " 'official/mnist/dataset.py from '\n",
+ " 'tensorflow/models.\\n'\n",
+ " '2020-05-21 10:23:16.050386: I '\n",
+ " 'tensorflow/core/platform/cpu_feature_guard.cc:141] '\n",
+ " 'Your CPU supports instructions that '\n",
+ " 'this TensorFlow binary was not compiled '\n",
+ " 'to use: AVX2 FMA\\n'\n",
+ " 'Successfully downloaded '\n",
+ " 'train-images-idx3-ubyte.gz 9912422 '\n",
+ " 'bytes.\\n'\n",
+ " 'Extracting '\n",
+ " '/tmp/tensorflow/mnist/input_data/train-images-idx3-ubyte.gz\\n'\n",
+ " 'Successfully downloaded '\n",
+ " 'train-labels-idx1-ubyte.gz 28881 '\n",
+ " 'bytes.\\n'\n",
+ " 'Extracting '\n",
+ " '/tmp/tensorflow/mnist/input_data/train-labels-idx1-ubyte.gz\\n'\n",
+ " 'Successfully downloaded '\n",
+ " 't10k-images-idx3-ubyte.gz 1648877 '\n",
+ " 'bytes.\\n'\n",
+ " 'Extracting '\n",
+ " '/tmp/tensorflow/mnist/input_data/t10k-images-idx3-ubyte.gz\\n'\n",
+ " 'Successfully downloaded '\n",
+ " 't10k-labels-idx1-ubyte.gz 4542 bytes.\\n'\n",
+ " 'Extracting '\n",
+ " '/tmp/tensorflow/mnist/input_data/t10k-labels-idx1-ubyte.gz\\n'\n",
+ " 'Accuracy at step 0: 0.1102\\n'\n",
+ " 'Accuracy at step 10: 0.6304\\n'\n",
+ " 'Accuracy at step 20: 0.8477\\n'\n",
+ " 'Accuracy at step 30: 0.886\\n'\n",
+ " 'Accuracy at step 40: 0.9034\\n'\n",
+ " 'Accuracy at step 50: 0.9165\\n'\n",
+ " 'Accuracy at step 60: 0.9248\\n'\n",
+ " 'Accuracy at step 70: 0.9211\\n'\n",
+ " 'Accuracy at step 80: 0.9304\\n'\n",
+ " 'Accuracy at step 90: 0.9333\\n'\n",
+ " 'Adding run metadata for 99\\n'\n",
+ " 'Accuracy at step 100: 0.9388\\n'\n",
+ " 'Accuracy at step 110: 0.9345\\n'\n",
+ " 'Accuracy at step 120: 0.9427\\n'\n",
+ " 'Accuracy at step 130: 0.9452\\n'\n",
+ " 'Accuracy at step 140: 0.9393\\n'\n",
+ " 'Accuracy at step 150: 0.9474\\n'\n",
+ " 'Accuracy at step 160: 0.9506\\n'\n",
+ " 'Accuracy at step 170: 0.9539\\n'\n",
+ " 'Accuracy at step 180: 0.9491\\n'\n",
+ " 'Accuracy at step 190: 0.946\\n'\n",
+ " 'Adding run metadata for 199\\n'\n",
+ " 'Accuracy at step 200: 0.9524\\n'\n",
+ " 'Accuracy at step 210: 0.9466\\n'\n",
+ " 'Accuracy at step 220: 0.9466\\n'\n",
+ " 'Accuracy at step 230: 0.9516\\n'\n",
+ " 'Accuracy at step 240: 0.9521\\n'\n",
+ " 'Accuracy at step 250: 0.9516\\n'\n",
+ " 'Accuracy at step 260: 0.9511\\n'\n",
+ " 'Accuracy at step 270: 0.9515\\n'\n",
+ " 'Accuracy at step 280: 0.9535\\n'\n",
+ " 'Accuracy at step 290: 0.956\\n'\n",
+ " 'Adding run metadata for 299\\n'\n",
+ " 'Accuracy at step 300: 0.9594\\n'\n",
+ " 'Accuracy at step 310: 0.9584\\n'\n",
+ " 'Accuracy at step 320: 0.9595\\n'\n",
+ " 'Accuracy at step 330: 0.9584\\n'\n",
+ " 'Accuracy at step 340: 0.9605\\n'\n",
+ " 'Accuracy at step 350: 0.9593\\n'\n",
+ " 'Accuracy at step 360: 0.9637\\n'\n",
+ " 'Accuracy at step 370: 0.9598\\n'\n",
+ " 'Accuracy at step 380: 0.96\\n'\n",
+ " 'Accuracy at step 390: 0.9576\\n'\n",
+ " 'Adding run metadata for 399\\n'\n",
+ " 'Accuracy at step 400: 0.9589\\n'\n",
+ " 'Accuracy at step 410: 0.9587\\n'\n",
+ " 'Accuracy at step 420: 0.9643\\n'\n",
+ " 'Accuracy at step 430: 0.9626\\n'\n",
+ " 'Accuracy at step 440: 0.964\\n'\n",
+ " 'Accuracy at step 450: 0.9641\\n'\n",
+ " 'Accuracy at step 460: 0.9622\\n'\n",
+ " 'Accuracy at step 470: 0.9634\\n'\n",
+ " 'Accuracy at step 480: 0.9669\\n'\n",
+ " 'Accuracy at step 490: 0.9675\\n'\n",
+ " 'Adding run metadata for 499\\n'\n",
+ " 'Accuracy at step 500: 0.9599\\n'\n",
+ " 'Accuracy at step 510: 0.9653\\n'\n",
+ " 'Accuracy at step 520: 0.9665\\n'\n",
+ " 'Accuracy at step 530: 0.9643\\n'\n",
+ " 'Accuracy at step 540: 0.96\\n'\n",
+ " 'Accuracy at step 550: 0.9582\\n'\n",
+ " 'Accuracy at step 560: 0.9644\\n'\n",
+ " 'Accuracy at step 570: 0.9587\\n'\n",
+ " 'Accuracy at step 580: 0.9662\\n'\n",
+ " 'Accuracy at step 590: 0.9612\\n'\n",
+ " 'Adding run metadata for 599\\n'\n",
+ " 'Accuracy at step 600: 0.9574\\n'\n",
+ " 'Accuracy at step 610: 0.9643\\n'\n",
+ " 'Accuracy at step 620: 0.9665\\n'\n",
+ " 'Accuracy at step 630: 0.9657\\n'\n",
+ " 'Accuracy at step 640: 0.9659\\n'\n",
+ " 'Accuracy at step 650: 0.9644\\n'\n",
+ " 'Accuracy at step 660: 0.9683\\n'\n",
+ " 'Accuracy at step 670: 0.969\\n'\n",
+ " 'Accuracy at step 680: 0.9618\\n'\n",
+ " 'Accuracy at step 690: 0.968\\n'\n",
+ " 'Adding run metadata for 699\\n'\n",
+ " 'Accuracy at step 700: 0.9641\\n'\n",
+ " 'Accuracy at step 710: 0.9681\\n'\n",
+ " 'Accuracy at step 720: 0.967\\n'\n",
+ " 'Accuracy at step 730: 0.9661\\n'\n",
+ " 'Accuracy at step 740: 0.9679\\n'\n",
+ " 'Accuracy at step 750: 0.9676\\n'\n",
+ " 'Accuracy at step 760: 0.9681\\n'\n",
+ " 'Accuracy at step 770: 0.9668\\n'\n",
+ " 'Accuracy at step 780: 0.9657\\n'\n",
+ " 'Accuracy at step 790: 0.9597\\n'\n",
+ " 'Adding run metadata for 799\\n'\n",
+ " 'Accuracy at step 800: 0.964\\n'\n",
+ " 'Accuracy at step 810: 0.9655\\n'\n",
+ " 'Accuracy at step 820: 0.966\\n'\n",
+ " 'Accuracy at step 830: 0.9706\\n'\n",
+ " 'Accuracy at step 840: 0.9702\\n'\n",
+ " 'Accuracy at step 850: 0.969\\n'\n",
+ " 'Accuracy at step 860: 0.97\\n'\n",
+ " 'Accuracy at step 870: 0.9712\\n'\n",
+ " 'Accuracy at step 880: 0.9677\\n'\n",
+ " 'Accuracy at step 890: 0.9655\\n'\n",
+ " 'Adding run metadata for 899\\n'\n",
+ " 'Accuracy at step 900: 0.9696\\n'\n",
+ " 'Accuracy at step 910: 0.9718\\n'\n",
+ " 'Accuracy at step 920: 0.9708\\n'\n",
+ " 'Accuracy at step 930: 0.971\\n'\n",
+ " 'Accuracy at step 940: 0.9697\\n'\n",
+ " 'Accuracy at step 950: 0.9698\\n'\n",
+ " 'Accuracy at step 960: 0.969\\n'\n",
+ " 'Accuracy at step 970: 0.9685\\n'\n",
+ " 'Accuracy at step 980: 0.968\\n'\n",
+ " 'Accuracy at step 990: 0.9682\\n'\n",
+ " 'Adding run metadata for 999\\n',\n",
+ " 'podName': 'mnist-worker-0'}]},\n",
+ " 'success': True}"
+ ]
+ },
+ "execution_count": 24,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "submarine_client.get_log(id)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Delete the experiment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {
+ "pycharm": {
+ "name": "#%%\n"
+ }
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "{'attributes': {},\n",
+ " 'code': 200,\n",
+ " 'result': {'acceptedTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'createdTime': '2020-05-21T18:23:09.000+08:00',\n",
+ " 'finishedTime': '2020-05-21T18:35:44.000+08:00',\n",
+ " 'jobId': 'job_1590056548552_0001',\n",
+ " 'name': 'mnist',\n",
+ " 'runningTime': '2020-05-21T18:23:11.000+08:00',\n",
+ " 'spec': {'librarySpec': {'cmd': 'python '\n",
+ " '/var/tf_mnist/mnist_with_summaries.py '\n",
+ " '--log_dir=/train/log '\n",
+ " '--learning_rate=0.01 '\n",
+ " '--batch_size=150',\n",
+ " 'envVars': {'ENV1': 'ENV1'},\n",
+ " 'image': 'gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0',\n",
+ " 'name': 'tensorflow',\n",
+ " 'version': '2.1.0'},\n",
+ " 'name': 'mnist',\n",
+ " 'namespace': 'submarine',\n",
+ " 'projects': None,\n",
+ " 'taskSpecs': {'Worker': {'cmd': None,\n",
+ " 'envVars': None,\n",
+ " 'image': None,\n",
+ " 'name': 'tensorlfow',\n",
+ " 'replicas': 1,\n",
+ " 'resourceMap': {'cpu': '4',\n",
+ " 'memory': '2048M'},\n",
+ " 'resources': 'cpu=4,memory=2048M'}}},\n",
+ " 'status': 'Deleted',\n",
+ " 'uid': '10b226b9-3710-40e9-9f13-9322e03dcb8d'},\n",
+ " 'success': True}"
+ ]
+ },
+ "execution_count": 25,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "submarine_client.delete_job(id)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ },
+ "pycharm": {
+ "stem_cell": {
+ "cell_type": "raw",
+ "source": [],
+ "metadata": {
+ "collapsed": false
+ }
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/submarine-sdk/pysubmarine/github-actions/lint.sh b/submarine-sdk/pysubmarine/github-actions/lint.sh
index d86e54c..3c403e1 100755
--- a/submarine-sdk/pysubmarine/github-actions/lint.sh
+++ b/submarine-sdk/pysubmarine/github-actions/lint.sh
@@ -21,6 +21,6 @@ cd "$FWDIR"
cd ..
pycodestyle --max-line-length=100 -- submarine tests
-pylint --msg-template="{path} ({line},{column}): [{msg_id} {symbol}] {msg}" --rcfile=pylintrc -- submarine tests
+pylint --ignore job --msg-template="{path} ({line},{column}): [{msg_id} {symbol}] {msg}" --rcfile=pylintrc -- submarine tests
set +ex
diff --git a/submarine-sdk/pysubmarine/github-actions/test-requirements.txt b/submarine-sdk/pysubmarine/github-actions/test-requirements.txt
index 664f654..d3a2735 100644
--- a/submarine-sdk/pysubmarine/github-actions/test-requirements.txt
+++ b/submarine-sdk/pysubmarine/github-actions/test-requirements.txt
@@ -23,6 +23,12 @@ attrdict==2.0.0
pytest==3.2.1
pytest-cov==2.6.0
pytest-localserver==0.5.0
+pylint==2.5.2
sqlalchemy==1.3.0
PyMySQL==0.9.3
-pytest-mock==1.13.0
\ No newline at end of file
+pytest-mock==1.13.0
+certifi >= 14.05.14
+six >= 1.10
+python_dateutil >= 2.5.3
+setuptools >= 21.0.0
+urllib3 >= 1.15.1
diff --git a/submarine-sdk/pysubmarine/setup.py b/submarine-sdk/pysubmarine/setup.py
index 8e4aac2..1f27e2f 100644
--- a/submarine-sdk/pysubmarine/setup.py
+++ b/submarine-sdk/pysubmarine/setup.py
@@ -30,11 +30,15 @@ setup(
'sqlparse',
'pymysql',
'tensorflow>=1.14.0,<2.0.0',
- 'requests'
+ 'requests',
+ 'urllib3 >= 1.15.1',
+ 'certifi >= 14.05.14',
+ 'python-dateutil >= 2.5.3'
],
classifiers=[
'Intended Audience :: Developers',
- 'Programming Language :: Python :: 2.7',
+ 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
],
)
diff --git a/submarine-sdk/pysubmarine/submarine/__init__.py b/submarine-sdk/pysubmarine/submarine/__init__.py
index 84b8890..6985d07 100644
--- a/submarine-sdk/pysubmarine/submarine/__init__.py
+++ b/submarine-sdk/pysubmarine/submarine/__init__.py
@@ -13,6 +13,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+
+from submarine.job import ApiClient, JobLibrarySpec, JobSpec, JobTaskSpec,\
+ Configuration, JobsApi
+
import submarine.tracking.fluent
import submarine.tracking as tracking
@@ -21,4 +25,15 @@ log_metric = submarine.tracking.fluent.log_metric
set_tracking_uri = tracking.set_tracking_uri
get_tracking_uri = tracking.get_tracking_uri
-__all__ = ["log_metric", "log_param", "set_tracking_uri", "get_tracking_uri"]
+
+__all__ = ["log_metric",
+ "log_param",
+ "set_tracking_uri",
+ "get_tracking_uri",
+ "ApiClient",
+ "JobLibrarySpec",
+ "JobSpec",
+ "JobTaskSpec",
+ "Configuration",
+ "JobsApi"
+ ]
diff --git a/submarine-sdk/pysubmarine/submarine/job/__init__.py b/submarine-sdk/pysubmarine/submarine/job/__init__.py
index c959729..59a892d 100644
--- a/submarine-sdk/pysubmarine/submarine/job/__init__.py
+++ b/submarine-sdk/pysubmarine/submarine/job/__init__.py
@@ -13,6 +13,28 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-from submarine.job.submarine_job_client import SubmarineJobClient
+# coding: utf-8
-__all__ = ['SubmarineJobClient']
+# flake8: noqa
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+from __future__ import absolute_import
+# import apis into sdk package
+from submarine.job.api.jobs_api import JobsApi
+# import ApiClient
+from submarine.job.api_client import ApiClient
+from submarine.job.configuration import Configuration
+# import models into sdk package
+from submarine.job.models.job_library_spec import JobLibrarySpec
+from submarine.job.models.job_spec import JobSpec
+from submarine.job.models.job_task_spec import JobTaskSpec
+from submarine.job.models.json_response import JsonResponse
diff --git a/submarine-sdk/pysubmarine/submarine/job/__init__.py b/submarine-sdk/pysubmarine/submarine/job/api/__init__.py
similarity index 85%
copy from submarine-sdk/pysubmarine/submarine/job/__init__.py
copy to submarine-sdk/pysubmarine/submarine/job/api/__init__.py
index c959729..90aad48 100644
--- a/submarine-sdk/pysubmarine/submarine/job/__init__.py
+++ b/submarine-sdk/pysubmarine/submarine/job/api/__init__.py
@@ -13,6 +13,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-from submarine.job.submarine_job_client import SubmarineJobClient
+from __future__ import absolute_import
-__all__ = ['SubmarineJobClient']
+# flake8: noqa
+
+# import apis into api package
+from submarine.job.api.jobs_api import JobsApi
diff --git a/submarine-sdk/pysubmarine/submarine/job/api/jobs_api.py b/submarine-sdk/pysubmarine/submarine/job/api/jobs_api.py
new file mode 100644
index 0000000..d80f5d4
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/api/jobs_api.py
@@ -0,0 +1,786 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+from __future__ import absolute_import
+
+import re # noqa: F401
+
+# python 2 and python 3 compatibility library
+import six
+
+from submarine.job.api_client import ApiClient
+
+
+class JobsApi(object):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Do not edit the class manually.
+ Ref: https://github.com/swagger-api/swagger-codegen
+ """
+
+ def __init__(self, api_client=None):
+ if api_client is None:
+ api_client = ApiClient()
+ self.api_client = api_client
+
+ def create_job(self, **kwargs): # noqa: E501
+ """Create a job # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.create_job(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param JobSpec body:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.create_job_with_http_info(**kwargs) # noqa: E501
+ else:
+ (data) = self.create_job_with_http_info(**kwargs) # noqa: E501
+ return data
+
+ def create_job_with_http_info(self, **kwargs): # noqa: E501
+ """Create a job # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.create_job_with_http_info(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param JobSpec body:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['body'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method create_job" % key
+ )
+ params[key] = val
+ del params['kwargs']
+
+ collection_formats = {}
+
+ path_params = {}
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ if 'body' in params:
+ body_params = params['body']
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # HTTP header `Content-Type`
+ header_params['Content-Type'] = self.api_client.select_header_content_type( # noqa: E501
+ ['application/yaml', 'application/json']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs', 'POST',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def delete_job(self, id, **kwargs): # noqa: E501
+ """Delete the job # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.delete_job(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.delete_job_with_http_info(id, **kwargs) # noqa: E501
+ else:
+ (data) = self.delete_job_with_http_info(id, **kwargs) # noqa: E501
+ return data
+
+ def delete_job_with_http_info(self, id, **kwargs): # noqa: E501
+ """Delete the job # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.delete_job_with_http_info(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['id'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method delete_job" % key
+ )
+ params[key] = val
+ del params['kwargs']
+ # verify the required parameter 'id' is set
+ if ('id' not in params or
+ params['id'] is None):
+ raise ValueError("Missing the required parameter `id` when calling `delete_job`") # noqa: E501
+
+ collection_formats = {}
+
+ path_params = {}
+ if 'id' in params:
+ path_params['id'] = params['id'] # noqa: E501
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/{id}', 'DELETE',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def get_job(self, id, **kwargs): # noqa: E501
+ """Find job by id # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.get_job(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.get_job_with_http_info(id, **kwargs) # noqa: E501
+ else:
+ (data) = self.get_job_with_http_info(id, **kwargs) # noqa: E501
+ return data
+
+ def get_job_with_http_info(self, id, **kwargs): # noqa: E501
+ """Find job by id # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.get_job_with_http_info(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['id'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method get_job" % key
+ )
+ params[key] = val
+ del params['kwargs']
+ # verify the required parameter 'id' is set
+ if ('id' not in params or
+ params['id'] is None):
+ raise ValueError("Missing the required parameter `id` when calling `get_job`") # noqa: E501
+
+ collection_formats = {}
+
+ path_params = {}
+ if 'id' in params:
+ path_params['id'] = params['id'] # noqa: E501
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/{id}', 'GET',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def get_log(self, id, **kwargs): # noqa: E501
+ """Log job by id # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.get_log(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.get_log_with_http_info(id, **kwargs) # noqa: E501
+ else:
+ (data) = self.get_log_with_http_info(id, **kwargs) # noqa: E501
+ return data
+
+ def get_log_with_http_info(self, id, **kwargs): # noqa: E501
+ """Log job by id # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.get_log_with_http_info(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['id'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method get_log" % key
+ )
+ params[key] = val
+ del params['kwargs']
+ # verify the required parameter 'id' is set
+ if ('id' not in params or
+ params['id'] is None):
+ raise ValueError("Missing the required parameter `id` when calling `get_log`") # noqa: E501
+
+ collection_formats = {}
+
+ path_params = {}
+ if 'id' in params:
+ path_params['id'] = params['id'] # noqa: E501
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/logs/{id}', 'GET',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def list_job(self, **kwargs): # noqa: E501
+ """List jobs # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.list_job(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str status:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.list_job_with_http_info(**kwargs) # noqa: E501
+ else:
+ (data) = self.list_job_with_http_info(**kwargs) # noqa: E501
+ return data
+
+ def list_job_with_http_info(self, **kwargs): # noqa: E501
+ """List jobs # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.list_job_with_http_info(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str status:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['status'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method list_job" % key
+ )
+ params[key] = val
+ del params['kwargs']
+
+ collection_formats = {}
+
+ path_params = {}
+
+ query_params = []
+ if 'status' in params:
+ query_params.append(('status', params['status'])) # noqa: E501
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs', 'GET',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def list_log(self, **kwargs): # noqa: E501
+ """Log jobs # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.list_log(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str status:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.list_log_with_http_info(**kwargs) # noqa: E501
+ else:
+ (data) = self.list_log_with_http_info(**kwargs) # noqa: E501
+ return data
+
+ def list_log_with_http_info(self, **kwargs): # noqa: E501
+ """Log jobs # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.list_log_with_http_info(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str status:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['status'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method list_log" % key
+ )
+ params[key] = val
+ del params['kwargs']
+
+ collection_formats = {}
+
+ path_params = {}
+
+ query_params = []
+ if 'status' in params:
+ query_params.append(('status', params['status'])) # noqa: E501
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/logs', 'GET',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def patch_job(self, id, **kwargs): # noqa: E501
+ """Update the job in the submarine server with job spec # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.patch_job(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :param JobSpec body:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.patch_job_with_http_info(id, **kwargs) # noqa: E501
+ else:
+ (data) = self.patch_job_with_http_info(id, **kwargs) # noqa: E501
+ return data
+
+ def patch_job_with_http_info(self, id, **kwargs): # noqa: E501
+ """Update the job in the submarine server with job spec # noqa: E501
+
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.patch_job_with_http_info(id, async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :param str id: (required)
+ :param JobSpec body:
+ :return: JsonResponse
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = ['id', 'body'] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method patch_job" % key
+ )
+ params[key] = val
+ del params['kwargs']
+ # verify the required parameter 'id' is set
+ if ('id' not in params or
+ params['id'] is None):
+ raise ValueError("Missing the required parameter `id` when calling `patch_job`") # noqa: E501
+
+ collection_formats = {}
+
+ path_params = {}
+ if 'id' in params:
+ path_params['id'] = params['id'] # noqa: E501
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ if 'body' in params:
+ body_params = params['body']
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # HTTP header `Content-Type`
+ header_params['Content-Type'] = self.api_client.select_header_content_type( # noqa: E501
+ ['application/yaml', 'application/json']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/{id}', 'PATCH',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='JsonResponse', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
+
+ def ping(self, **kwargs): # noqa: E501
+ """Ping submarine server # noqa: E501
+
+ Return the Pong message for test the connectivity # noqa: E501
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.ping(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :return: str
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+ kwargs['_return_http_data_only'] = True
+ if kwargs.get('async_req'):
+ return self.ping_with_http_info(**kwargs) # noqa: E501
+ else:
+ (data) = self.ping_with_http_info(**kwargs) # noqa: E501
+ return data
+
+ def ping_with_http_info(self, **kwargs): # noqa: E501
+ """Ping submarine server # noqa: E501
+
+ Return the Pong message for test the connectivity # noqa: E501
+ This method makes a synchronous HTTP request by default. To make an
+ asynchronous HTTP request, please pass async_req=True
+ >>> thread = api.ping_with_http_info(async_req=True)
+ >>> result = thread.get()
+
+ :param async_req bool
+ :return: str
+ If the method is called asynchronously,
+ returns the request thread.
+ """
+
+ all_params = [] # noqa: E501
+ all_params.append('async_req')
+ all_params.append('_return_http_data_only')
+ all_params.append('_preload_content')
+ all_params.append('_request_timeout')
+
+ params = locals()
+ for key, val in six.iteritems(params['kwargs']):
+ if key not in all_params:
+ raise TypeError(
+ "Got an unexpected keyword argument '%s'"
+ " to method ping" % key
+ )
+ params[key] = val
+ del params['kwargs']
+
+ collection_formats = {}
+
+ path_params = {}
+
+ query_params = []
+
+ header_params = {}
+
+ form_params = []
+ local_var_files = {}
+
+ body_params = None
+ # HTTP header `Accept`
+ header_params['Accept'] = self.api_client.select_header_accept(
+ ['application/json; charset=utf-8']) # noqa: E501
+
+ # Authentication setting
+ auth_settings = [] # noqa: E501
+
+ return self.api_client.call_api(
+ '/v1/jobs/ping', 'GET',
+ path_params,
+ query_params,
+ header_params,
+ body=body_params,
+ post_params=form_params,
+ files=local_var_files,
+ response_type='str', # noqa: E501
+ auth_settings=auth_settings,
+ async_req=params.get('async_req'),
+ _return_http_data_only=params.get('_return_http_data_only'),
+ _preload_content=params.get('_preload_content', True),
+ _request_timeout=params.get('_request_timeout'),
+ collection_formats=collection_formats)
diff --git a/submarine-sdk/pysubmarine/submarine/job/api_client.py b/submarine-sdk/pysubmarine/submarine/job/api_client.py
new file mode 100644
index 0000000..59b6743
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/api_client.py
@@ -0,0 +1,643 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+from __future__ import absolute_import
+
+import datetime
+import json
+import mimetypes
+from multiprocessing.pool import ThreadPool
+import os
+import re
+import tempfile
+
+# python 2 and python 3 compatibility library
+import six
+from six.moves.urllib.parse import quote
+
+from submarine.job.configuration import Configuration
+import submarine.job.models
+from submarine.job import rest
+
+
+class ApiClient(object):
+ """Generic API client for Swagger client library builds.
+
+ Swagger generic API client. This client handles the client-
+ server communication, and is invariant across implementations. Specifics of
+ the methods and models for each application are generated from the Swagger
+ templates.
+
+ NOTE: This class is auto generated by the swagger code generator program.
+ Ref: https://github.com/swagger-api/swagger-codegen
+ Do not edit the class manually.
+
+ :param configuration: .Configuration object for this client
+ :param header_name: a header to pass when making calls to the API.
+ :param header_value: a header value to pass when making calls to
+ the API.
+ :param cookie: a cookie to include in the header when making calls
+ to the API
+ """
+
+ PRIMITIVE_TYPES = (float, bool, bytes, six.text_type) + six.integer_types
+ NATIVE_TYPES_MAPPING = {
+ 'int': int,
+ 'long': int if six.PY3 else long, # noqa: F821
+ 'float': float,
+ 'str': str,
+ 'bool': bool,
+ 'date': datetime.date,
+ 'datetime': datetime.datetime,
+ 'object': object,
+ }
+
+ def __init__(self, configuration=None, header_name=None, header_value=None,
+ cookie=None):
+ if configuration is None:
+ configuration = Configuration()
+ self.configuration = configuration
+
+ self.pool = ThreadPool()
+ self.rest_client = rest.RESTClientObject(configuration)
+ self.default_headers = {}
+ if header_name is not None:
+ self.default_headers[header_name] = header_value
+ self.cookie = cookie
+ # Set default User-Agent.
+ self.user_agent = 'Swagger-Codegen/1.0.0/python'
+
+ def __del__(self):
+ self.pool.close()
+ self.pool.join()
+
+ @property
+ def user_agent(self):
+ """User agent for this API client"""
+ return self.default_headers['User-Agent']
+
+ @user_agent.setter
+ def user_agent(self, value):
+ self.default_headers['User-Agent'] = value
+
+ def set_default_header(self, header_name, header_value):
+ self.default_headers[header_name] = header_value
+
+ def __call_api(
+ self, resource_path, method, path_params=None,
+ query_params=None, header_params=None, body=None, post_params=None,
+ files=None, response_type=None, auth_settings=None,
+ _return_http_data_only=None, collection_formats=None,
+ _preload_content=True, _request_timeout=None):
+
+ config = self.configuration
+
+ # header parameters
+ header_params = header_params or {}
+ header_params.update(self.default_headers)
+ if self.cookie:
+ header_params['Cookie'] = self.cookie
+ if header_params:
+ header_params = self.sanitize_for_serialization(header_params)
+ header_params = dict(self.parameters_to_tuples(header_params,
+ collection_formats))
+
+ # path parameters
+ if path_params:
+ path_params = self.sanitize_for_serialization(path_params)
+ path_params = self.parameters_to_tuples(path_params,
+ collection_formats)
+ for k, v in path_params:
+ # specified safe chars, encode everything
+ resource_path = resource_path.replace(
+ '{%s}' % k,
+ quote(str(v), safe=config.safe_chars_for_path_param)
+ )
+
+ # query parameters
+ if query_params:
+ query_params = self.sanitize_for_serialization(query_params)
+ query_params = self.parameters_to_tuples(query_params,
+ collection_formats)
+
+ # post parameters
+ if post_params or files:
+ post_params = self.prepare_post_parameters(post_params, files)
+ post_params = self.sanitize_for_serialization(post_params)
+ post_params = self.parameters_to_tuples(post_params,
+ collection_formats)
+
+ # auth setting
+ self.update_params_for_auth(header_params, query_params, auth_settings)
+
+ # body
+ if body:
+ body = self.sanitize_for_serialization(body)
+
+ # request url
+ url = self.configuration.host + resource_path
+
+ # perform request and return response
+ response_data = self.request(
+ method, url, query_params=query_params, headers=header_params,
+ post_params=post_params, body=body,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout)
+
+ self.last_response = response_data
+
+ return_data = response_data
+ if _preload_content:
+ # deserialize response data
+ if response_type:
+ return_data = self.deserialize(response_data, response_type)
+ else:
+ return_data = None
+
+ if _return_http_data_only:
+ return (return_data)
+ else:
+ return (return_data, response_data.status,
+ response_data.getheaders())
+
+ def sanitize_for_serialization(self, obj):
+ """Builds a JSON POST object.
+
+ If obj is None, return None.
+ If obj is str, int, long, float, bool, return directly.
+ If obj is datetime.datetime, datetime.date
+ convert to string in iso8601 format.
+ If obj is list, sanitize each element in the list.
+ If obj is dict, return the dict.
+ If obj is swagger model, return the properties dict.
+
+ :param obj: The data to serialize.
+ :return: The serialized form of data.
+ """
+ if obj is None:
+ return None
+ elif isinstance(obj, self.PRIMITIVE_TYPES):
+ return obj
+ elif isinstance(obj, list):
+ return [self.sanitize_for_serialization(sub_obj)
+ for sub_obj in obj]
+ elif isinstance(obj, tuple):
+ return tuple(self.sanitize_for_serialization(sub_obj)
+ for sub_obj in obj)
+ elif isinstance(obj, (datetime.datetime, datetime.date)):
+ return obj.isoformat()
+
+ if isinstance(obj, dict):
+ obj_dict = obj
+ else:
+ # Convert model obj to dict except
+ # attributes `swagger_types`, `attribute_map`
+ # and attributes which value is not None.
+ # Convert attribute name to json key in
+ # model definition for request.
+ obj_dict = {obj.attribute_map[attr]: getattr(obj, attr)
+ for attr, _ in six.iteritems(obj.swagger_types)
+ if getattr(obj, attr) is not None}
+
+ return {key: self.sanitize_for_serialization(val)
+ for key, val in six.iteritems(obj_dict)}
+
+ def deserialize(self, response, response_type):
+ """Deserializes response into an object.
+
+ :param response: RESTResponse object to be deserialized.
+ :param response_type: class literal for
+ deserialized object, or string of class name.
+
+ :return: deserialized object.
+ """
+ # handle file downloading
+ # save response body into a tmp file and return the instance
+ if response_type == "file":
+ return self.__deserialize_file(response)
+
+ # fetch data from response object
+ try:
+ data = json.loads(response.data)
+ except ValueError:
+ data = response.data
+
+ return self.__deserialize(data, response_type)
+
+ def __deserialize(self, data, klass):
+ """Deserializes dict, list, str into an object.
+
+ :param data: dict, list or str.
+ :param klass: class literal, or string of class name.
+
+ :return: object.
+ """
+ if data is None:
+ return None
+
+ if type(klass) == str:
+ if klass.startswith('list['):
+ sub_kls = re.match(r'list\[(.*)\]', klass).group(1)
+ return [self.__deserialize(sub_data, sub_kls)
+ for sub_data in data]
+
+ if klass.startswith('dict('):
+ sub_kls = re.match(r'dict\(([^,]*), (.*)\)', klass).group(2)
+ return {k: self.__deserialize(v, sub_kls)
+ for k, v in six.iteritems(data)}
+
+ # convert str to class
+ if klass in self.NATIVE_TYPES_MAPPING:
+ klass = self.NATIVE_TYPES_MAPPING[klass]
+ else:
+ klass = getattr(submarine.job.models, klass)
+
+ if klass in self.PRIMITIVE_TYPES:
+ return self.__deserialize_primitive(data, klass)
+ elif klass == object:
+ return self.__deserialize_object(data)
+ elif klass == datetime.date:
+ return self.__deserialize_date(data)
+ elif klass == datetime.datetime:
+ return self.__deserialize_datatime(data)
+ else:
+ return self.__deserialize_model(data, klass)
+
+ def call_api(self, resource_path, method,
+ path_params=None, query_params=None, header_params=None,
+ body=None, post_params=None, files=None,
+ response_type=None, auth_settings=None, async_req=None,
+ _return_http_data_only=None, collection_formats=None,
+ _preload_content=True, _request_timeout=None):
+ """Makes the HTTP request (synchronous) and returns deserialized data.
+
+ To make an async request, set the async_req parameter.
+
+ :param resource_path: Path to method endpoint.
+ :param method: Method to call.
+ :param path_params: Path parameters in the url.
+ :param query_params: Query parameters in the url.
+ :param header_params: Header parameters to be
+ placed in the request header.
+ :param body: Request body.
+ :param post_params dict: Request post form parameters,
+ for `application/x-www-form-urlencoded`, `multipart/form-data`.
+ :param auth_settings list: Auth Settings names for the request.
+ :param response: Response data type.
+ :param files dict: key -> filename, value -> filepath,
+ for `multipart/form-data`.
+ :param async_req bool: execute request asynchronously
+ :param _return_http_data_only: response data without head status code
+ and headers
+ :param collection_formats: dict of collection formats for path, query,
+ header, and post parameters.
+ :param _preload_content: if False, the urllib3.HTTPResponse object will
+ be returned without reading/decoding response
+ data. Default is True.
+ :param _request_timeout: timeout setting for this request. If one
+ number provided, it will be total request
+ timeout. It can also be a pair (tuple) of
+ (connection, read) timeouts.
+ :return:
+ If async_req parameter is True,
+ the request will be called asynchronously.
+ The method will return the request thread.
+ If parameter async_req is False or missing,
+ then the method will return the response directly.
+ """
+ if not async_req:
+ return self.__call_api(resource_path, method,
+ path_params, query_params, header_params,
+ body, post_params, files,
+ response_type, auth_settings,
+ _return_http_data_only, collection_formats,
+ _preload_content, _request_timeout)
+ else:
+ thread = self.pool.apply_async(self.__call_api, (resource_path,
+ method, path_params, query_params,
+ header_params, body,
+ post_params, files,
+ response_type, auth_settings,
+ _return_http_data_only,
+ collection_formats,
+ _preload_content, _request_timeout))
+ return thread
+
+ def request(self, method, url, query_params=None, headers=None,
+ post_params=None, body=None, _preload_content=True,
+ _request_timeout=None):
+ """Makes the HTTP request using RESTClient."""
+ if method == "GET":
+ return self.rest_client.GET(url,
+ query_params=query_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ headers=headers)
+ elif method == "HEAD":
+ return self.rest_client.HEAD(url,
+ query_params=query_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ headers=headers)
+ elif method == "OPTIONS":
+ return self.rest_client.OPTIONS(url,
+ query_params=query_params,
+ headers=headers,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+ elif method == "POST":
+ return self.rest_client.POST(url,
+ query_params=query_params,
+ headers=headers,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+ elif method == "PUT":
+ return self.rest_client.PUT(url,
+ query_params=query_params,
+ headers=headers,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+ elif method == "PATCH":
+ return self.rest_client.PATCH(url,
+ query_params=query_params,
+ headers=headers,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+ elif method == "DELETE":
+ return self.rest_client.DELETE(url,
+ query_params=query_params,
+ headers=headers,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+ else:
+ raise ValueError(
+ "http method must be `GET`, `HEAD`, `OPTIONS`,"
+ " `POST`, `PATCH`, `PUT` or `DELETE`."
+ )
+
+ def parameters_to_tuples(self, params, collection_formats):
+ """Get parameters as list of tuples, formatting collections.
+
+ :param params: Parameters as dict or list of two-tuples
+ :param dict collection_formats: Parameter collection formats
+ :return: Parameters as list of tuples, collections formatted
+ """
+ new_params = []
+ if collection_formats is None:
+ collection_formats = {}
+ for k, v in six.iteritems(params) if isinstance(params, dict) else params: # noqa: E501
+ if k in collection_formats:
+ collection_format = collection_formats[k]
+ if collection_format == 'multi':
+ new_params.extend((k, value) for value in v)
+ else:
+ if collection_format == 'ssv':
+ delimiter = ' '
+ elif collection_format == 'tsv':
+ delimiter = '\t'
+ elif collection_format == 'pipes':
+ delimiter = '|'
+ else: # csv is the default
+ delimiter = ','
+ new_params.append(
+ (k, delimiter.join(str(value) for value in v)))
+ else:
+ new_params.append((k, v))
+ return new_params
+
+ def prepare_post_parameters(self, post_params=None, files=None):
+ """Builds form parameters.
+
+ :param post_params: Normal form parameters.
+ :param files: File parameters.
+ :return: Form parameters with files.
+ """
+ params = []
+
+ if post_params:
+ params = post_params
+
+ if files:
+ for k, v in six.iteritems(files):
+ if not v:
+ continue
+ file_names = v if type(v) is list else [v]
+ for n in file_names:
+ with open(n, 'rb') as f:
+ filename = os.path.basename(f.name)
+ filedata = f.read()
+ mimetype = (mimetypes.guess_type(filename)[0] or
+ 'application/octet-stream')
+ params.append(
+ tuple([k, tuple([filename, filedata, mimetype])]))
+
+ return params
+
+ def select_header_accept(self, accepts):
+ """Returns `Accept` based on an array of accepts provided.
+
+ :param accepts: List of headers.
+ :return: Accept (e.g. application/json).
+ """
+ if not accepts:
+ return
+
+ accepts = [x.lower() for x in accepts]
+
+ if 'application/json' in accepts:
+ return 'application/json'
+ else:
+ return ', '.join(accepts)
+
+ def select_header_content_type(self, content_types):
+ """Returns `Content-Type` based on an array of content_types provided.
+
+ :param content_types: List of content-types.
+ :return: Content-Type (e.g. application/json).
+ """
+ if not content_types:
+ return 'application/json'
+
+ content_types = [x.lower() for x in content_types]
+
+ if 'application/json' in content_types or '*/*' in content_types:
+ return 'application/json'
+ else:
+ return content_types[0]
+
+ def update_params_for_auth(self, headers, querys, auth_settings):
+ """Updates header and query params based on authentication setting.
+
+ :param headers: Header parameters dict to be updated.
+ :param querys: Query parameters tuple list to be updated.
+ :param auth_settings: Authentication setting identifiers list.
+ """
+ if not auth_settings:
+ return
+
+ for auth in auth_settings:
+ auth_setting = self.configuration.auth_settings().get(auth)
+ if auth_setting:
+ if not auth_setting['value']:
+ continue
+ elif auth_setting['in'] == 'header':
+ headers[auth_setting['key']] = auth_setting['value']
+ elif auth_setting['in'] == 'query':
+ querys.append((auth_setting['key'], auth_setting['value']))
+ else:
+ raise ValueError(
+ 'Authentication token must be in `query` or `header`'
+ )
+
+ def __deserialize_file(self, response):
+ """Deserializes body to file
+
+ Saves response body into a file in a temporary folder,
+ using the filename from the `Content-Disposition` header if provided.
+
+ :param response: RESTResponse.
+ :return: file path.
+ """
+ fd, path = tempfile.mkstemp(dir=self.configuration.temp_folder_path)
+ os.close(fd)
+ os.remove(path)
+
+ content_disposition = response.getheader("Content-Disposition")
+ if content_disposition:
+ filename = re.search(r'filename=[\'"]?([^\'"\s]+)[\'"]?',
+ content_disposition).group(1)
+ path = os.path.join(os.path.dirname(path), filename)
+
+ with open(path, "wb") as f:
+ f.write(response.data)
+
+ return path
+
+ def __deserialize_primitive(self, data, klass):
+ """Deserializes string to primitive type.
+
+ :param data: str.
+ :param klass: class literal.
+
+ :return: int, long, float, str, bool.
+ """
+ try:
+ return klass(data)
+ except UnicodeEncodeError:
+ return six.text_type(data)
+ except TypeError:
+ return data
+
+ def __deserialize_object(self, value):
+ """Return a original value.
+
+ :return: object.
+ """
+ return value
+
+ def __deserialize_date(self, string):
+ """Deserializes string to date.
+
+ :param string: str.
+ :return: date.
+ """
+ try:
+ from dateutil.parser import parse
+ return parse(string).date()
+ except ImportError:
+ return string
+ except ValueError:
+ raise rest.ApiException(
+ status=0,
+ reason="Failed to parse `{0}` as date object".format(string)
+ )
+
+ def __deserialize_datatime(self, string):
+ """Deserializes string to datetime.
+
+ The string should be in iso8601 datetime format.
+
+ :param string: str.
+ :return: datetime.
+ """
+ try:
+ from dateutil.parser import parse
+ return parse(string)
+ except ImportError:
+ return string
+ except ValueError:
+ raise rest.ApiException(
+ status=0,
+ reason=(
+ "Failed to parse `{0}` as datetime object"
+ .format(string)
+ )
+ )
+
+ def __hasattr(self, object, name):
+ return name in object.__class__.__dict__
+
+ def __deserialize_model(self, data, klass):
+ """Deserializes list or dict to model.
+
+ :param data: dict, list.
+ :param klass: class literal.
+ :return: model object.
+ """
+
+ if not klass.swagger_types and not self.__hasattr(klass, 'get_real_child_model'):
+ return data
+
+ kwargs = {}
+ if klass.swagger_types is not None:
+ for attr, attr_type in six.iteritems(klass.swagger_types):
+ if (data is not None and
+ klass.attribute_map[attr] in data and
+ isinstance(data, (list, dict))):
+ value = data[klass.attribute_map[attr]]
+ kwargs[attr] = self.__deserialize(value, attr_type)
+
+ instance = klass(**kwargs)
+
+ if (isinstance(instance, dict) and
+ klass.swagger_types is not None and
+ isinstance(data, dict)):
+ for key, value in data.items():
+ if key not in klass.swagger_types:
+ instance[key] = value
+ if self.__hasattr(instance, 'get_real_child_model'):
+ klass_name = instance.get_real_child_model(data)
+ if klass_name:
+ instance = self.__deserialize(data, klass_name)
+ return instance
diff --git a/submarine-sdk/pysubmarine/submarine/job/configuration.py b/submarine-sdk/pysubmarine/submarine/job/configuration.py
new file mode 100644
index 0000000..105f775
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/configuration.py
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+from __future__ import absolute_import
+
+import copy
+import logging
+import multiprocessing
+import sys
+import urllib3
+
+import six
+from six.moves import http_client as httplib
+
+
+class TypeWithDefault(type):
+ def __init__(cls, name, bases, dct):
+ super(TypeWithDefault, cls).__init__(name, bases, dct)
+ cls._default = None
+
+ def __call__(cls):
+ if cls._default is None:
+ cls._default = type.__call__(cls)
+ return copy.copy(cls._default)
+
+ def set_default(cls, default):
+ cls._default = copy.copy(default)
+
+
+class Configuration(six.with_metaclass(TypeWithDefault, object)):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Ref: https://github.com/swagger-api/swagger-codegen
+ Do not edit the class manually.
+ """
+
+ def __init__(self):
+ """Constructor"""
+ # Default Base url
+ self.host = "/api"
+ # Temp file folder for downloading files
+ self.temp_folder_path = None
+
+ # Authentication Settings
+ # dict to store API key(s)
+ self.api_key = {}
+ # dict to store API prefix (e.g. Bearer)
+ self.api_key_prefix = {}
+ # function to refresh API key if expired
+ self.refresh_api_key_hook = None
+ # Username for HTTP basic authentication
+ self.username = ""
+ # Password for HTTP basic authentication
+ self.password = ""
+ # Logging Settings
+ self.logger = {}
+ self.logger["package_logger"] = logging.getLogger("submarine.job")
+ self.logger["urllib3_logger"] = logging.getLogger("urllib3")
+ # Log format
+ self.logger_format = '%(asctime)s %(levelname)s %(message)s'
+ # Log stream handler
+ self.logger_stream_handler = None
+ # Log file handler
+ self.logger_file_handler = None
+ # Debug file location
+ self.logger_file = None
+ # Debug switch
+ self.debug = False
+
+ # SSL/TLS verification
+ # Set this to false to skip verifying SSL certificate when calling API
+ # from https server.
+ self.verify_ssl = True
+ # Set this to customize the certificate file to verify the peer.
+ self.ssl_ca_cert = None
+ # client certificate file
+ self.cert_file = None
+ # client key file
+ self.key_file = None
+ # Set this to True/False to enable/disable SSL hostname verification.
+ self.assert_hostname = None
+
+ # urllib3 connection pool's maximum number of connections saved
+ # per pool. urllib3 uses 1 connection as default value, but this is
+ # not the best value when you are making a lot of possibly parallel
+ # requests to the same host, which is often the case here.
+ # cpu_count * 5 is used as default value to increase performance.
+ self.connection_pool_maxsize = multiprocessing.cpu_count() * 5
+
+ # Proxy URL
+ self.proxy = None
+ # Safe chars for path_param
+ self.safe_chars_for_path_param = ''
+
+ @property
+ def logger_file(self):
+ """The logger file.
+
+ If the logger_file is None, then add stream handler and remove file
+ handler. Otherwise, add file handler and remove stream handler.
+
+ :param value: The logger_file path.
+ :type: str
+ """
+ return self.__logger_file
+
+ @logger_file.setter
+ def logger_file(self, value):
+ """The logger file.
+
+ If the logger_file is None, then add stream handler and remove file
+ handler. Otherwise, add file handler and remove stream handler.
+
+ :param value: The logger_file path.
+ :type: str
+ """
+ self.__logger_file = value
+ if self.__logger_file:
+ # If set logging file,
+ # then add file handler and remove stream handler.
+ self.logger_file_handler = logging.FileHandler(self.__logger_file)
+ self.logger_file_handler.setFormatter(self.logger_formatter)
+ for _, logger in six.iteritems(self.logger):
+ logger.addHandler(self.logger_file_handler)
+ if self.logger_stream_handler:
+ logger.removeHandler(self.logger_stream_handler)
+ else:
+ # If not set logging file,
+ # then add stream handler and remove file handler.
+ self.logger_stream_handler = logging.StreamHandler()
+ self.logger_stream_handler.setFormatter(self.logger_formatter)
+ for _, logger in six.iteritems(self.logger):
+ logger.addHandler(self.logger_stream_handler)
+ if self.logger_file_handler:
+ logger.removeHandler(self.logger_file_handler)
+
+ @property
+ def debug(self):
+ """Debug status
+
+ :param value: The debug status, True or False.
+ :type: bool
+ """
+ return self.__debug
+
+ @debug.setter
+ def debug(self, value):
+ """Debug status
+
+ :param value: The debug status, True or False.
+ :type: bool
+ """
+ self.__debug = value
+ if self.__debug:
+ # if debug status is True, turn on debug logging
+ for _, logger in six.iteritems(self.logger):
+ logger.setLevel(logging.DEBUG)
+ # turn on httplib debug
+ httplib.HTTPConnection.debuglevel = 1
+ else:
+ # if debug status is False, turn off debug logging,
+ # setting log level to default `logging.WARNING`
+ for _, logger in six.iteritems(self.logger):
+ logger.setLevel(logging.WARNING)
+ # turn off httplib debug
+ httplib.HTTPConnection.debuglevel = 0
+
+ @property
+ def logger_format(self):
+ """The logger format.
+
+ The logger_formatter will be updated when sets logger_format.
+
+ :param value: The format string.
+ :type: str
+ """
+ return self.__logger_format
+
+ @logger_format.setter
+ def logger_format(self, value):
+ """The logger format.
+
+ The logger_formatter will be updated when sets logger_format.
+
+ :param value: The format string.
+ :type: str
+ """
+ self.__logger_format = value
+ self.logger_formatter = logging.Formatter(self.__logger_format)
+
+ def get_api_key_with_prefix(self, identifier):
+ """Gets API key (with prefix if set).
+
+ :param identifier: The identifier of apiKey.
+ :return: The token for api key authentication.
+ """
+ if self.refresh_api_key_hook:
+ self.refresh_api_key_hook(self)
+
+ key = self.api_key.get(identifier)
+ if key:
+ prefix = self.api_key_prefix.get(identifier)
+ if prefix:
+ return "%s %s" % (prefix, key)
+ else:
+ return key
+
+ def get_basic_auth_token(self):
+ """Gets HTTP basic authentication header (string).
+
+ :return: The token for basic HTTP authentication.
+ """
+ return urllib3.util.make_headers(
+ basic_auth=self.username + ':' + self.password
+ ).get('authorization')
+
+ def auth_settings(self):
+ """Gets Auth Settings dict for api client.
+
+ :return: The Auth Settings information dict.
+ """
+ return {
+ }
+
+ def to_debug_report(self):
+ """Gets the essential information for debugging.
+
+ :return: The report for debugging.
+ """
+ return "Python SDK Debug Report:\n"\
+ "OS: {env}\n"\
+ "Python Version: {pyversion}\n"\
+ "Version of the API: 0.4.0-SNAPSHOT\n"\
+ "SDK Package Version: 1.0.0".\
+ format(env=sys.platform, pyversion=sys.version)
diff --git a/submarine-sdk/pysubmarine/submarine/job/models/__init__.py b/submarine-sdk/pysubmarine/submarine/job/models/__init__.py
new file mode 100644
index 0000000..c827ebe
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/models/__init__.py
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+# flake8: noqa
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+from __future__ import absolute_import
+
+# import models into model package
+from submarine.job.models.job_library_spec import JobLibrarySpec
+from submarine.job.models.job_spec import JobSpec
+from submarine.job.models.job_task_spec import JobTaskSpec
+from submarine.job.models.json_response import JsonResponse
diff --git a/submarine-sdk/pysubmarine/submarine/job/models/job_library_spec.py b/submarine-sdk/pysubmarine/submarine/job/models/job_library_spec.py
new file mode 100644
index 0000000..9a0dc81
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/models/job_library_spec.py
@@ -0,0 +1,230 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+import pprint
+import re # noqa: F401
+
+import six
+
+
+class JobLibrarySpec(object):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Do not edit the class manually.
+ """
+ """
+ Attributes:
+ swagger_types (dict): The key is attribute name
+ and the value is attribute type.
+ attribute_map (dict): The key is attribute name
+ and the value is json key in definition.
+ """
+ swagger_types = {
+ 'name': 'str',
+ 'version': 'str',
+ 'image': 'str',
+ 'cmd': 'str',
+ 'env_vars': 'dict(str, str)'
+ }
+
+ attribute_map = {
+ 'name': 'name',
+ 'version': 'version',
+ 'image': 'image',
+ 'cmd': 'cmd',
+ 'env_vars': 'envVars'
+ }
+
+ def __init__(self, name=None, version=None, image=None, cmd=None, env_vars=None): # noqa: E501
+ """JobLibrarySpec - a model defined in Swagger""" # noqa: E501
+ self._name = None
+ self._version = None
+ self._image = None
+ self._cmd = None
+ self._env_vars = None
+ self.discriminator = None
+ if name is not None:
+ self.name = name
+ if version is not None:
+ self.version = version
+ if image is not None:
+ self.image = image
+ if cmd is not None:
+ self.cmd = cmd
+ if env_vars is not None:
+ self.env_vars = env_vars
+
+ @property
+ def name(self):
+ """Gets the name of this JobLibrarySpec. # noqa: E501
+
+
+ :return: The name of this JobLibrarySpec. # noqa: E501
+ :rtype: str
+ """
+ return self._name
+
+ @name.setter
+ def name(self, name):
+ """Sets the name of this JobLibrarySpec.
+
+
+ :param name: The name of this JobLibrarySpec. # noqa: E501
+ :type: str
+ """
+
+ self._name = name
+
+ @property
+ def version(self):
+ """Gets the version of this JobLibrarySpec. # noqa: E501
+
+
+ :return: The version of this JobLibrarySpec. # noqa: E501
+ :rtype: str
+ """
+ return self._version
+
+ @version.setter
+ def version(self, version):
+ """Sets the version of this JobLibrarySpec.
+
+
+ :param version: The version of this JobLibrarySpec. # noqa: E501
+ :type: str
+ """
+
+ self._version = version
+
+ @property
+ def image(self):
+ """Gets the image of this JobLibrarySpec. # noqa: E501
+
+
+ :return: The image of this JobLibrarySpec. # noqa: E501
+ :rtype: str
+ """
+ return self._image
+
+ @image.setter
+ def image(self, image):
+ """Sets the image of this JobLibrarySpec.
+
+
+ :param image: The image of this JobLibrarySpec. # noqa: E501
+ :type: str
+ """
+
+ self._image = image
+
+ @property
+ def cmd(self):
+ """Gets the cmd of this JobLibrarySpec. # noqa: E501
+
+
+ :return: The cmd of this JobLibrarySpec. # noqa: E501
+ :rtype: str
+ """
+ return self._cmd
+
+ @cmd.setter
+ def cmd(self, cmd):
+ """Sets the cmd of this JobLibrarySpec.
+
+
+ :param cmd: The cmd of this JobLibrarySpec. # noqa: E501
+ :type: str
+ """
+
+ self._cmd = cmd
+
+ @property
+ def env_vars(self):
+ """Gets the env_vars of this JobLibrarySpec. # noqa: E501
+
+
+ :return: The env_vars of this JobLibrarySpec. # noqa: E501
+ :rtype: dict(str, str)
+ """
+ return self._env_vars
+
+ @env_vars.setter
+ def env_vars(self, env_vars):
+ """Sets the env_vars of this JobLibrarySpec.
+
+
+ :param env_vars: The env_vars of this JobLibrarySpec. # noqa: E501
+ :type: dict(str, str)
+ """
+
+ self._env_vars = env_vars
+
+ def to_dict(self):
+ """Returns the model properties as a dict"""
+ result = {}
+
+ for attr, _ in six.iteritems(self.swagger_types):
+ value = getattr(self, attr)
+ if isinstance(value, list):
+ result[attr] = list(map(
+ lambda x: x.to_dict() if hasattr(x, "to_dict") else x,
+ value
+ ))
+ elif hasattr(value, "to_dict"):
+ result[attr] = value.to_dict()
+ elif isinstance(value, dict):
+ result[attr] = dict(map(
+ lambda item: (item[0], item[1].to_dict())
+ if hasattr(item[1], "to_dict") else item,
+ value.items()
+ ))
+ else:
+ result[attr] = value
+ if issubclass(JobLibrarySpec, dict):
+ for key, value in self.items():
+ result[key] = value
+
+ return result
+
+ def to_str(self):
+ """Returns the string representation of the model"""
+ return pprint.pformat(self.to_dict())
+
+ def __repr__(self):
+ """For `print` and `pprint`"""
+ return self.to_str()
+
+ def __eq__(self, other):
+ """Returns true if both objects are equal"""
+ if not isinstance(other, JobLibrarySpec):
+ return False
+
+ return self.__dict__ == other.__dict__
+
+ def __ne__(self, other):
+ """Returns true if both objects are not equal"""
+ return not self == other
diff --git a/submarine-sdk/pysubmarine/submarine/job/models/job_spec.py b/submarine-sdk/pysubmarine/submarine/job/models/job_spec.py
new file mode 100644
index 0000000..8da643c
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/models/job_spec.py
@@ -0,0 +1,230 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+import pprint
+import re # noqa: F401
+
+import six
+
+
+class JobSpec(object):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Do not edit the class manually.
+ """
+ """
+ Attributes:
+ swagger_types (dict): The key is attribute name
+ and the value is attribute type.
+ attribute_map (dict): The key is attribute name
+ and the value is json key in definition.
+ """
+ swagger_types = {
+ 'name': 'str',
+ 'namespace': 'str',
+ 'library_spec': 'JobLibrarySpec',
+ 'task_specs': 'dict(str, JobTaskSpec)',
+ 'projects': 'str'
+ }
+
+ attribute_map = {
+ 'name': 'name',
+ 'namespace': 'namespace',
+ 'library_spec': 'librarySpec',
+ 'task_specs': 'taskSpecs',
+ 'projects': 'projects'
+ }
+
+ def __init__(self, name=None, namespace=None, library_spec=None, task_specs=None, projects=None): # noqa: E501
+ """JobSpec - a model defined in Swagger""" # noqa: E501
+ self._name = None
+ self._namespace = None
+ self._library_spec = None
+ self._task_specs = None
+ self._projects = None
+ self.discriminator = None
+ if name is not None:
+ self.name = name
+ if namespace is not None:
+ self.namespace = namespace
+ if library_spec is not None:
+ self.library_spec = library_spec
+ if task_specs is not None:
+ self.task_specs = task_specs
+ if projects is not None:
+ self.projects = projects
+
+ @property
+ def name(self):
+ """Gets the name of this JobSpec. # noqa: E501
+
+
+ :return: The name of this JobSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._name
+
+ @name.setter
+ def name(self, name):
+ """Sets the name of this JobSpec.
+
+
+ :param name: The name of this JobSpec. # noqa: E501
+ :type: str
+ """
+
+ self._name = name
+
+ @property
+ def namespace(self):
+ """Gets the namespace of this JobSpec. # noqa: E501
+
+
+ :return: The namespace of this JobSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._namespace
+
+ @namespace.setter
+ def namespace(self, namespace):
+ """Sets the namespace of this JobSpec.
+
+
+ :param namespace: The namespace of this JobSpec. # noqa: E501
+ :type: str
+ """
+
+ self._namespace = namespace
+
+ @property
+ def library_spec(self):
+ """Gets the library_spec of this JobSpec. # noqa: E501
+
+
+ :return: The library_spec of this JobSpec. # noqa: E501
+ :rtype: JobLibrarySpec
+ """
+ return self._library_spec
+
+ @library_spec.setter
+ def library_spec(self, library_spec):
+ """Sets the library_spec of this JobSpec.
+
+
+ :param library_spec: The library_spec of this JobSpec. # noqa: E501
+ :type: JobLibrarySpec
+ """
+
+ self._library_spec = library_spec
+
+ @property
+ def task_specs(self):
+ """Gets the task_specs of this JobSpec. # noqa: E501
+
+
+ :return: The task_specs of this JobSpec. # noqa: E501
+ :rtype: dict(str, JobTaskSpec)
+ """
+ return self._task_specs
+
+ @task_specs.setter
+ def task_specs(self, task_specs):
+ """Sets the task_specs of this JobSpec.
+
+
+ :param task_specs: The task_specs of this JobSpec. # noqa: E501
+ :type: dict(str, JobTaskSpec)
+ """
+
+ self._task_specs = task_specs
+
+ @property
+ def projects(self):
+ """Gets the projects of this JobSpec. # noqa: E501
+
+
+ :return: The projects of this JobSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._projects
+
+ @projects.setter
+ def projects(self, projects):
+ """Sets the projects of this JobSpec.
+
+
+ :param projects: The projects of this JobSpec. # noqa: E501
+ :type: str
+ """
+
+ self._projects = projects
+
+ def to_dict(self):
+ """Returns the model properties as a dict"""
+ result = {}
+
+ for attr, _ in six.iteritems(self.swagger_types):
+ value = getattr(self, attr)
+ if isinstance(value, list):
+ result[attr] = list(map(
+ lambda x: x.to_dict() if hasattr(x, "to_dict") else x,
+ value
+ ))
+ elif hasattr(value, "to_dict"):
+ result[attr] = value.to_dict()
+ elif isinstance(value, dict):
+ result[attr] = dict(map(
+ lambda item: (item[0], item[1].to_dict())
+ if hasattr(item[1], "to_dict") else item,
+ value.items()
+ ))
+ else:
+ result[attr] = value
+ if issubclass(JobSpec, dict):
+ for key, value in self.items():
+ result[key] = value
+
+ return result
+
+ def to_str(self):
+ """Returns the string representation of the model"""
+ return pprint.pformat(self.to_dict())
+
+ def __repr__(self):
+ """For `print` and `pprint`"""
+ return self.to_str()
+
+ def __eq__(self, other):
+ """Returns true if both objects are equal"""
+ if not isinstance(other, JobSpec):
+ return False
+
+ return self.__dict__ == other.__dict__
+
+ def __ne__(self, other):
+ """Returns true if both objects are not equal"""
+ return not self == other
diff --git a/submarine-sdk/pysubmarine/submarine/job/models/job_task_spec.py b/submarine-sdk/pysubmarine/submarine/job/models/job_task_spec.py
new file mode 100644
index 0000000..901b3d4
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/models/job_task_spec.py
@@ -0,0 +1,334 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+import pprint
+import re # noqa: F401
+
+import six
+
+
+class JobTaskSpec(object):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Do not edit the class manually.
+ """
+ """
+ Attributes:
+ swagger_types (dict): The key is attribute name
+ and the value is attribute type.
+ attribute_map (dict): The key is attribute name
+ and the value is json key in definition.
+ """
+ swagger_types = {
+ 'name': 'str',
+ 'image': 'str',
+ 'cmd': 'str',
+ 'env_vars': 'dict(str, str)',
+ 'resources': 'str',
+ 'replicas': 'int',
+ 'cpu': 'str',
+ 'gpu': 'str',
+ 'memory': 'str'
+ }
+
+ attribute_map = {
+ 'name': 'name',
+ 'image': 'image',
+ 'cmd': 'cmd',
+ 'env_vars': 'envVars',
+ 'resources': 'resources',
+ 'replicas': 'replicas',
+ 'cpu': 'cpu',
+ 'gpu': 'gpu',
+ 'memory': 'memory'
+ }
+
+ def __init__(self, name=None, image=None, cmd=None, env_vars=None, resources=None, replicas=None, cpu=None, gpu=None, memory=None): # noqa: E501
+ """JobTaskSpec - a model defined in Swagger""" # noqa: E501
+ self._name = None
+ self._image = None
+ self._cmd = None
+ self._env_vars = None
+ self._resources = None
+ self._replicas = None
+ self._cpu = None
+ self._gpu = None
+ self._memory = None
+ self.discriminator = None
+ if name is not None:
+ self.name = name
+ if image is not None:
+ self.image = image
+ if cmd is not None:
+ self.cmd = cmd
+ if env_vars is not None:
+ self.env_vars = env_vars
+ if resources is not None:
+ self.resources = resources
+ if replicas is not None:
+ self.replicas = replicas
+ if cpu is not None:
+ self.cpu = cpu
+ if gpu is not None:
+ self.gpu = gpu
+ if memory is not None:
+ self.memory = memory
+
+ @property
+ def name(self):
+ """Gets the name of this JobTaskSpec. # noqa: E501
+
+
+ :return: The name of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._name
+
+ @name.setter
+ def name(self, name):
+ """Sets the name of this JobTaskSpec.
+
+
+ :param name: The name of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._name = name
+
+ @property
+ def image(self):
+ """Gets the image of this JobTaskSpec. # noqa: E501
+
+
+ :return: The image of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._image
+
+ @image.setter
+ def image(self, image):
+ """Sets the image of this JobTaskSpec.
+
+
+ :param image: The image of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._image = image
+
+ @property
+ def cmd(self):
+ """Gets the cmd of this JobTaskSpec. # noqa: E501
+
+
+ :return: The cmd of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._cmd
+
+ @cmd.setter
+ def cmd(self, cmd):
+ """Sets the cmd of this JobTaskSpec.
+
+
+ :param cmd: The cmd of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._cmd = cmd
+
+ @property
+ def env_vars(self):
+ """Gets the env_vars of this JobTaskSpec. # noqa: E501
+
+
+ :return: The env_vars of this JobTaskSpec. # noqa: E501
+ :rtype: dict(str, str)
+ """
+ return self._env_vars
+
+ @env_vars.setter
+ def env_vars(self, env_vars):
+ """Sets the env_vars of this JobTaskSpec.
+
+
+ :param env_vars: The env_vars of this JobTaskSpec. # noqa: E501
+ :type: dict(str, str)
+ """
+
+ self._env_vars = env_vars
+
+ @property
+ def resources(self):
+ """Gets the resources of this JobTaskSpec. # noqa: E501
+
+
+ :return: The resources of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._resources
+
+ @resources.setter
+ def resources(self, resources):
+ """Sets the resources of this JobTaskSpec.
+
+
+ :param resources: The resources of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._resources = resources
+
+ @property
+ def replicas(self):
+ """Gets the replicas of this JobTaskSpec. # noqa: E501
+
+
+ :return: The replicas of this JobTaskSpec. # noqa: E501
+ :rtype: int
+ """
+ return self._replicas
+
+ @replicas.setter
+ def replicas(self, replicas):
+ """Sets the replicas of this JobTaskSpec.
+
+
+ :param replicas: The replicas of this JobTaskSpec. # noqa: E501
+ :type: int
+ """
+
+ self._replicas = replicas
+
+ @property
+ def cpu(self):
+ """Gets the cpu of this JobTaskSpec. # noqa: E501
+
+
+ :return: The cpu of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._cpu
+
+ @cpu.setter
+ def cpu(self, cpu):
+ """Sets the cpu of this JobTaskSpec.
+
+
+ :param cpu: The cpu of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._cpu = cpu
+
+ @property
+ def gpu(self):
+ """Gets the gpu of this JobTaskSpec. # noqa: E501
+
+
+ :return: The gpu of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._gpu
+
+ @gpu.setter
+ def gpu(self, gpu):
+ """Sets the gpu of this JobTaskSpec.
+
+
+ :param gpu: The gpu of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._gpu = gpu
+
+ @property
+ def memory(self):
+ """Gets the memory of this JobTaskSpec. # noqa: E501
+
+
+ :return: The memory of this JobTaskSpec. # noqa: E501
+ :rtype: str
+ """
+ return self._memory
+
+ @memory.setter
+ def memory(self, memory):
+ """Sets the memory of this JobTaskSpec.
+
+
+ :param memory: The memory of this JobTaskSpec. # noqa: E501
+ :type: str
+ """
+
+ self._memory = memory
+
+ def to_dict(self):
+ """Returns the model properties as a dict"""
+ result = {}
+
+ for attr, _ in six.iteritems(self.swagger_types):
+ value = getattr(self, attr)
+ if isinstance(value, list):
+ result[attr] = list(map(
+ lambda x: x.to_dict() if hasattr(x, "to_dict") else x,
+ value
+ ))
+ elif hasattr(value, "to_dict"):
+ result[attr] = value.to_dict()
+ elif isinstance(value, dict):
+ result[attr] = dict(map(
+ lambda item: (item[0], item[1].to_dict())
+ if hasattr(item[1], "to_dict") else item,
+ value.items()
+ ))
+ else:
+ result[attr] = value
+ if issubclass(JobTaskSpec, dict):
+ for key, value in self.items():
+ result[key] = value
+
+ return result
+
+ def to_str(self):
+ """Returns the string representation of the model"""
+ return pprint.pformat(self.to_dict())
+
+ def __repr__(self):
+ """For `print` and `pprint`"""
+ return self.to_str()
+
+ def __eq__(self, other):
+ """Returns true if both objects are equal"""
+ if not isinstance(other, JobTaskSpec):
+ return False
+
+ return self.__dict__ == other.__dict__
+
+ def __ne__(self, other):
+ """Returns true if both objects are not equal"""
+ return not self == other
diff --git a/submarine-sdk/pysubmarine/submarine/job/models/json_response.py b/submarine-sdk/pysubmarine/submarine/job/models/json_response.py
new file mode 100644
index 0000000..cf86c70
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/models/json_response.py
@@ -0,0 +1,204 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+import pprint
+import re # noqa: F401
+
+import six
+
+
+class JsonResponse(object):
+ """NOTE: This class is auto generated by the swagger code generator program.
+
+ Do not edit the class manually.
+ """
+ """
+ Attributes:
+ swagger_types (dict): The key is attribute name
+ and the value is attribute type.
+ attribute_map (dict): The key is attribute name
+ and the value is json key in definition.
+ """
+ swagger_types = {
+ 'code': 'int',
+ 'success': 'bool',
+ 'result': 'object',
+ 'attributes': 'dict(str, object)'
+ }
+
+ attribute_map = {
+ 'code': 'code',
+ 'success': 'success',
+ 'result': 'result',
+ 'attributes': 'attributes'
+ }
+
+ def __init__(self, code=None, success=None, result=None, attributes=None): # noqa: E501
+ """JsonResponse - a model defined in Swagger""" # noqa: E501
+ self._code = None
+ self._success = None
+ self._result = None
+ self._attributes = None
+ self.discriminator = None
+ if code is not None:
+ self.code = code
+ if success is not None:
+ self.success = success
+ if result is not None:
+ self.result = result
+ if attributes is not None:
+ self.attributes = attributes
+
+ @property
+ def code(self):
+ """Gets the code of this JsonResponse. # noqa: E501
+
+
+ :return: The code of this JsonResponse. # noqa: E501
+ :rtype: int
+ """
+ return self._code
+
+ @code.setter
+ def code(self, code):
+ """Sets the code of this JsonResponse.
+
+
+ :param code: The code of this JsonResponse. # noqa: E501
+ :type: int
+ """
+
+ self._code = code
+
+ @property
+ def success(self):
+ """Gets the success of this JsonResponse. # noqa: E501
+
+
+ :return: The success of this JsonResponse. # noqa: E501
+ :rtype: bool
+ """
+ return self._success
+
+ @success.setter
+ def success(self, success):
+ """Sets the success of this JsonResponse.
+
+
+ :param success: The success of this JsonResponse. # noqa: E501
+ :type: bool
+ """
+
+ self._success = success
+
+ @property
+ def result(self):
+ """Gets the result of this JsonResponse. # noqa: E501
+
+
+ :return: The result of this JsonResponse. # noqa: E501
+ :rtype: object
+ """
+ return self._result
+
+ @result.setter
+ def result(self, result):
+ """Sets the result of this JsonResponse.
+
+
+ :param result: The result of this JsonResponse. # noqa: E501
+ :type: object
+ """
+
+ self._result = result
+
+ @property
+ def attributes(self):
+ """Gets the attributes of this JsonResponse. # noqa: E501
+
+
+ :return: The attributes of this JsonResponse. # noqa: E501
+ :rtype: dict(str, object)
+ """
+ return self._attributes
+
+ @attributes.setter
+ def attributes(self, attributes):
+ """Sets the attributes of this JsonResponse.
+
+
+ :param attributes: The attributes of this JsonResponse. # noqa: E501
+ :type: dict(str, object)
+ """
+
+ self._attributes = attributes
+
+ def to_dict(self):
+ """Returns the model properties as a dict"""
+ result = {}
+
+ for attr, _ in six.iteritems(self.swagger_types):
+ value = getattr(self, attr)
+ if isinstance(value, list):
+ result[attr] = list(map(
+ lambda x: x.to_dict() if hasattr(x, "to_dict") else x,
+ value
+ ))
+ elif hasattr(value, "to_dict"):
+ result[attr] = value.to_dict()
+ elif isinstance(value, dict):
+ result[attr] = dict(map(
+ lambda item: (item[0], item[1].to_dict())
+ if hasattr(item[1], "to_dict") else item,
+ value.items()
+ ))
+ else:
+ result[attr] = value
+ if issubclass(JsonResponse, dict):
+ for key, value in self.items():
+ result[key] = value
+
+ return result
+
+ def to_str(self):
+ """Returns the string representation of the model"""
+ return pprint.pformat(self.to_dict())
+
+ def __repr__(self):
+ """For `print` and `pprint`"""
+ return self.to_str()
+
+ def __eq__(self, other):
+ """Returns true if both objects are equal"""
+ if not isinstance(other, JsonResponse):
+ return False
+
+ return self.__dict__ == other.__dict__
+
+ def __ne__(self, other):
+ """Returns true if both objects are not equal"""
+ return not self == other
diff --git a/submarine-sdk/pysubmarine/submarine/job/rest.py b/submarine-sdk/pysubmarine/submarine/job/rest.py
new file mode 100644
index 0000000..0d9c379
--- /dev/null
+++ b/submarine-sdk/pysubmarine/submarine/job/rest.py
@@ -0,0 +1,337 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# coding: utf-8
+
+"""
+ Submarine Experiment API
+
+ The Submarine REST API allows you to create, list, and get experiments. TheAPI is hosted under the /v1/jobs route on the Submarine server. For example,to list experiments on a server hosted at http://localhost:8080, accesshttp://localhost:8080/api/v1/jobs/ # noqa: E501
+
+ OpenAPI spec version: 0.4.0-SNAPSHOT
+ Contact: submarine-dev@submarine.apache.org
+ Generated by: https://github.com/swagger-api/swagger-codegen.git
+"""
+
+from __future__ import absolute_import
+
+import io
+import json
+import logging
+import re
+import ssl
+
+import certifi
+# python 2 and python 3 compatibility library
+import six
+from six.moves.urllib.parse import urlencode
+
+try:
+ import urllib3
+except ImportError:
+ raise ImportError('Swagger python client requires urllib3.')
+
+
+logger = logging.getLogger(__name__)
+
+
+class RESTResponse(io.IOBase):
+
+ def __init__(self, resp):
+ self.urllib3_response = resp
+ self.status = resp.status
+ self.reason = resp.reason
+ self.data = resp.data
+
+ def getheaders(self):
+ """Returns a dictionary of the response headers."""
+ return self.urllib3_response.getheaders()
+
+ def getheader(self, name, default=None):
+ """Returns a given response header."""
+ return self.urllib3_response.getheader(name, default)
+
+
+class RESTClientObject(object):
+
+ def __init__(self, configuration, pools_size=4, maxsize=None):
+ # urllib3.PoolManager will pass all kw parameters to connectionpool
+ # https://github.com/shazow/urllib3/blob/f9409436f83aeb79fbaf090181cd81b784f1b8ce/urllib3/poolmanager.py#L75 # noqa: E501
+ # https://github.com/shazow/urllib3/blob/f9409436f83aeb79fbaf090181cd81b784f1b8ce/urllib3/connectionpool.py#L680 # noqa: E501
+ # maxsize is the number of requests to host that are allowed in parallel # noqa: E501
+ # Custom SSL certificates and client certificates: http://urllib3.readthedocs.io/en/latest/advanced-usage.html # noqa: E501
+
+ # cert_reqs
+ if configuration.verify_ssl:
+ cert_reqs = ssl.CERT_REQUIRED
+ else:
+ cert_reqs = ssl.CERT_NONE
+
+ # ca_certs
+ if configuration.ssl_ca_cert:
+ ca_certs = configuration.ssl_ca_cert
+ else:
+ # if not set certificate file, use Mozilla's root certificates.
+ ca_certs = certifi.where()
+
+ addition_pool_args = {}
+ if configuration.assert_hostname is not None:
+ addition_pool_args['assert_hostname'] = configuration.assert_hostname # noqa: E501
+
+ if maxsize is None:
+ if configuration.connection_pool_maxsize is not None:
+ maxsize = configuration.connection_pool_maxsize
+ else:
+ maxsize = 4
+
+ # https pool manager
+ if configuration.proxy:
+ self.pool_manager = urllib3.ProxyManager(
+ num_pools=pools_size,
+ maxsize=maxsize,
+ cert_reqs=cert_reqs,
+ ca_certs=ca_certs,
+ cert_file=configuration.cert_file,
+ key_file=configuration.key_file,
+ proxy_url=configuration.proxy,
+ **addition_pool_args
+ )
+ else:
+ self.pool_manager = urllib3.PoolManager(
+ num_pools=pools_size,
+ maxsize=maxsize,
+ cert_reqs=cert_reqs,
+ ca_certs=ca_certs,
+ cert_file=configuration.cert_file,
+ key_file=configuration.key_file,
+ **addition_pool_args
+ )
+
+ def request(self, method, url, query_params=None, headers=None,
+ body=None, post_params=None, _preload_content=True,
+ _request_timeout=None):
+ """Perform requests.
+
+ :param method: http request method
+ :param url: http request url
+ :param query_params: query parameters in the url
+ :param headers: http request headers
+ :param body: request json body, for `application/json`
+ :param post_params: request post parameters,
+ `application/x-www-form-urlencoded`
+ and `multipart/form-data`
+ :param _preload_content: if False, the urllib3.HTTPResponse object will
+ be returned without reading/decoding response
+ data. Default is True.
+ :param _request_timeout: timeout setting for this request. If one
+ number provided, it will be total request
+ timeout. It can also be a pair (tuple) of
+ (connection, read) timeouts.
+ """
+ method = method.upper()
+ assert method in ['GET', 'HEAD', 'DELETE', 'POST', 'PUT',
+ 'PATCH', 'OPTIONS']
+
+ if post_params and body:
+ raise ValueError(
+ "body parameter cannot be used with post_params parameter."
+ )
+
+ post_params = post_params or {}
+ headers = headers or {}
+
+ timeout = None
+ if _request_timeout:
+ if isinstance(_request_timeout, (int, ) if six.PY3 else (int, long)): # noqa: E501,F821
+ timeout = urllib3.Timeout(total=_request_timeout)
+ elif (isinstance(_request_timeout, tuple) and
+ len(_request_timeout) == 2):
+ timeout = urllib3.Timeout(
+ connect=_request_timeout[0], read=_request_timeout[1])
+
+ if 'Content-Type' not in headers:
+ headers['Content-Type'] = 'application/json'
+
+ try:
+ # For `POST`, `PUT`, `PATCH`, `OPTIONS`, `DELETE`
+ if method in ['POST', 'PUT', 'PATCH', 'OPTIONS', 'DELETE']:
+ if query_params:
+ url += '?' + urlencode(query_params)
+ if re.search('json', headers['Content-Type'], re.IGNORECASE):
+ request_body = '{}'
+ if body is not None:
+ request_body = json.dumps(body)
+ r = self.pool_manager.request(
+ method, url,
+ body=request_body,
+ preload_content=_preload_content,
+ timeout=timeout,
+ headers=headers)
+ elif headers['Content-Type'] == 'application/x-www-form-urlencoded': # noqa: E501
+ r = self.pool_manager.request(
+ method, url,
+ fields=post_params,
+ encode_multipart=False,
+ preload_content=_preload_content,
+ timeout=timeout,
+ headers=headers)
+ elif headers['Content-Type'] == 'multipart/form-data':
+ # must del headers['Content-Type'], or the correct
+ # Content-Type which generated by urllib3 will be
+ # overwritten.
+ del headers['Content-Type']
+ r = self.pool_manager.request(
+ method, url,
+ fields=post_params,
+ encode_multipart=True,
+ preload_content=_preload_content,
+ timeout=timeout,
+ headers=headers)
+ # Pass a `string` parameter directly in the body to support
+ # other content types than Json when `body` argument is
+ # provided in serialized form
+ elif isinstance(body, str):
+ request_body = body
+ r = self.pool_manager.request(
+ method, url,
+ body=request_body,
+ preload_content=_preload_content,
+ timeout=timeout,
+ headers=headers)
+ else:
+ # Cannot generate the request from given parameters
+ msg = """Cannot prepare a request message for provided
+ arguments. Please check that your arguments match
+ declared content type."""
+ raise ApiException(status=0, reason=msg)
+ # For `GET`, `HEAD`
+ else:
+ r = self.pool_manager.request(method, url,
+ fields=query_params,
+ preload_content=_preload_content,
+ timeout=timeout,
+ headers=headers)
+ except urllib3.exceptions.SSLError as e:
+ msg = "{0}\n{1}".format(type(e).__name__, str(e))
+ raise ApiException(status=0, reason=msg)
+
+ if _preload_content:
+ r = RESTResponse(r)
+
+ # In the python 3, the response.data is bytes.
+ # we need to decode it to string.
+ if six.PY3:
+ r.data = r.data.decode('utf8')
+
+ # log response body
+ logger.debug("response body: %s", r.data)
+
+ if not 200 <= r.status <= 299:
+ raise ApiException(http_resp=r)
+
+ return r
+
+ def GET(self, url, headers=None, query_params=None, _preload_content=True,
+ _request_timeout=None):
+ return self.request("GET", url,
+ headers=headers,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ query_params=query_params)
+
+ def HEAD(self, url, headers=None, query_params=None, _preload_content=True,
+ _request_timeout=None):
+ return self.request("HEAD", url,
+ headers=headers,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ query_params=query_params)
+
+ def OPTIONS(self, url, headers=None, query_params=None, post_params=None,
+ body=None, _preload_content=True, _request_timeout=None):
+ return self.request("OPTIONS", url,
+ headers=headers,
+ query_params=query_params,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+
+ def DELETE(self, url, headers=None, query_params=None, body=None,
+ _preload_content=True, _request_timeout=None):
+ return self.request("DELETE", url,
+ headers=headers,
+ query_params=query_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+
+ def POST(self, url, headers=None, query_params=None, post_params=None,
+ body=None, _preload_content=True, _request_timeout=None):
+ return self.request("POST", url,
+ headers=headers,
+ query_params=query_params,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+
+ def PUT(self, url, headers=None, query_params=None, post_params=None,
+ body=None, _preload_content=True, _request_timeout=None):
+ return self.request("PUT", url,
+ headers=headers,
+ query_params=query_params,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+
+ def PATCH(self, url, headers=None, query_params=None, post_params=None,
+ body=None, _preload_content=True, _request_timeout=None):
+ return self.request("PATCH", url,
+ headers=headers,
+ query_params=query_params,
+ post_params=post_params,
+ _preload_content=_preload_content,
+ _request_timeout=_request_timeout,
+ body=body)
+
+
+class ApiException(Exception):
+
+ def __init__(self, status=None, reason=None, http_resp=None):
+ if http_resp:
+ self.status = http_resp.status
+ self.reason = http_resp.reason
+ self.body = http_resp.data
+ self.headers = http_resp.getheaders()
+ else:
+ self.status = status
+ self.reason = reason
+ self.body = None
+ self.headers = None
+
+ def __str__(self):
+ """Custom error messages for exception"""
+ error_message = "({0})\n"\
+ "Reason: {1}\n".format(self.status, self.reason)
+ if self.headers:
+ error_message += "HTTP response headers: {0}\n".format(
+ self.headers)
+
+ if self.body:
+ error_message += "HTTP response body: {0}\n".format(self.body)
+
+ return error_message
diff --git a/submarine-sdk/pysubmarine/submarine/job/submarine_job_client.py b/submarine-sdk/pysubmarine/submarine/job/submarine_job_client.py
deleted file mode 100644
index 6aead7d..0000000
--- a/submarine-sdk/pysubmarine/submarine/job/submarine_job_client.py
+++ /dev/null
@@ -1,52 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from submarine.utils.rest_utils import http_request
-import logging
-import json
-
-_logger = logging.getLogger(__name__)
-
-_PATH_PREFIX = "/api/v1/"
-JOBS = 'jobs'
-
-
-class SubmarineJobClient:
- def __init__(self, hostname, port):
- self.base_url = 'http://' + hostname + ':' + str(port)
-
- def submit_job(self, conf_path):
- """
- Submit a job to submarine server
- :param conf_path: The location of the configuration file
- :return: requests.Response
- """
- endpoint = _PATH_PREFIX + JOBS
- with open(conf_path) as json_file:
- json_body = json.load(json_file)
- response = http_request(self.base_url, endpoint=endpoint,
- method='POST', json_body=json_body)
- return response
-
- def delete_job(self, job_id):
- """
- delete a submarine job
- :param job_id: submarine job ID
- :return: requests.Response: the detailed info about deleted job
- """
- endpoint = _PATH_PREFIX + JOBS + '/' + job_id
- response = http_request(self.base_url, endpoint=endpoint,
- method='DELETE', json_body=None)
- return response
diff --git a/submarine-sdk/pysubmarine/tests/client/test_client.py b/submarine-sdk/pysubmarine/tests/client/test_client.py
deleted file mode 100644
index 2ae59ca..0000000
--- a/submarine-sdk/pysubmarine/tests/client/test_client.py
+++ /dev/null
@@ -1,56 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from submarine.job import SubmarineJobClient
-import mock
-import pytest
-import json
-
-
-@pytest.fixture(scope="function")
-def output_json_filepath():
- dummy = {'a': 200, 'b': 2, 'c': 3}
- path = '/tmp/data.json'
- with open(path, 'w') as f:
- json.dump(dummy, f)
- return path
-
-
-@mock.patch('submarine.job.submarine_job_client.http_request')
-class TestSubmarineJobClient:
- def test_submit_job(self, mock_http_request, output_json_filepath):
- client = SubmarineJobClient('submarine', 8080)
- mock_http_request.return_value = {'jobId': 'job_1582524742595_0040',
- 'name': 'submarine', 'identifier': 'test'}
- response = client.submit_job(output_json_filepath)
-
- with open(output_json_filepath) as json_file:
- json_body = json.load(json_file)
-
- mock_http_request.assert_called_with('http://submarine:8080',
- json_body=json_body,
- endpoint='/api/v1/jobs', method='POST')
-
- assert response['jobId'] == 'job_1582524742595_0040'
- assert response['name'] == 'submarine'
- assert response['identifier'] == 'test'
-
- def test_delete_job(self, mock_http_request):
- client = SubmarineJobClient('submarine', 8080)
- client.delete_job('job_1582524742595_004')
- mock_http_request.assert_called_with('http://submarine:8080',
- json_body=None,
- endpoint='/api/v1/jobs/job_1582524742595_004',
- method='DELETE')
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org