You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2022/01/03 12:12:23 UTC
[GitHub] [submarine] KUAN-HSUN-LI opened a new pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
KUAN-HSUN-LI opened a new pull request #860:
URL: https://github.com/apache/submarine/pull/860
### What is this PR for?
Create the Python API and CLI command for serving a registered model.
### What type of PR is it?
[Feature]
### Todos
* [x] - Save Pytorch model via torchscript
* [x] - Generate the Serve API
* [ ] - Transfer the model description file to the config which is used in the Triton Inference Server. (It will be handled in this issue https://issues.apache.org/jira/browse/SUBMARINE-1173)
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1157
### How should this be tested?
The tests will provide in another PR because the transfer of the config is not finished.
### Screenshots (if appropriate)
https://user-images.githubusercontent.com/38066413/147928471-7e8cc70b-99df-4b49-89bb-bc1627bbb885.mp4
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #860:
URL: https://github.com/apache/submarine/pull/860#discussion_r777775564
##########
File path: submarine-sdk/pysubmarine/submarine/cli/main.py
##########
@@ -67,6 +73,11 @@ def cmdgrp_config():
cmdgrp_list.add_command(environment_cmd.list_environment)
cmdgrp_get.add_command(environment_cmd.get_environment)
cmdgrp_delete.add_command(environment_cmd.delete_environment)
+# # serve
Review comment:
[Trivial] Delete one '#'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on pull request #860:
URL: https://github.com/apache/submarine/pull/860#issuecomment-1004052025
@jeff-901 @atosystem Please help me review the PR. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] asfgit closed pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #860:
URL: https://github.com/apache/submarine/pull/860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] jeff-901 commented on a change in pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
jeff-901 commented on a change in pull request #860:
URL: https://github.com/apache/submarine/pull/860#discussion_r777804471
##########
File path: submarine-sdk/pysubmarine/submarine/cli/serve/command.py
##########
@@ -0,0 +1,124 @@
+"""
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+"""
+import json
+import time
+
+import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+
+from submarine.cli.config.config import loadConfig
+from submarine.client.api.serve_client import ServeClient
+from submarine.client.api_client import ApiException
+
+submarineCliConfig = loadConfig()
+serveClient = ServeClient(
+ f"http://{submarineCliConfig.connection.hostname}:{submarineCliConfig.connection.port}"
+ if submarineCliConfig
+ else "http://localhost:32080"
+)
+
+POLLING_INTERVAL = 1 # sec
+TIMEOUT = 30 # sec
+
+
+@click.command("serve")
+def list_serve():
+ """List serves"""
+ click.echo("The command is not supported yet.") # TODO(kuanhsun)
+ click.echo("list serve!")
+
+
+@click.command("serve")
+@click.argument("model_name")
+@click.argument("model_version", type=int)
+def get_serve(model_name: str, model_version: int):
+ """Get serve"""
+ click.echo("The command is not supported yet.") # TODO(kuanhsun)
+ click.echo(f"get serve! model name: {model_name}, model version: {model_version}")
+
+
+@click.command("serve")
+@click.argument("model_name")
+@click.argument("model_version", type=int)
+def create_serve(model_name: str, model_version: int):
+ """Create serve"""
+ console = Console()
+ try:
+ thread = serveClient.create_serve(model_name, model_version, async_req=True)
+ timeout = time.time() + TIMEOUT
+ with console.status(
+ f"[bold green] Creating Serve with name: {model_name}, version: {model_version}"
+ ):
+ while not thread.ready():
+ time.sleep(POLLING_INTERVAL)
+ if time.time() > timeout:
+ console.print("[bold red] Timeout!")
+ return
+ result = thread.get()
+ click.echo(result)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo(f"[Api Error] {errbody['message']}")
+ else:
+ click.echo(f"[Api Error] {err}")
+
+
+@click.command("serve")
+@click.argument("model_name")
+@click.argument("model_version", type=int)
+@click.option("--wait", is_flag=True, default=False)
+def delete_serve(model_name: str, model_version: int, wait: bool):
+ """Delete serve"""
+ console = Console()
+ try:
+ thread = serveClient.delete_serve(model_name, model_version, async_req=True)
+ timeout = time.time() + TIMEOUT
+ with console.status(
+ f"[bold green] Deleting Serve with name: {model_name}, version: {model_version}"
+ ):
+ while not thread.ready():
+ time.sleep(POLLING_INTERVAL)
+ if time.time() > timeout:
+ console.print("[bold red] Timeout!")
+ return
+
+ result = thread.get()
+ click.echo(result)
+
+ if wait:
+ if result["status"] == "Deleted":
+ console.print(
+ f"[bold green] Serve: model name:{model_name}, version: {model_version}"
Review comment:
Maybe we can console "f"[bold green] Serve Deleted: model name:{model_name}, version: {model_version}""
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] asfgit closed pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #860:
URL: https://github.com/apache/submarine/pull/860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #860: SUBMARINE-1157. Serving model via Python API or CLI
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #860:
URL: https://github.com/apache/submarine/pull/860#discussion_r777775564
##########
File path: submarine-sdk/pysubmarine/submarine/cli/main.py
##########
@@ -67,6 +73,11 @@ def cmdgrp_config():
cmdgrp_list.add_command(environment_cmd.list_environment)
cmdgrp_get.add_command(environment_cmd.get_environment)
cmdgrp_delete.add_command(environment_cmd.delete_environment)
+# # serve
Review comment:
[Trivial] Delete one '#'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org