You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2021/12/14 03:58:10 UTC
[GitHub] [submarine] atosystem opened a new pull request #831: Submarine 1117. Connect API for CLI Experiments
atosystem opened a new pull request #831:
URL: https://github.com/apache/submarine/pull/831
### What is this PR for?
1. Implement API with CLI for experiments with nice format and async fetching status
2. Implement e2e test for CLI experiments
### What type of PR is it?
[Feature]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1117
### How should this be tested?
e2e test is implemented
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r773542189
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ result = result.result
+
+ json_data = richJSON.from_data(result)
+ console.print(Panel(json_data, title="Experiment(id = {} )".format(id)))
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def delete_experiment(id):
Review comment:
Add the subcommand `wait` if the user wants to wait until the resource is deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768529224
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
Review comment:
OK! But, I think that 6 sec will be enough. The time for fetching API for experiments will not last too long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768535205
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
OK!
But some of the values are not available upon creating the experiment.
Do I have to wait for the experiment to finish?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r773347944
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ result = result.result
+
+ json_data = richJSON.from_data(result)
+ console.print(Panel(json_data, title="Experiment(id = {} )".format(id)))
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def delete_experiment(id):
Review comment:
Actually, I think that the server-side java code is already implemented in a blocking manner?
What I do is just merely check if the deletion process is successful.
In a normal situation, I will get the response only if `status` == `Deleted `.
If the API responds with the status not equal to `Deleted`, it doesn't mean that the deletion process is still continuing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768535205
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
OK! @KUAN-HSUN-LI
But some of the values are not available upon creating the experiment.
Do I have to wait for the experiment to finish?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768487079
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
Review comment:
Also, set a timeout value, maybe 10 secs or 30 secs?
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
Review comment:
same as above
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
+ # test get experiment
+ result = runner.invoke(main.entry_point, ["get", "experiment", experiment["experimentId"]])
+ assert "Experiment(id = {} )".format(experiment["experimentId"]) in result.output
+ assert experiment["spec"]["environment"]["image"] in result.output
+
+ # test delete experiment
+ result = runner.invoke(main.entry_point, ["delete", "experiment", experiment["experimentId"]])
+ assert "Experiment(id = {} ) deleted".format(experiment["experimentId"]) in result.output
Review comment:
Also, test the get fails after the experiment is deleted.
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
Also, check the "finishedTime", "createdTime", "runningTime" and status are not empty.
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
Review comment:
what is "191" mean? column width?
If it is a column width make it a const variable
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ result = result.result
+
+ json_data = richJSON.from_data(result)
+ console.print(Panel(json_data, title="Experiment(id = {} )".format(id)))
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def delete_experiment(id):
Review comment:
@pingsutw, Do you think it is necessary to block the cli until the resources in k8s are correctly deleted?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768535205
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
OK!
But the `finishedTime` should be empty since the experiment is still running?
Do I have to wait for the experiment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r777295107
##########
File path: submarine-sdk/pysubmarine/submarine/client/api/experiment_client.py
##########
@@ -14,26 +14,27 @@
# limitations under the License.
import logging
-import os
import time
+from submarine.cli.config.config import loadConfig
from submarine.client.api.experiment_api import ExperimentApi
from submarine.client.api_client import ApiClient
from submarine.client.configuration import Configuration
logger = logging.getLogger(__name__)
logging.basicConfig(format="%(message)s")
logging.getLogger().setLevel(logging.INFO)
+submarineCliConfig = loadConfig()
def generate_host():
"""
Generate submarine host
:return: submarine host
"""
- submarine_server_dns_name = str(os.environ.get("SUBMARINE_SERVER_DNS_NAME"))
- submarine_server_port = str(os.environ.get("SUBMARINE_SERVER_PORT"))
- host = submarine_server_dns_name + ":" + submarine_server_port
+ submarine_server_dns_name = str(submarineCliConfig.connection.hostname)
+ submarine_server_port = str(submarineCliConfig.connection.port)
+ host = "http://" + submarine_server_dns_name + ":" + submarine_server_port
Review comment:
Don't change this function. In k8s, pods can connect through the DNS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768551722
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
+ # test get experiment
+ result = runner.invoke(main.entry_point, ["get", "experiment", experiment["experimentId"]])
+ assert "Experiment(id = {} )".format(experiment["experimentId"]) in result.output
+ assert experiment["spec"]["environment"]["image"] in result.output
+
+ # test delete experiment
+ result = runner.invoke(main.entry_point, ["delete", "experiment", experiment["experimentId"]])
+ assert "Experiment(id = {} ) deleted".format(experiment["experimentId"]) in result.output
Review comment:
Got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] pingsutw commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
pingsutw commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r772094351
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ result = result.result
+
+ json_data = richJSON.from_data(result)
+ console.print(Panel(json_data, title="Experiment(id = {} )".format(id)))
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def delete_experiment(id):
Review comment:
yes, we should, but it should be optional. For example, `submarine delete experiment id --wait`. If users didn't use `wait` in the command, we should not block the cli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r784620933
##########
File path: submarine-sdk/pysubmarine/submarine/client/api/experiment_client.py
##########
@@ -34,6 +34,7 @@ def generate_host():
submarine_server_dns_name = str(os.environ.get("SUBMARINE_SERVER_DNS_NAME"))
submarine_server_port = str(os.environ.get("SUBMARINE_SERVER_PORT"))
host = submarine_server_dns_name + ":" + submarine_server_port
+ host = "http://" + submarine_server_dns_name + ":" + submarine_server_port
Review comment:
Remove the first one. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-1012867048
> @KUAN-HSUN-LI I found out that calling wait for experiment to finish in e2e test causes the github action to fail. I do not know the reason for this error. It works well on my computer.
>
> So, currently, I do not wait for the experiment to finish and execute my test. I neglect the columns that are not available upon calling `submarine experiment list`. I think it is not necessary to check all the columns? Because if some values are missing calling `submarine experiment list`, it should be something wrong in `experiment api client ` or other, but not my cli code.
The experiment pod cannot reach the `Succeeded` status in the test and the test will stop until timeout. I am also not sure what happens in the CICD therefore I think we can skip the `wait` in it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-1007921289
@KUAN-HSUN-LI I found out that calling wait for experiment to finish in e2e test causes the github action to fail. I do not know the reason for this error. It works well on my computer.
So, currently, I do not wait for the experiment to finish and execute my test. I neglect the columns that are not available upon calling `submarine experiment list`. I think it is not necessary to check all the columns? Because if some values are missing calling `submarine experiment list`, it should be something wrong in `experiment api client ` or other, but not my cli code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on pull request #831: Submarine 1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-993145220
@KUAN-HSUN-LI please review my code, Thanks!
I decide to split other commands (for environment and notebook) in future PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768529742
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
+
+ result = thread.get()
+ results = result.result
+
+ results = list(
+ map(
+ lambda r: [
+ r["spec"]["meta"]["name"],
+ r["experimentId"],
+ ",".join(r["spec"]["meta"]["tags"]),
+ r["finishedTime"],
+ r["createdTime"],
+ r["runningTime"],
+ r["status"],
+ ],
+ results,
+ )
+ )
+
+ table = Table(title="List of Experiments")
+
+ for col in COLS_TO_SHOW:
+ table.add_column(col, overflow="fold")
+ for res in results:
+ table.add_row(*res)
+
+ console.print(table)
+
+ except ApiException as err:
+ if err.body is not None:
+ errbody = json.loads(err.body)
+ click.echo("[Api Error] {}".format(errbody["message"]))
+ else:
+ click.echo("[Api Error] {}".format(err))
@click.command("experiment")
@click.argument("id")
def get_experiment(id):
"""Get experiments"""
- click.echo("get experiment! id={}".format(id))
+ console = Console()
+ try:
+ thread = experimentClient.get_experiment_async(id)
+ with console.status("[bold green] Fetching Experiment(id = {} )...".format(id)):
+ while not thread.ready():
+ sleep(1)
Review comment:
Got it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on pull request #831: Submarine 1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-993209478
@atosystem Thanks for your contribution.
The pull request title must follow the following format:
SUBMARINE-${jira_number}. ${jira_summary}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] asfgit closed pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #831:
URL: https://github.com/apache/submarine/pull/831
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on pull request #831: Submarine-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-993324450
Sorry for the mistake~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768522557
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
Review comment:
191 is the width of the console for testing
If the width is not big enough, the values in the table will be split into multiple columns, making it difficult to conduct e2e test.
I will make it constant variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r775915668
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,132 @@
under the License.
"""
+import json
+import time
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.client.api.experiment_client import ExperimentClient
+from submarine.client.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
Review comment:
The default port should be 32080 or use the API client path from the config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r769175198
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
@KUAN-HSUN-LI In the last commit, I have added codes to test the output of CLI after the created experiment is finished.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768544597
##########
File path: submarine-sdk/pysubmarine/submarine/cli/experiment/command.py
##########
@@ -15,24 +15,115 @@
under the License.
"""
+import json
+from time import sleep
+
import click
+from rich.console import Console
+from rich.json import JSON as richJSON
+from rich.panel import Panel
+from rich.table import Table
+
+from submarine.experiment.api.experiment_client import ExperimentClient
+from submarine.experiment.exceptions import ApiException
+
+experimentClient = ExperimentClient("http://localhost:8080")
@click.command("experiment")
def list_experiment():
"""List experiments"""
- click.echo("list experiment!")
+ COLS_TO_SHOW = ["Name", "Id", "Tags", "Finished Time", "Created Time", "Running Time", "Status"]
+ console = Console()
+ try:
+ thread = experimentClient.list_experiments_async()
+ with console.status("[bold green] Fetching Experiments..."):
+ while not thread.ready():
+ sleep(1)
Review comment:
OK!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] KUAN-HSUN-LI commented on a change in pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
KUAN-HSUN-LI commented on a change in pull request #831:
URL: https://github.com/apache/submarine/pull/831#discussion_r768851387
##########
File path: submarine-sdk/pysubmarine/tests/cli/test_experiment.py
##########
@@ -13,29 +13,58 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+import pytest
from click.testing import CliRunner
+import submarine
from submarine.cli import main
+from submarine.experiment.models.code_spec import CodeSpec
+from submarine.experiment.models.environment_spec import EnvironmentSpec
+from submarine.experiment.models.experiment_meta import ExperimentMeta
+from submarine.experiment.models.experiment_spec import ExperimentSpec
+from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec
-def test_list_experiment():
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["list", "experiment"])
- assert result.exit_code == 0
- assert "list experiment!" in result.output
+@pytest.mark.e2e
+def test_all_experiment_e2e():
+ submarine_client = submarine.ExperimentClient(host="http://localhost:8080")
+ environment = EnvironmentSpec(image="apache/submarine:tf-dist-mnist-test-1.0")
+ experiment_meta = ExperimentMeta(
+ name="mnist-dist",
+ namespace="default",
+ framework="Tensorflow",
+ cmd="python /var/tf_dist_mnist/dist_mnist.py --train_steps=100",
+ env_vars={"ENV1": "ENV1"},
+ )
+ worker_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
+ ps_spec = ExperimentTaskSpec(resources="cpu=1,memory=1024M", replicas=1)
-def test_get_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["get", "experiment", mock_experiment_id])
- assert result.exit_code == 0
- assert "get experiment! id={}".format(mock_experiment_id) in result.output
+ code_spec = CodeSpec(sync_mode="git", url="https://github.com/apache/submarine.git")
+
+ experiment_spec = ExperimentSpec(
+ meta=experiment_meta,
+ environment=environment,
+ code=code_spec,
+ spec={"Ps": ps_spec, "Worker": worker_spec},
+ )
+ experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
-def test_delete_experiment():
- mock_experiment_id = "0"
- runner = CliRunner()
- result = runner.invoke(main.entry_point, ["delete", "experiment", mock_experiment_id])
+ # set env to display full table
+ runner = CliRunner(env={"COLUMNS": "191"})
+ # test list experiment
+ result = runner.invoke(main.entry_point, ["list", "experiment"])
assert result.exit_code == 0
- assert "delete experiment! id={}".format(mock_experiment_id) in result.output
+ assert "List of Experiments" in result.output
+ assert experiment["experimentId"] in result.output
+ assert experiment["spec"]["meta"]["name"] in result.output
+
Review comment:
If it can completely test the cli.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] atosystem commented on pull request #831: SUBMARINE-1117. Connect API for CLI Experiments
Posted by GitBox <gi...@apache.org>.
atosystem commented on pull request #831:
URL: https://github.com/apache/submarine/pull/831#issuecomment-1007921289
@KUAN-HSUN-LI I found out that calling wait for experiment to finish in e2e test causes the github action to fail. I do not know the reason for this error. It works well on my computer.
So, currently, I do not wait for the experiment to finish and execute my test. I neglect the columns that are not available upon calling `submarine experiment list`. I think it is not necessary to check all the columns? Because if some values are missing calling `submarine experiment list`, it should be something wrong in `experiment api client ` or other, but not my cli code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org