You are viewing a plain text version of this content. The canonical link for it is here.
Posted to submarine-dev@hadoop.apache.org by li...@apache.org on 2019/10/20 02:23:45 UTC
[hadoop-submarine] branch master updated: SUBMARINE-213. [SDK] Add
pysubmarine-tracking example in mini-submarine
This is an automated email from the ASF dual-hosted git repository.
liuxun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 399adc1 SUBMARINE-213. [SDK] Add pysubmarine-tracking example in mini-submarine
399adc1 is described below
commit 399adc1b4d734fcb636caef5d310034cdd41f8e7
Author: pingsutw <pi...@gmail.com>
AuthorDate: Sat Oct 19 00:35:47 2019 +0800
SUBMARINE-213. [SDK] Add pysubmarine-tracking example in mini-submarine
### What is this PR for?
Add a tracking api example in mini-submarine
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-213
### How should this be tested?
https://travis-ci.org/pingsutw/hadoop-submarine/builds/599649334
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: pingsutw <pi...@gmail.com>
Author: HUAN-PING SU <pi...@gmail.com>
Closes #55 from pingsutw/SUBMARINE-213 and squashes the following commits:
0218786 [HUAN-PING SU] Update tracking.md
881c051 [pingsutw] SUBMARINE-213. [SDK] Add pysubmarine-tracking example in mini-submarine
---
dev-support/mini-submarine/Dockerfile | 1 +
dev-support/mini-submarine/build_mini-submarine.sh | 1 +
.../submarine/build_python_virtual_env.sh | 1 +
.../mini-submarine/submarine/mnist_distributed.py | 6 ++++++
docs/submarine-sdk/pysubmarine/tracking.md | 12 +++++++++++-
.../pysubmarine/submarine/store/database/models.py | 20 ++++++++++----------
6 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/dev-support/mini-submarine/Dockerfile b/dev-support/mini-submarine/Dockerfile
index 70c5128..1f65149 100644
--- a/dev-support/mini-submarine/Dockerfile
+++ b/dev-support/mini-submarine/Dockerfile
@@ -116,6 +116,7 @@ ENV IMAGE_N=${IMAGE_NAME}
ARG SUBMARINE_VERSION=
ENV SUBMARINE_VER=${SUBMARINE_VERSION}
ADD submarine-dist-${SUBMARINE_VER}*.tar.gz /opt
+ADD pysubmarine /opt/pysubmarine
RUN ln -s /opt/submarine-dist-${SUBMARINE_VER}* /opt/submarine-current
ADD submarine /home/yarn/submarine
ADD database /home/yarn/database
diff --git a/dev-support/mini-submarine/build_mini-submarine.sh b/dev-support/mini-submarine/build_mini-submarine.sh
index f5df7c7..d83d1fa 100755
--- a/dev-support/mini-submarine/build_mini-submarine.sh
+++ b/dev-support/mini-submarine/build_mini-submarine.sh
@@ -58,6 +58,7 @@ if [[ -z "${submarine_dist_exists}" ]]; then
fi
cp submarine-dist/target/submarine-dist-${submarine_v}*.tar.gz ${MINI_PATH}
+cp -r submarine-sdk/pysubmarine ${MINI_PATH}
cp -r docs/database ${MINI_PATH}
cd ${MINI_PATH}
diff --git a/dev-support/mini-submarine/submarine/build_python_virtual_env.sh b/dev-support/mini-submarine/submarine/build_python_virtual_env.sh
index 0bb6088..4ecfc01 100755
--- a/dev-support/mini-submarine/submarine/build_python_virtual_env.sh
+++ b/dev-support/mini-submarine/submarine/build_python_virtual_env.sh
@@ -21,6 +21,7 @@ tar xf virtualenv-16.0.0.tar.gz
python3 virtualenv-16.0.0/virtualenv.py venv
. venv/bin/activate
pip3 install tensorflow==1.13.1
+pip3 install /opt/pysubmarine/.
zip -r myvenv.zip venv
deactivate
diff --git a/dev-support/mini-submarine/submarine/mnist_distributed.py b/dev-support/mini-submarine/submarine/mnist_distributed.py
index 29d6b75..58a9a34 100644
--- a/dev-support/mini-submarine/submarine/mnist_distributed.py
+++ b/dev-support/mini-submarine/submarine/mnist_distributed.py
@@ -37,6 +37,7 @@ import sys
import tensorboard.program as tb_program
import tensorflow as tf
+import submarine
# Environment variable containing port to launch TensorBoard on, set by TonY.
TB_PORT_ENV_VAR = 'TB_PORT'
@@ -247,7 +248,10 @@ def main(_):
# Train
logging.info('Starting training')
i = 0
+ worker_index = "worker-" + str(task_index)
while not sess.should_stop():
+ # Before use submarine-sdk, start Mysql server first
+ # submarine.log_param("batch_size", FLAGS.batch_size, worker_index)
batch = mnist.train.next_batch(FLAGS.batch_size)
if i % 100 == 0:
step, _, train_accuracy = sess.run(
@@ -256,6 +260,8 @@ def main(_):
keep_prob: 1.0})
logging.info('Step %d, training accuracy: %g' % (
step, train_accuracy))
+ # Before use submarine-sdk, start Mysql server first
+ # submarine.log_metric("accuracy", train_accuracy, worker_index, i)
else:
sess.run([global_step, train_step],
feed_dict={features: batch[0], labels: batch[1],
diff --git a/docs/submarine-sdk/pysubmarine/tracking.md b/docs/submarine-sdk/pysubmarine/tracking.md
index abb0134..056e0ef 100644
--- a/docs/submarine-sdk/pysubmarine/tracking.md
+++ b/docs/submarine-sdk/pysubmarine/tracking.md
@@ -20,6 +20,16 @@
- [TODO] (WEB) Metric tracking ui in workbench-web
- [TODO] (WEB) Metric graphical display in workbench-web
+## Quickstart
+1. [Start mini-submarine](../../../dev-support/mini-submarine#run-mini-submarine-image)
+
+2. [Start Mysql server in mini-submarine](../../../dev-support/mini-submarine#run-workbench-server)
+
+3. Uncomment the log_param and log_metric in
+[mnist_distributed.py](../../../dev-support/mini-submarine/submarine/mnist_distributed.py)
+
+4. Start Submarine job (e.g., run_submarine_mnist_tony.sh)
+
## Functions
### `submarine.get_tracking_uri()`
@@ -60,4 +70,4 @@ logs a single key-value parameter. The key and value are both strings.
**Parameters**
- key - Parameter name (string).
- value - Parameter value (string).
-- worker_index - The index of worker (string). Examples are "rank-1", "worker-2"
\ No newline at end of file
+- worker_index - The index of worker (string). Examples are "rank-1", "worker-2"
diff --git a/submarine-sdk/pysubmarine/submarine/store/database/models.py b/submarine-sdk/pysubmarine/submarine/store/database/models.py
index c63dc2e..6f501db 100644
--- a/submarine-sdk/pysubmarine/submarine/store/database/models.py
+++ b/submarine-sdk/pysubmarine/submarine/store/database/models.py
@@ -37,17 +37,17 @@ Base = declarative_base()
class SqlMetric(Base):
__tablename__ = 'metrics'
- key = Column(String(250))
+ key = Column(String(190))
"""
- Metric key: `String` (limit 250 characters). Part of *Primary Key* for ``metrics`` table.
+ Metric key: `String` (limit 190 characters). Part of *Primary Key* for ``metrics`` table.
"""
value = Column(sa.types.Float(precision=53), nullable=False)
"""
Metric value: `Float`. Defined as *Non-null* in schema.
"""
- worker_index = Column(String(250))
+ worker_index = Column(String(32))
"""
- Metric worker_index: `String` (limit 250 characters). Part of *Primary Key* for
+ Metric worker_index: `String` (limit 32 characters). Part of *Primary Key* for
``metrics`` table.
"""
timestamp = Column(BigInteger, default=lambda: int(time.time()))
@@ -102,17 +102,17 @@ class SqlMetric(Base):
class SqlParam(Base):
__tablename__ = 'params'
- key = Column(String(250))
+ key = Column(String(190))
"""
- Param key: `String` (limit 250 characters). Part of *Primary Key* for ``params`` table.
+ Param key: `String` (limit 190 characters). Part of *Primary Key* for ``params`` table.
"""
- value = Column(String(250), nullable=False)
+ value = Column(String(190), nullable=False)
"""
- Param value: `String` (limit 250 characters). Defined as *Non-null* in schema.
+ Param value: `String` (limit 190 characters). Defined as *Non-null* in schema.
"""
- worker_index = Column(String(250), nullable=False)
+ worker_index = Column(String(32), nullable=False)
"""
- Param worker_index: `String` (limit 250 characters). Part of *Primary Key* for
+ Param worker_index: `String` (limit 32 characters). Part of *Primary Key* for
``metrics`` table.
"""
job_name = Column(String(32))