You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by di...@apache.org on 2016/03/23 16:09:23 UTC
svn commit: r1736332 - in /incubator/singa/site/trunk/content/markdown/docs:
docker.md mesos.md
Author: dinhtta
Date: Wed Mar 23 15:09:23 2016
New Revision: 1736332
URL: http://svn.apache.org/viewvc?rev=1736332&view=rev
Log:
Update Docker and Mesos documentation
Modified:
incubator/singa/site/trunk/content/markdown/docs/docker.md
incubator/singa/site/trunk/content/markdown/docs/mesos.md
Modified: incubator/singa/site/trunk/content/markdown/docs/docker.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/docker.md?rev=1736332&r1=1736331&r2=1736332&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/docker.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/docker.md Wed Mar 23 15:09:23 2016
@@ -3,11 +3,12 @@
This guide explains how to set up a development environment for SINGA using Docker. It requires only Docker to be installed. The resulting image contains the complete working environment for SINGA. The image can then be used to set up cluster environment over one or multiple physical nodes.
1. [Build SINGA base](#build_base)
-2. [Build SINGA with Mesos and Hadoop](#build_mesos)
-3. [Pre-built images](#pre_built)
-4. [Launch and stop SINGA (stand alone mode)](#launch_stand_alone)
-5. [Launch pseudo-distributed SINGA on one node](#launch_pseudo)
-6. [Launch fully distributed SINGA on multiple nodes](#launch_distributed)
+2. [Build GPU-enabled SINGA](#build_gpu)
+3. [Build SINGA with Mesos and Hadoop](#build_mesos)
+4. [Pre-built images](#pre_built)
+5. [Launch and stop SINGA (stand alone mode)](#launch_stand_alone)
+6. [Launch pseudo-distributed SINGA on one node](#launch_pseudo)
+7. [Launch fully distributed SINGA on multiple nodes](#launch_distributed)
---
@@ -15,11 +16,11 @@ This guide explains how to set up a deve
#### Build SINGA base image
````
-$ cd tool/docker/singa
-$ sudo docker build -t singa/base .
+$ cd $SINGA_HOME/..
+$ sudo docker build -t singa/base -f incubator-singa/tool/docker/singa/Dockerfile .
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-singa/base latest XXXX XXX 2.01 GB
+singa/base latest XXXX XXX XXX GB
````
The result is the image containing a built version of SINGA.
@@ -30,14 +31,27 @@ The result is the image containing a bui
---
+<a name="build_gpu"></a>
+#### Build SINGA with GPU support
+
+````
+$ cd $SINGA_HOME/..
+$ sudo docker build -t singa/gpu -f incubator-singa/tool/docker/singa/Dockerfile_gpu .
+$ sudo docker images
+REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
+singa/gpu latest XXXX XXX XXX GB
+````
+
+---
+
<a name="build_mesos"></a>
#### Build SINGA with Mesos and Hadoop
````
-$ cd tool/docker/mesos
-$ sudo docker build -t singa/mesos .
+$ cd $SINGA_HOME/..
+$ sudo docker build -t singa/mesos -f incubator-singa/tool/docker/mesos/Dockerfile .
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-singa/mesos latest XXXX XXX 4.935 GB
+singa/mesos latest XXXX XXX XXX GB
````
@@ -83,6 +97,17 @@ $ sudo docker exec -it XYZ /bin/bash
Inside the launched container, the SINGA source directory can be found at `/root/incubator-singa`.
+**Launching GPU-enabled container**
+First, make sure that the host GPUs are up and running. The list of `NVIDIA` devices should be listed at
+`/dev/nvidiaYYY`.
+
+Next, start a new container, passing it all the devices
+
+````
+$ sudo docker run -dt --device /dev/nvidiaYYY --device /dev/nvidiaYYY ... --name XYZ singa/gpu /usr/bin/supervisord
+$ sudo docker exec -it XYZ /bin/bash
+````
+
**Stopping the container**
````
@@ -124,7 +149,24 @@ guide](mesos.html)
for details of how to start training with multiple SINGA instances.
**Important:** the container that assumes the role of Hadoop's namenode (and often Mesos's and Zookeeper's mater node as well) **must** be named `node0`. Otherwise, the user must log in to individual containers and change the Hadoop configuration separately.
-
+
+**Notes on Docker version >=1.9** Newer version of Docker adopted a built-in DNS server at the deamon. As a consequence,
+name resolution inside containers now **cannot** depend on the automatically updated `/etc/hosts` files as in version
+1.8 and earlier. Here we recommend two ways to make pseudo-distributed and distributed SINGA containers work as before
+
+1. Downgrade to docker version 1.8 and earlier
+ ````
+ $ sudo apt-get install docker-engine=1.8.3-0~trusty
+ ````
+2. Manually log in to each running container, by `sudo exec -it <name> /bin/bash`, and edit the `/etc/hosts` with the
+assigned IP addresses of all other running containers.
+
+ ````
+ node0 <ip0>
+ node1 <ip1>
+ ...
+ ````
+
---
<a name="launch_distributed"></a>
Modified: incubator/singa/site/trunk/content/markdown/docs/mesos.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/docs/mesos.md?rev=1736332&r1=1736331&r2=1736332&view=diff
==============================================================================
--- incubator/singa/site/trunk/content/markdown/docs/mesos.md (original)
+++ incubator/singa/site/trunk/content/markdown/docs/mesos.md Wed Mar 23 15:09:23 2016
@@ -32,6 +32,9 @@ To check if the setup has been successfu
hadoop dfsadmin -report
````
+**_Important_** If the Docker version is 1.9 or newer, make sure [name resolution is set up
+properly](docker.html#launch_pseudo)
+
#### Mesos logs
Mesos logs are stored at `/opt/lt-mesos-master.INFO` on `node0` and `/opt/lt-mesos-slave.INFO` at other nodes.