You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2015/07/27 23:02:05 UTC
svn commit: r959809 - in /websites/staging/slider/trunk/content: ./
docs/slider_specs/application_pkg_docker.html
Author: buildbot
Date: Mon Jul 27 21:02:04 2015
New Revision: 959809
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Mon Jul 27 21:02:04 2015
@@ -1 +1 @@
-1688925
+1692951
Modified: websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/application_pkg_docker.html Mon Jul 27 21:02:04 2015
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,8 +196,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-docker-based-application-packaging-support-tech-preview">Apache Slider Docker Based Application Packaging Support (Tech Preview)</h1>
-<h2 id="introduction">Introduction</h2>
+<h1 id="apache-slider-docker-based-application-packaging-support-tech-preview">Apache Slider Docker Based Application Packaging Support (Tech Preview)<a class="headerlink" href="#apache-slider-docker-based-application-packaging-support-tech-preview" title="Permanent link">¶</a></h1>
+<h2 id="introduction">Introduction<a class="headerlink" href="#introduction" title="Permanent link">¶</a></h2>
<p>Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications.</p>
<ul>
<li>Users can use their existing Docker images without any modifications.</li>
@@ -194,20 +205,20 @@ Latest release: <strong>0.80.0-incubatin
<li>Slider can deploy their Docker containers within YARN cluster, with resource management capability from YARN</li>
<li>They can use Slider to monitor and manage their Docker containers/applications</li>
</ul>
-<h2 id="terminology">Terminology</h2>
-<h3 id="docker-image">Docker Image</h3>
+<h2 id="terminology">Terminology<a class="headerlink" href="#terminology" title="Permanent link">¶</a></h2>
+<h3 id="docker-image">Docker Image<a class="headerlink" href="#docker-image" title="Permanent link">¶</a></h3>
<ul>
<li>Applications definition file</li>
<li>Is composed of one or multiple layers</li>
<li>Users create Docker containers out of Docker images</li>
</ul>
-<h3 id="docker-container">Docker Container</h3>
+<h3 id="docker-container">Docker Container<a class="headerlink" href="#docker-container" title="Permanent link">¶</a></h3>
<ul>
<li>A running instance of a Docker image</li>
<li>Comprises the application and its dependencies</li>
<li>Enjoys the resource isolation and allocation benefits of VMs but is much more portable and efficient.</li>
</ul>
-<h2 id="specifications">Specifications</h2>
+<h2 id="specifications">Specifications<a class="headerlink" href="#specifications" title="Permanent link">¶</a></h2>
<p>Refer to <a href="https://www.docker.com/whatisdocker/"><em>What is Docker</em></a> for a quick overview of Docker and why it excels VM.</p>
<p><strong>Goal</strong></p>
<ul>
@@ -235,90 +246,89 @@ Latest release: <strong>0.80.0-incubatin
<p>Below is an example of how we can use Slider to deploy a multi-component Dockerized application</p>
-<p><strong>appConfig.json</strong></p>
+<p><strong>metainfo.json</strong></p>
<div class="codehilite"><pre><span class="p">{</span>
- "<span class="n">schema</span>"<span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
- "<span class="n">metadata</span>"<span class="p">:</span> <span class="p">{</span> <span class="p">},</span>
- "<span class="k">global</span>"<span class="p">:</span> <span class="p">{</span> <span class="p">},</span>
- "<span class="n">components</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">REDIS</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">memcached</span><span class="p">.</span><span class="n">commandPath</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">user</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">docker</span>"<span class="p">,</span>
- "<span class="n">memcached</span><span class="p">.</span><span class="n">options</span>"<span class="p">:</span> "<span class="o">-</span><span class="n">d</span> <span class="o">-</span><span class="n">e</span> <span class="n">REDIS_PASS</span><span class="p">=</span><span class="o">\</span>"<span class="o">**</span><span class="n">None</span><span class="o">**\</span>""<span class="p">,</span>
- "<span class="n">memcached</span><span class="p">.</span><span class="n">statusCommand</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">docker</span> <span class="n">ps</span>"<span class="p">,</span>
- "<span class="n">memcached</span><span class="p">.</span><span class="n">inputFiles</span>"<span class="p">:</span> <span class="p">[</span>
+"<span class="n">schemaVersion</span>"<span class="p">:</span> "2<span class="p">.</span>1"<span class="p">,</span>
+"<span class="n">application</span>"<span class="p">:</span> <span class="p">{</span>
+ "<span class="n">name</span>"<span class="p">:</span> "<span class="n">NODEJS</span><span class="o">-</span><span class="n">REDIS</span>"<span class="p">,</span>
+ "<span class="n">components</span>"<span class="p">:</span> <span class="p">[</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span class="p">:</span> "<span class="n">NODEJS</span>"<span class="p">,</span>
+ "<span class="n">type</span>"<span class="p">:</span> "<span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">dockerContainers</span>"<span class="p">:</span> <span class="p">[</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span class="p">:</span> "<span class="n">nodejs</span>"<span class="p">,</span>
+ "<span class="n">commandPath</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">image</span>"<span class="p">:</span> "<span class="n">rsahahw</span><span class="o">/</span><span class="n">centos</span><span class="o">-</span><span class="n">node</span><span class="o">-</span><span class="n">redis</span>"<span class="p">,</span>
+ "<span class="n">ports</span>"<span class="p">:</span> <span class="p">[{</span>
+ "<span class="n">containerPort</span>" <span class="p">:</span> "8000"
+
+ <span class="p">}]</span>
+ <span class="p">}</span>
+ <span class="p">]</span>
+ <span class="p">},</span>
+ <span class="p">{</span>
+ "<span class="n">name</span>"<span class="p">:</span> "<span class="n">REDIS</span>"<span class="p">,</span>
+ "<span class="n">type</span>"<span class="p">:</span> "<span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">dockerContainers</span>"<span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
- "<span class="n">containerPath</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">tmp</span><span class="o">/</span><span class="n">input</span>"<span class="p">,</span>
- "<span class="n">fileLocalPath</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">Users</span><span class="o">/</span><span class="n">peter</span><span class="o">/</span><span class="n">config</span><span class="p">.</span><span class="n">json</span>"
- <span class="p">}</span>
- <span class="p">]</span>
- <span class="p">}</span>
+ "<span class="n">name</span>"<span class="p">:</span> "<span class="n">redis</span>"<span class="p">,</span>
+ "<span class="n">commandPath</span>"<span class="p">:</span> "<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">docker</span>"<span class="p">,</span>
+ "<span class="n">image</span>"<span class="p">:</span> "<span class="n">tutum</span><span class="o">/</span><span class="n">redis</span>"<span class="p">,</span>
+ "<span class="n">ports</span>"<span class="p">:</span> <span class="p">[{</span>
+ "<span class="n">containerPort</span>" <span class="p">:</span> "6379"<span class="p">,</span>
+ "<span class="n">hostPort</span>"<span class="p">:</span> "6379"
+
+ <span class="p">}]</span>
+
+ <span class="p">}]</span>
+ <span class="p">}</span>
+
+ <span class="p">]</span>
<span class="p">}</span>
+
<span class="p">}</span>
</pre></div>
-<p>Please note that in this example, we are specifying a different docker command path than the default <code>/usr/bin/docker</code> in appConfig.json. We also specify the options that we need to include in the <code>docker run</code> command (<code>-d</code> is by default included). This is just a demo as how to include docker run command options. We also need to expose a port to listen on for the application. Thus, we are specifying <code>site.global.listen_port</code> in appConfig.json and the export config in metainfo.json below</p>
-<p>The config structure in appConfig.json(map) -> components(list) -> component(map) -> containers(list)-> container(map) have to match the one in metainfo.json. </p>
-<p>We are also adding a few configurations for the component <em>memcached</em> in appConfig.json here. These configurations are runtime parameters of the component, as opposed to being part of the Docker image definition in metainfo.json.</p>
-<p><strong>metainfo.json</strong></p>
+<p>Metainfo.json is the specification about how your image should run. All fields in the config file are not different than those in a non Docker application, except for those under "dockerContainers". All fields under "dockerContainers" are required unless specified "optional" and they described as below:</p>
+<ul>
+<li><strong>name</strong>: Name of your container. It doesn't affect how you launch your application, but Slider will use it to identify the overriding value in appConfig.json as shown later</li>
+<li><strong>image</strong>: The full name of you Docker image.</li>
+<li><strong>additionalParam</strong>: (optional) Additional params that need to be passed to the command to be invoked during starting your Docker container</li>
+<li><strong>commandPath</strong>: The path to your docker command</li>
+<li><strong>statusCommand</strong>: (optional) The command you can specify to check the health of your running application. Slider relies on the return value of the command to judge if healthy. 0 as healthy and non-0 as unhealthy. If not provided, Slider will execute "docker top ${CONTAINER_ID} | grep"</li>
+<li><strong>port, containerPort and hostPort</strong>: (optional) the port of the container that will be binded to the hostPort field, which will be translated into <code>-p hostPort:containerPort</code> when starting the container with <code>docker run</code></li>
+<li><strong>mount, containerMount and hostMount</strong>: (optional) the directories from the host that will be mounted into the container, which will be translated into â-v hostMount:containerMountâ when starting the container</li>
+<li><strong>options</strong>: (optional) allow users to specify any additional docker run command options to use. Those will be passed to 'docker run' command when starting your application. If not specified, "-d" will be used</li>
+</ul>
+<p>Please note configs specified in metainfo.json can be overridden in appConfig.json: <strong>commandPath, options, statusCommand, inputFiles, mounts, ports</strong></p>
+<p><strong>appConfig.json</strong></p>
<div class="codehilite"><pre>{
- "schemaVersion": "2.1",
- "application": {
- "name": "FAKEAPP",
- "components": [
- {
- "name": "MEMCACHED",
- "type": "docker",
- "dockerContainers": [
- {
- "name": "memcached",
- "image": "borja/memcached",
- "additionalParam": "--appendonly yes",
- "commandPath": "/usr/bin/docker",
- "statusCommand": "docker inspect <span class="cp">${</span><span class="n">container_id</span><span class="cp">}</span>"
- }
- ]
- },
- {
- "name": "REDIS",
- "type": "docker",
- "dockerContainers": [
- {
- "name": "redis",
- "image": "dockerhub/redis",
- "additionalParam": "--appendonly yes",
- "commandPath": "/usr/bin/docker",
- "ports": [
- {
- "containerPort": "11211",
- "hostPort": "<span class="cp">${</span><span class="n">conf</span><span class="p">:</span><span class="n">configuration</span><span class="o">/</span><span class="k">global</span><span class="o">/</span><span class="n">port1</span><span class="cp">}</span>"
- }
- ],
- "mounts": [
- {
- "containerMount": "/tmp/confg",
- "hostMount": "<span class="cp">${</span><span class="n">conf</span><span class="p">:</span><span class="n">configuration</span><span class="o">/</span><span class="k">global</span><span class="o">/</span><span class="n">app_root</span><span class="cp">}</span>/conf"
- }
- ]
- }
- ]
+ "schema": "http://example.org/specification/v2.0.0",
+ "metadata": {
+ },
+ "global": {
+ },
+ "components": {
+ "NODEJS": {
+ "nodejs.commandPath": "/user/local/bin/docker",
+ "nodejs.options":"-d -e REDIS_HOST=<span class="cp">${</span><span class="n">REDIS_HOST</span><span class="cp">}</span> --net=host",
+ "nodejs.statusCommand":"docker inspect -f {{.State.Running}} <span class="cp">${</span><span class="n">CONTAINER_ID</span><span class="cp">}</span> | grep true"
+ },
+ "REDIS": {
+ "redis.options":"-d -e REDIS_PASS=**None**",
+ "redis.statusCommand":"docker top <span class="cp">${</span><span class="n">CONTAINER_ID</span><span class="cp">}</span> | grep redis-server"
}
- ]
}
}
</pre></div>
-<p>Please note, in metainfo.json (we are trying to migrate from metainfo.xml to metainfo.json as an improvement to the packaging approach) we are adding some new fields in the component section to support Docker based applications:</p>
-<ul>
-<li><strong>type</strong>: if specified as <code>docker</code>, Slider will start Docker containers for the application; by default it is <code>process</code>, which means Slider will instantiate the application as normal process as today</li>
-<li><strong>containers/container</strong>: the Docker image of the application, the name of which will be specified in <code>image</code> field</li>
-<li><strong>port, containerPort and hostPort</strong>: the port of the container that will be binded to the hostPort field, which will be translated into <code>-p hostPort:containerPort</code> when starting the container with <code>docker run</code></li>
-<li><strong>mount, containerMount and hostMount</strong>: the directories from the host that will be mounted into the container, which will be translated into â-v hostMount:containerMountâ when starting the container</li>
-<li><strong>options</strong>: allow users to specify any additional docker run command options to use</li>
-<li><strong>status_command</strong>: the command that Slider can use to query the status of the application component running in the container</li>
-<li>Properties specified in metainfo.json can be overridden in appConfig.json: <strong>commandPath, options, statusCommand, inputFiles, mounts, ports</strong></li>
-</ul>
+<p>AppConfig.json is where you can provide overriding config value for those you defined in metainfo.json. This may be needed when you want to provide some runtime variance.</p>
+<p>Please note that you need to specify the container name before your config key: "nodejs.commandPath" instead of "commandPath". This is to distinguish different containers under the same component as defined in metainfo.json</p>
+<p>In this example, we are specifying a different docker command path than the default <code>/usr/bin/docker</code> in metainfo.json and a different set of options that we need to include in the <code>docker run</code> command. </p>
+<p>The config structure in appConfig.json(map) -> components(list) -> component(map) -> containers(list)-> container(map) have to match the one in metainfo.json. </p>
<p><strong>resources.json</strong></p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>"<span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
@@ -328,12 +338,12 @@ Latest release: <strong>0.80.0-incubatin
"<span class="n">slider</span><span class="o">-</span><span class="n">appmaster</span>"<span class="p">:</span> <span class="p">{</span> <span class="p">},</span>
"<span class="n">REDIS</span>"<span class="p">:</span> <span class="p">{</span>
"<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "1"<span class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "2"<span class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "1"<span class="p">,</span>
"<span class="n">yarn</span><span class="p">.</span><span class="n">memory</span>"<span class="p">:</span> "512"
<span class="p">},</span>
- "<span class="n">MEMCACHED</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "1"<span class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "2"<span class="p">,</span>
+ "<span class="n">NODEJS</span>"<span class="p">:</span> <span class="p">{</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "2"<span class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "1"<span class="p">,</span>
"<span class="n">yarn</span><span class="p">.</span><span class="n">memory</span>"<span class="p">:</span> "512"
<span class="p">}</span>
<span class="p">}</span>
@@ -345,11 +355,11 @@ Latest release: <strong>0.80.0-incubatin
<ul>
<li>For redis components, it will run:</li>
</ul>
-<p><code>docker run -d -p hostPort:containerPort -v hostMount:containerMount -net=bridge dockerhub/redis</code></p>
+<p><code>docker run -d -e REDIS_PASS=**None** -p hostPort:containerPort -name ${CONTAINER_ID} tutum/redis</code></p>
<ul>
-<li>For memcached components, it will run:</li>
+<li>For nodejs components, it will run:</li>
</ul>
-<p><code>docker run -d borja/memcached memcached_server.sh --appendonly</code></p>
+<p><code>docker run -d -e REDIS_HOST=${REDIS_HOST} -name ${CONTAINER_ID} --net=host rsahahw/centos-node-redis</code></p>
</div>
<div id="footer">