You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by dl...@apache.org on 2015/10/09 12:09:30 UTC
svn commit: r1707687 - in /mesos/site:
publish/documentation/app-framework-development-guide/index.html
publish/documentation/latest/app-framework-development-guide/index.html
source/documentation/latest/app-framework-development-guide.md
Author: dlester
Date: Fri Oct 9 10:09:30 2015
New Revision: 1707687
URL: http://svn.apache.org/viewvc?rev=1707687&view=rev
Log:
Update documentation.
Modified:
mesos/site/publish/documentation/app-framework-development-guide/index.html
mesos/site/publish/documentation/latest/app-framework-development-guide/index.html
mesos/site/source/documentation/latest/app-framework-development-guide.md
Modified: mesos/site/publish/documentation/app-framework-development-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/app-framework-development-guide/index.html?rev=1707687&r1=1707686&r2=1707687&view=diff
==============================================================================
--- mesos/site/publish/documentation/app-framework-development-guide/index.html (original)
+++ mesos/site/publish/documentation/app-framework-development-guide/index.html Fri Oct 9 10:09:30 2015
@@ -206,11 +206,30 @@ virtual void executorLost(SchedulerDrive
virtual void error(SchedulerDriver* driver, const std::string& message) = 0;
</code></pre>
-<h2>Create your Framework Executor</h2>
+<h2>Working with Executors</h2>
+
+<h3>Using the Mesos Command Executor</h3>
+
+<p>Mesos provides a simple executor that can execute shell commands and Docker containers on behalf of the framework scheduler; enough functionality for a wide variety of framework requirements.</p>
+
+<p>Any scheduler can make use of the Mesos command executor by filling in the optional <code>CommandInfo</code> member of the <code>TaskInfo</code> protobuf message.</p>
+
+<pre><code class="{.proto}">message TaskInfo {
+ ...
+ optional CommandInfo command = 7;
+ ...
+}
+</code></pre>
+
+<p>The Mesos slave will fill in the rest of the ExecutorInfo for you when tasks are specified this way.</p>
+
+<h3>Creating a custom Framework Executor</h3>
+
+<p>If your framework has special requirements, you might want to provide your own Executor implementation. For example, you may not want a 1:1 relationship between tasks and processes.</p>
<p>Your framework executor must inherit from the Executor class. It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where Mesos is running from.</p>
-<h3>Executor API</h3>
+<h4>Executor API</h4>
<p>Declared in <code>MESOS_HOME/include/mesos/executor.hpp</code></p>
@@ -281,9 +300,13 @@ virtual void shutdown(ExecutorDriver* dr
virtual void error(ExecutorDriver* driver, const std::string& message) = 0;
</code></pre>
-<h2>Install your Framework</h2>
+<h4>Install your custom Framework Executor</h4>
+
+<p>After creating your custom executor, you need to make it available to all slaves in the cluster.</p>
+
+<p>One way to distribute your framework executor is to let the <a href="/documentation/latest/fetcher/">Mesos fetcher</a> download it on-demand when your scheduler launches tasks on that slave. <code>ExecutorInfo</code> is a Protocol Buffer Message class (defined in <code>include/mesos/mesos.proto</code>), and it contains a field of type <code>CommandInfo</code>. <code>CommandInfo</code> allows schedulers to specify, among other things, a number of resources as URIs. These resources are fetched to a sandbox directory on the slave before attempting to execute the <code>ExecutorInfo</code> command. Several URI schemes are supported, including HTTP, FTP, HDFS, and S3 (e.g. see src/examples/java/TestFramework.java for an example of this).</p>
-<p>You need to put your framework somewhere that all slaves on the cluster can get it from. If you are running HDFS, you can put your executor into HDFS. Then, you tell Mesos where it is via the <code>ExecutorInfo</code> parameter of <code>MesosSchedulerDriver</code>’s constructor (e.g. see src/examples/java/TestFramework.java for an example of this). ExecutorInfo is a Protocol Buffer Message class (defined in <code>include/mesos/mesos.proto</code>), and you set its URI field to something like “HDFS://path/to/executor/”. Also, you can pass the <code>frameworks_home</code> configuration option (defaults to: <code>MESOS_HOME/frameworks</code>) to your <code>mesos-slave</code> daemons when you launch them to specify where all of your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then set <code>ExecutorInfo</code> to be a relative path, and the slave will prepend the value of frameworks_home to the relative path provided.</p>
+<p>Alternatively, you can pass the <code>frameworks_home</code> configuration option (defaults to: <code>MESOS_HOME/frameworks</code>) to your <code>mesos-slave</code> daemons when you launch them to specify where your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then use a relative path in <code>CommandInfo.uris</code>, and the slave will prepend the value of <code>frameworks_home</code> to the relative path provided.</p>
<p>Once you are sure that your executors are available to the mesos-slaves, you should be able to run your scheduler, which will register with the Mesos master, and start receiving resource offers!</p>
Modified: mesos/site/publish/documentation/latest/app-framework-development-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/app-framework-development-guide/index.html?rev=1707687&r1=1707686&r2=1707687&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/app-framework-development-guide/index.html (original)
+++ mesos/site/publish/documentation/latest/app-framework-development-guide/index.html Fri Oct 9 10:09:30 2015
@@ -206,11 +206,30 @@ virtual void executorLost(SchedulerDrive
virtual void error(SchedulerDriver* driver, const std::string& message) = 0;
</code></pre>
-<h2>Create your Framework Executor</h2>
+<h2>Working with Executors</h2>
+
+<h3>Using the Mesos Command Executor</h3>
+
+<p>Mesos provides a simple executor that can execute shell commands and Docker containers on behalf of the framework scheduler; enough functionality for a wide variety of framework requirements.</p>
+
+<p>Any scheduler can make use of the Mesos command executor by filling in the optional <code>CommandInfo</code> member of the <code>TaskInfo</code> protobuf message.</p>
+
+<pre><code class="{.proto}">message TaskInfo {
+ ...
+ optional CommandInfo command = 7;
+ ...
+}
+</code></pre>
+
+<p>The Mesos slave will fill in the rest of the ExecutorInfo for you when tasks are specified this way.</p>
+
+<h3>Creating a custom Framework Executor</h3>
+
+<p>If your framework has special requirements, you might want to provide your own Executor implementation. For example, you may not want a 1:1 relationship between tasks and processes.</p>
<p>Your framework executor must inherit from the Executor class. It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where Mesos is running from.</p>
-<h3>Executor API</h3>
+<h4>Executor API</h4>
<p>Declared in <code>MESOS_HOME/include/mesos/executor.hpp</code></p>
@@ -281,9 +300,13 @@ virtual void shutdown(ExecutorDriver* dr
virtual void error(ExecutorDriver* driver, const std::string& message) = 0;
</code></pre>
-<h2>Install your Framework</h2>
+<h4>Install your custom Framework Executor</h4>
+
+<p>After creating your custom executor, you need to make it available to all slaves in the cluster.</p>
+
+<p>One way to distribute your framework executor is to let the <a href="/documentation/latest/fetcher/">Mesos fetcher</a> download it on-demand when your scheduler launches tasks on that slave. <code>ExecutorInfo</code> is a Protocol Buffer Message class (defined in <code>include/mesos/mesos.proto</code>), and it contains a field of type <code>CommandInfo</code>. <code>CommandInfo</code> allows schedulers to specify, among other things, a number of resources as URIs. These resources are fetched to a sandbox directory on the slave before attempting to execute the <code>ExecutorInfo</code> command. Several URI schemes are supported, including HTTP, FTP, HDFS, and S3 (e.g. see src/examples/java/TestFramework.java for an example of this).</p>
-<p>You need to put your framework somewhere that all slaves on the cluster can get it from. If you are running HDFS, you can put your executor into HDFS. Then, you tell Mesos where it is via the <code>ExecutorInfo</code> parameter of <code>MesosSchedulerDriver</code>’s constructor (e.g. see src/examples/java/TestFramework.java for an example of this). ExecutorInfo is a Protocol Buffer Message class (defined in <code>include/mesos/mesos.proto</code>), and you set its URI field to something like “HDFS://path/to/executor/”. Also, you can pass the <code>frameworks_home</code> configuration option (defaults to: <code>MESOS_HOME/frameworks</code>) to your <code>mesos-slave</code> daemons when you launch them to specify where all of your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then set <code>ExecutorInfo</code> to be a relative path, and the slave will prepend the value of frameworks_home to the relative path provided.</p>
+<p>Alternatively, you can pass the <code>frameworks_home</code> configuration option (defaults to: <code>MESOS_HOME/frameworks</code>) to your <code>mesos-slave</code> daemons when you launch them to specify where your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then use a relative path in <code>CommandInfo.uris</code>, and the slave will prepend the value of <code>frameworks_home</code> to the relative path provided.</p>
<p>Once you are sure that your executors are available to the mesos-slaves, you should be able to run your scheduler, which will register with the Mesos master, and start receiving resource offers!</p>
Modified: mesos/site/source/documentation/latest/app-framework-development-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/app-framework-development-guide.md?rev=1707687&r1=1707686&r2=1707687&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/app-framework-development-guide.md (original)
+++ mesos/site/source/documentation/latest/app-framework-development-guide.md Fri Oct 9 10:09:30 2015
@@ -128,11 +128,31 @@ virtual void executorLost(SchedulerDrive
virtual void error(SchedulerDriver* driver, const std::string& message) = 0;
~~~
-## Create your Framework Executor
+## Working with Executors
+
+### Using the Mesos Command Executor
+
+Mesos provides a simple executor that can execute shell commands and Docker containers on behalf of the framework scheduler; enough functionality for a wide variety of framework requirements.
+
+Any scheduler can make use of the Mesos command executor by filling in the optional `CommandInfo` member of the `TaskInfo` protobuf message.
+
+~~~{.proto}
+message TaskInfo {
+ ...
+ optional CommandInfo command = 7;
+ ...
+}
+~~~
+
+The Mesos slave will fill in the rest of the ExecutorInfo for you when tasks are specified this way.
+
+### Creating a custom Framework Executor
+
+If your framework has special requirements, you might want to provide your own Executor implementation. For example, you may not want a 1:1 relationship between tasks and processes.
Your framework executor must inherit from the Executor class. It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where Mesos is running from.
-### Executor API
+#### Executor API
Declared in `MESOS_HOME/include/mesos/executor.hpp`
@@ -204,9 +224,13 @@ virtual void shutdown(ExecutorDriver* dr
virtual void error(ExecutorDriver* driver, const std::string& message) = 0;
~~~
-## Install your Framework
+#### Install your custom Framework Executor
+
+After creating your custom executor, you need to make it available to all slaves in the cluster.
+
+One way to distribute your framework executor is to let the [Mesos fetcher](/documentation/latest/fetcher/) download it on-demand when your scheduler launches tasks on that slave. `ExecutorInfo` is a Protocol Buffer Message class (defined in `include/mesos/mesos.proto`), and it contains a field of type `CommandInfo`. `CommandInfo` allows schedulers to specify, among other things, a number of resources as URIs. These resources are fetched to a sandbox directory on the slave before attempting to execute the `ExecutorInfo` command. Several URI schemes are supported, including HTTP, FTP, HDFS, and S3 (e.g. see src/examples/java/TestFramework.java for an example of this).
-You need to put your framework somewhere that all slaves on the cluster can get it from. If you are running HDFS, you can put your executor into HDFS. Then, you tell Mesos where it is via the `ExecutorInfo` parameter of `MesosSchedulerDriver`'s constructor (e.g. see src/examples/java/TestFramework.java for an example of this). ExecutorInfo is a Protocol Buffer Message class (defined in `include/mesos/mesos.proto`), and you set its URI field to something like "HDFS://path/to/executor/". Also, you can pass the `frameworks_home` configuration option (defaults to: `MESOS_HOME/frameworks`) to your `mesos-slave` daemons when you launch them to specify where all of your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then set `ExecutorInfo` to be a relative path, and the slave will prepend the value of frameworks_home to the relative path provided.
+Alternatively, you can pass the `frameworks_home` configuration option (defaults to: `MESOS_HOME/frameworks`) to your `mesos-slave` daemons when you launch them to specify where your framework executors are stored (e.g. on an NFS mount that is available to all slaves), then use a relative path in `CommandInfo.uris`, and the slave will prepend the value of `frameworks_home` to the relative path provided.
Once you are sure that your executors are available to the mesos-slaves, you should be able to run your scheduler, which will register with the Mesos master, and start receiving resource offers!