You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by dl...@apache.org on 2015/10/09 15:11:15 UTC
[1/3] mesos git commit: Documentation file naming cleanup
Repository: mesos
Updated Branches:
refs/heads/master e8e9768c7 -> b29ec4f11
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/ssl.md
----------------------------------------------------------------------
diff --git a/docs/ssl.md b/docs/ssl.md
new file mode 100644
index 0000000..9df34db
--- /dev/null
+++ b/docs/ssl.md
@@ -0,0 +1,92 @@
+---
+layout: documentation
+---
+
+# Configuration
+There is currently only one implementation of the [libprocess socket interface](https://github.com/apache/mesos/blob/master/3rdparty/libprocess/include/process/socket.hpp) that supports SSL. This implementation uses [libevent](https://github.com/libevent/libevent). Specifically it relies on the `libevent-openssl` library that wraps `openssl`.
+
+After building Mesos 0.23.0 from source, assuming you have installed the required [Dependencies](#Dependencies), you can modify your configure line to enable SSL as follows:
+
+~~~
+../configure --enable-libevent --enable-ssl
+~~~
+
+# Running
+Once you have successfully built and installed your new binaries, here are the environment variables that are applicable to the `Master`, `Slave`, `Framework Scheduler/Executor`, or any `libprocess process`:
+
+#### SSL_ENABLED=(false|0,true|1) [default=false|0]
+Turn on or off SSL. When it is turned off it is the equivalent of default mesos with libevent as the backing for events. All sockets default to the non-SSL implementation. When it is turned on, the default configuration for sockets is SSL. This means outgoing connections will use SSL, and incoming connections will be expected to speak SSL as well. None of the below flags are relevant if SSL is not enabled.
+
+#### SSL_SUPPORT_DOWNGRADE=(false|0,true|1) [default=false|0]
+Control whether or not non-SSL connections can be established. If this is enabled __on the accepting side__, then the accepting side will downgrade to a non-SSL socket if the connecting side is attempting to communicate via non-SSL. (e.g. HTTP). See [Upgrading Your Cluster](#Upgrading) for more details.
+
+#### SSL_CERT_FILE=(path to certificate)
+The location of the certificate that will be presented.
+
+#### SSL_KEY_FILE=(path to key)
+The location of the private key used by OpenSSL.
+
+#### SSL_VERIFY_CERT=(false|0,true|1) [default=false|0]
+Control whether certificates are verified when presented. If this is false, even when a certificate is presented, it will not be verified. When `SSL_REQUIRE_CERT` is true, `SSL_VERIFY_CERT` is overridden and all certificates will be verified _and_ required.
+
+#### SSL_REQUIRE_CERT=(false|0,true|1) [default=false|0]
+Enforce that certificates must be presented by connecting clients. This means all connections (including tools hitting endpoints) must present valid certificates in order to establish a connection.
+
+#### SSL_VERIFY_DEPTH=(N) [default=4]
+The maximum depth used to verify certificates. The default is 4. See the OpenSSL documentation or contact your system administrator to learn why you may want to change this.
+
+#### SSL_CA_DIR=(path to CA directory)
+The directory used to find the certificate authority / authorities. You can specify `SSL_CA_DIR` or `SSL_CA_FILE` depending on how you want to restrict your certificate authorization.
+
+#### SSL_CA_FILE=(path to CA file)
+The file used to find the certificate authority. You can specify `SSL_CA_DIR` or `SSL_CA_FILE` depending on how you want to restrict your certificate authorization.
+
+#### SSL_CIPHERS=(accepted ciphers separated by ':') [default=AES128-SHA:AES256-SHA:RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA]
+A list of `:`-separated ciphers. Use these if you want to restrict or open up the accepted ciphers for OpenSSL. Read the OpenSSL documentation or contact your system administrators to see whether you want to override the default values.
+
+#### SSL_ENABLE_SSL_V3=(false|0,true|1) [default=false|0]
+#### SSL_ENABLE_TLS_V1_0=(false|0,true|1) [default=false|0]
+#### SSL_ENABLE_TLS_V1_1=(false|0,true|1) [default=false|0]
+#### SSL_ENABLE_TLS_V1_2=(false|0,true|1) [default=true|1]
+The above switches enable / disable the specified protocols. By default only TLS V1.2 is enabled. SSL V2 is always disabled; there is no switch to enable it. The mentality here is to restrict security by default, and force users to open it up explicitly. Many older version of the protocols have known vulnerabilities, so only enable these if you fully understand the risks.
+_SSLv2 is disabled completely because modern versions of OpenSSL disable it using multiple compile time configuration options._
+#<a name="Dependencies"></a>Dependencies
+
+### libevent
+We require the OpenSSL support from libevent. The suggested version of libevent is [`2.0.22-stable`](https://github.com/libevent/libevent/releases/tag/release-2.0.22-stable). As new releases come out we will try to maintain compatibility.
+
+~~~
+// For example, on OSX:
+brew install libevent
+~~~
+
+### OpenSSL
+We require [OpenSSL](https://github.com/openssl/openssl). There are multiple branches of OpenSSL that are being maintained by the community. Since security requires being vigilant, we recommend reading the release notes for the current releases of OpenSSL and deciding on a version within your organization based on your security needs. Mesos is not too deeply dependent on specific OpenSSL versions, so there is room for you to make security decisions as an organization.
+Please ensure the `event2` and `openssl` headers are available for building mesos.
+
+~~~
+// For example, on OSX:
+brew install openssl
+~~~
+
+# <a name="Upgrading"></a>Upgrading Your Cluster
+_There is no SSL specific requirement for upgrading different components in a specific order._
+
+The recommended strategy is to restart all your components to enable SSL with downgrades support enabled. Once all components have SSL enabled, then do a second restart of all your components to disable downgrades. This strategy will allow each component to be restarted independently at your own convenience with no time restrictions. It will also allow you to try SSL in a subset of your cluster. __NOTE:__ While different components in your cluster are serving SSL vs non-SSL traffic, any relative links in the WebUI may be broken. Please see the [WebUI](#WebUI) section for details. Here are sample commands for upgrading your cluster:
+
+~~~
+// Restart each component with downgrade support (master, slave, framework):
+SSL_ENABLED=true SSL_SUPPORT_DOWNGRADE=true SSL_KEY_FILE=<path-to-your-private-key> SSL_CERT_FILE=<path-to-your-certificate> <Any other SSL_* environment variables you may choose> <your-component (e.g. bin/master.sh)> <your-flags>
+
+// Restart each component WITHOUT downgrade support (master, slave, framework):
+SSL_ENABLED=true SSL_SUPPORT_DOWNGRADE=false SSL_KEY_FILE=<path-to-your-private-key> SSL_CERT_FILE=<path-to-your-certificate> <Any other SSL_* environment variables you may choose> <your-component (e.g. bin/master.sh)> <your-flags>
+~~~
+The end state is a cluster that is only communicating with SSL.
+
+__NOTE:__ Any tools you may use that communicate with your components must be able to speak SSL, or they will be denied. You may choose to maintain `SSL_SUPPORT_DOWNGRADE=true` for some time as you upgrade your internal tooling. The advantage of `SSL_SUPPORT_DOWNGRADE=true` is that all components that speak SSL will do so, while other components may still communicate over insecure channels.
+
+# <a name="WebUI"></a>WebUI
+The default Mesos WebUI uses relative links. Some of these links transition between endpoints served by the master and slaves. The WebUI currently does not have enough information to change the 'http' vs 'https' links based on whether the target endpoint is currently being served by an SSL-enabled binary. This may cause certain links in the WebUI to be broken when a cluster is in a transition state between SSL and non-SSL. Any tools that hit these endpoints will still be able to access them as long as they hit the endpoint using the right protocol, or the `SSL_SUPPORT_DOWNGRADE` option is set to true.
+
+### Certificates
+Most browsers have built in protection that guard transitions between pages served using different certificates. For this reason you may choose to serve both the master and slave endpoints using a common certificate that covers multiple hostnames. If you do not do this, certain links, such as those to slave sandboxes, may seem broken as the browser treats the transition between differing certificates transition as unsafe.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/submitting-a-patch.md
----------------------------------------------------------------------
diff --git a/docs/submitting-a-patch.md b/docs/submitting-a-patch.md
index 78ffcfb..cbd99fe 100644
--- a/docs/submitting-a-patch.md
+++ b/docs/submitting-a-patch.md
@@ -30,10 +30,10 @@ layout: documentation
### Create your patch
1. Create one or more test cases to exercise the bug or the feature (the Mesos team uses [test-driven development](http://en.wikipedia.org/wiki/Test-driven_development)). Before you start coding, make sure these test cases all fail.
- 1. The [testing patterns](mesos-testing-patterns.md) page has some suggestions for writing test cases.
+ 1. The [testing patterns](testing-patterns.md) page has some suggestions for writing test cases.
2. Make your changes to the code (using whatever IDE/editor you choose) to actually fix the bug or implement the feature.
- 1. Before beginning, please read the [Mesos C++ Style Guide](mesos-c++-style-guide.md). It is recommended to use the git pre-commit hook (`support/hooks/pre-commit`) to automatically check for style errors. See the hook script for instructions to enable it.
+ 1. Before beginning, please read the [Mesos C++ Style Guide](c++-style-guide.md). It is recommended to use the git pre-commit hook (`support/hooks/pre-commit`) to automatically check for style errors. See the hook script for instructions to enable it.
2. Most of your changes will probably be to files inside of `BASE_MESOS_DIR`
3. From inside of the root Mesos directory: `./bootstrap` (Only required if building from git repository).
4. To build, we recommend that you don't build inside of the src directory. We recommend you do the following:
@@ -77,4 +77,4 @@ layout: documentation
4. The last step is to ensure that the necessary documentation gets created or updated so the whole world knows about your new feature or bug fix.
## Style Guides
-* For patches to the core, we ask that you follow the [Mesos C++ Style Guide](mesos-c++-style-guide.md).
+* For patches to the core, we ask that you follow the [Mesos C++ Style Guide](c++-style-guide.md).
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/testing-patterns.md
----------------------------------------------------------------------
diff --git a/docs/testing-patterns.md b/docs/testing-patterns.md
new file mode 100644
index 0000000..d0d92ff
--- /dev/null
+++ b/docs/testing-patterns.md
@@ -0,0 +1,81 @@
+---
+layout: documentation
+---
+
+# Mesos Testing Patterns
+
+A collection of common testing patterns used in Mesos tests. If you have found a good way to test a certain condition that you think may be useful for other cases, please document it here together with motivation and background.
+
+## Using `Clock` magic to ensure an event is processed
+Scheduling a sequence of events in an asynchronous environment is not easy: a function call usually initiates an action and returns immediately, while the action runs in background. A simple, obvious, and bad solution is to use `os::sleep()` to wait for action completion. The time the action needs to finish may vary on different machines, while increasing sleep duration increases the test execution time and slows down `make check`. One of the right ways to do it is to wait for an action to finish and proceed right after. This is possible using libprocess' `Clock` routines.
+
+
+Every message enqueued in a libprocess process' (or actor's, to avoid ambiguity with OS processes) mailbox is processed by `ProcessManager` (right now there is a single instance of `ProcessManager` per OS process, but this may change in the future). `ProcessManager` fetches actors from the runnable actors list and services all events from the actor's mailbox. Using `Clock::settle()` call we can block the calling thread until `ProcessManager` empties mailboxes of all actors. Here is the example of this pattern:
+
+~~~{.cpp}
+// As Master::killTask isn't doing anything, we shouldn't get a status update.
+EXPECT_CALL(sched, statusUpdate(&driver, _))
+ .Times(0);
+
+// Set expectation that Master receives killTask message.
+Future<KillTaskMessage> killTaskMessage =
+ FUTURE_PROTOBUF(KillTaskMessage(), _, master.get());
+
+// Attempt to kill unknown task while slave is transitioning.
+TaskID unknownTaskId;
+unknownTaskId.set_value("2");
+
+// Stop the clock.
+Clock::pause();
+
+// Initiate an action.
+driver.killTask(unknownTaskId);
+
+// Make sure the event associated with the action has been queued.
+AWAIT_READY(killTaskMessage);
+
+// Wait for all messages to be dispatched and processed completely to satisfy
+// the expectation that we didn't receive a status update.
+Clock::settle();
+
+Clock::resume();
+~~~
+
+## Intercepting a message sent to a different OS process
+Intercepting messages sent between libprocess processes (let's call them actors to avoid ambiguity with OS processes) that live in the same OS process is easy, e.g.:
+
+~~~{.cpp}
+Future<SlaveReregisteredMessage> slaveReregisteredMessage =
+ FUTURE_PROTOBUF(SlaveReregisteredMessage(), _, _);
+...
+AWAIT_READY(slaveReregisteredMessage);
+~~~
+
+However, this won't work if we want to intercept a message sent to an actor (technically a `UPID`) that lives in another OS process. For example, `CommandExecutor` spawned by a slave will live in a separate OS process, though master and slave instances live in the same OS process together with our test (see `mesos/src/tests/cluster.hpp`). The wait in this code will fail:
+
+~~~{.cpp}
+Future<ExecutorRegisteredMessage> executorRegisteredMessage =
+ FUTURE_PROTOBUF(ExecutorRegisteredMessage(), _, _);
+...
+AWAIT_READY(executorRegisteredMessage);
+~~~
+
+### Why messages sent outside the OS process are not intercepted?
+Libprocess events may be filtered (see `libprocess/include/process/filter.hpp`). `FUTURE_PROTOBUF` uses this ability and sets an expectation on a `filter` method of `TestsFilter` class with a `MessageMatcher`, that matches the message we want to intercept. The actual filtering happens in `ProcessManager::resume()`, which fetches messages from the queue of the received events.
+
+*No* filtering happens when sending, encoding, or transporting the message (see e.g. `ProcessManager::deliver()` or `SocketManager::send()`). Therefore in the aforementioned example, `ExecutorRegisteredMessage` leaves the slave undetected by the filter, reaches another OS process where executor lives, gets enqueued into the `CommandExecutorProcess`' mailbox and can be filtered there, but remember our expectation is set in another OS process!
+
+### How to workaround
+Consider setting expectations on corresponding incoming messages ensuring they are processed and therefore ACK message is sent.
+
+For the aforementioned example, instead of intercepting `ExecutorRegisteredMessage`, we can intercept `RegisterExecutorMessage` and wait until its processed, which includes sending `ExecutorRegisteredMessage` (see `Slave::registerExecutor()`):
+
+~~~{.cpp}
+Future<RegisterExecutorMessage> registerExecutorMessage =
+ FUTURE_PROTOBUF(RegisterExecutorMessage(), _, _);
+...
+AWAIT_READY(registerExecutorMessage);
+Clock::pause();
+Clock::settle();
+Clock::resume();
+~~~
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/tools.md
----------------------------------------------------------------------
diff --git a/docs/tools.md b/docs/tools.md
index 414decf..3e5f4d7 100644
--- a/docs/tools.md
+++ b/docs/tools.md
@@ -20,10 +20,10 @@ These tools make it easy to set up and run a Mesos cluster.
If you want to hack on Mesos or write a new framework, these tools will help.
-* [clang-format](/documentation/latest/clang-format/) to automatically apply some of the style rules dictated by the [Mesos C++ Style Guide](/documentation/latest/mesos-c++-style-guide/).
+* [clang-format](/documentation/latest/clang-format/) to automatically apply some of the style rules dictated by the [Mesos C++ Style Guide](/documentation/latest/c++-style-guide/).
* [Go Bindings and Examples](https://github.com/mesosphere/mesos-go) Write a Mesos framework in Go! Comes with an example scheduler and executor.
* [Mesos Framework giter8 Template](https://github.com/mesosphere/scala-sbt-mesos-framework.g8) This is a giter8 template. The result of applying this template is a bare-bones Apache Mesos framework in Scala using SBT for builds and Vagrant for testing on a singleton cluster.
* [Scala Hello World](https://gist.github.com/guenter/7471695) A simple Mesos "Hello World": downloads and starts a web server on every node in the cluster.
* [Xcode Workspace](https://github.com/tillt/xcode-mesos) Hack on Mesos in Xcode.
-Can't find yours in the list? Please submit a patch, or email user@mesos.apache.org and we'll add you!
\ No newline at end of file
+Can't find yours in the list? Please submit a patch, or email user@mesos.apache.org and we'll add you!
[3/3] mesos git commit: Documentation file naming cleanup
Posted by dl...@apache.org.
Documentation file naming cleanup
Part of https://issues.apache.org/jira/browse/MESOS-3686
Clean up of filenames in documentation, prefer hyphens and drop 'mesos' from filenames
This closes: #70
Review: https://github.com/apache/mesos/pull/70
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/b29ec4f1
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/b29ec4f1
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/b29ec4f1
Branch: refs/heads/master
Commit: b29ec4f110483555a5e1a65ef25a7ecc13e31b7f
Parents: e8e9768
Author: Elsmore <ch...@demandlogic.co.uk>
Authored: Fri Oct 9 14:09:20 2015 +0100
Committer: Dave Lester <dl...@Daves-MacBook-Pro.local>
Committed: Fri Oct 9 14:09:20 2015 +0100
----------------------------------------------------------------------
docs/architecture.md | 32 ++
docs/c++-style-guide.md | 579 ++++++++++++++++++++++++++++++++
docs/clang-format.md | 2 +-
docs/containerizer.md | 82 +++++
docs/documentation-guide.md | 37 ++
docs/doxygen-style-guide.md | 196 +++++++++++
docs/effective-code-reviewing.md | 4 +-
docs/frameworks.md | 34 ++
docs/home.md | 24 +-
docs/markdown-style-guide.md | 118 +++++++
docs/mesos-architecture.md | 32 --
docs/mesos-c++-style-guide.md | 579 --------------------------------
docs/mesos-containerizer.md | 82 -----
docs/mesos-documentation-guide.md | 2 +
docs/mesos-doxygen-style-guide.md | 196 -----------
docs/mesos-frameworks.md | 34 --
docs/mesos-markdown-style-guide.md | 118 -------
docs/mesos-presentations.md | 92 -----
docs/mesos-roadmap.md | 13 -
docs/mesos-ssl.md | 92 -----
docs/mesos-testing-patterns.md | 81 -----
docs/powered-by-mesos.md | 2 +-
docs/presentations.md | 92 +++++
docs/roadmap.md | 13 +
docs/scheduler-http-api.md | 519 ++++++++++++++++++++++++++++
docs/scheduler_http_api.md | 519 ----------------------------
docs/ssl.md | 92 +++++
docs/submitting-a-patch.md | 6 +-
docs/testing-patterns.md | 81 +++++
docs/tools.md | 4 +-
30 files changed, 1898 insertions(+), 1859 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/architecture.md
----------------------------------------------------------------------
diff --git a/docs/architecture.md b/docs/architecture.md
new file mode 100644
index 0000000..439a56d
--- /dev/null
+++ b/docs/architecture.md
@@ -0,0 +1,32 @@
+---
+layout: documentation
+---
+
+# Mesos Architecture
+
+
+
+The above figure shows the main components of Mesos. Mesos consists of a *master* daemon that manages *slave* daemons running on each cluster node, and *mesos applications* (also called *frameworks*) that run *tasks* on these slaves.
+
+The master enables fine-grained sharing of resources (cpu, ram, ...) across applications by making them *resource offers*. Each resource offer contains a list of <slave ID, resource1: amount1, resource2, amount2, ...>. The master decides *how many* resources to offer to each framework according to a given organizational policy, such as fair sharing, or strict priority. To support a diverse set of policies, the master employs a modular architecture that makes it easy to add new allocation modules via a plugin mechanism.
+
+A framework running on top of Mesos consists of two components: a *scheduler* that registers with the master to be offered resources, and an *executor* process that is launched on slave nodes to run the framework's tasks (see the [App/Framework development guide](app-framework-development-guide.md) for more details about application schedulers and executors). While the master determines **how many** resources are offered to each framework, the frameworks' schedulers select **which** of the offered resources to use. When a frameworks accepts offered resources, it passes to Mesos a description of the tasks it wants to run on them. In turn, Mesos launches the tasks on the corresponding slaves.
+
+## Example of resource offer
+
+The figure below shows an example of how a framework gets scheduled to run a task.
+
+
+
+Let's walk through the events in the figure.
+
+1. Slave 1 reports to the master that it has 4 CPUs and 4 GB of memory free. The master then invokes the allocation policy module, which tells it that framework 1 should be offered all available resources.
+1. The master sends a resource offer describing what is available on slave 1 to framework 1.
+1. The framework's scheduler replies to the master with information about two tasks to run on the slave, using <2 CPUs, 1 GB RAM> for the first task, and <1 CPUs, 2 GB RAM> for the second task.
+1. Finally, the master sends the tasks to the slave, which allocates appropriate resources to the framework's executor, which in turn launches the two tasks (depicted with dotted-line borders in the figure). Because 1 CPU and 1 GB of RAM are still unallocated, the allocation module may now offer them to framework 2.
+
+In addition, this resource offer process repeats when tasks finish and new resources become free.
+
+While the thin interface provided by Mesos allows it to scale and allows the frameworks to evolve independently, one question remains: how can the constraints of a framework be satisfied without Mesos knowing about these constraints? For example, how can a framework achieve data locality without Mesos knowing which nodes store the data required by the framework? Mesos answers these questions by simply giving frameworks the ability to **reject** offers. A framework will reject the offers that do not satisfy its constraints and accept the ones that do. In particular, we have found that a simple policy called delay scheduling, in which frameworks wait for a limited time to acquire nodes storing the input data, yields nearly optimal data locality.
+
+You can also read much more about the Mesos architecture in this [technical paper](http://mesos.berkeley.edu/mesos_tech_report.pdf).
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/c++-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/c++-style-guide.md b/docs/c++-style-guide.md
new file mode 100644
index 0000000..0b61891
--- /dev/null
+++ b/docs/c++-style-guide.md
@@ -0,0 +1,579 @@
+---
+layout: documentation
+---
+
+# Mesos C++ Style Guide
+
+The Mesos codebase follows the [Google C++ Style Guide](http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml) with some notable differences, as described below. Note that the [clang-format](/documentation/latest/clang-format/) tool can be helpful to ensure that some of the mechanical style rules are obeyed.
+
+## Scoping
+
+### Namespaces
+* We avoid `using namespace foo` statements as it is not explicit about which symbols are pulled in, and it can often pull in a lot of symbols, which sometimes lead to conflicts.
+* It is OK to use namespace aliases to help pull in sub-namespaces, such as `namespace http = process::http;`. These should only be present at the top of the .cpp file.
+
+## Naming
+
+### Variable Names
+* We use [lowerCamelCase](http://en.wikipedia.org/wiki/CamelCase#Variations_and_synonyms) for variable names (Google uses snake_case, and their class member variables have trailing underscores).
+* We prepend constructor and function arguments with a leading underscore to avoid ambiguity and / or shadowing:
+
+~~~{.cpp}
+Try(State _state, T* _t = NULL, const std::string& _message = "")
+ : state(_state), t(_t), message(_message) {}
+~~~
+
+* Prefer trailing underscores for use as member fields (but not required). Some trailing underscores are used to distinguish between similar variables in the same scope (think prime symbols), *but this should be avoided as much as possible, including removing existing instances in the code base.*
+
+* If you find yourself creating a copy of an argument passed by const reference, consider passing it by value instead (if you don't want to use a leading underscore and copy in the body of the function):
+
+~~~{.cpp}
+// You can pass-by-value in ProtobufProcess::install() handlers.
+void Slave::statusUpdate(StatusUpdate update, const UPID& pid)
+{
+ ...
+ update.mutable_status()->set_source(
+ pid == UPID() ? TaskStatus::SOURCE_SLAVE : TaskStatus::SOURCE_EXECUTOR);
+ ...
+}
+~~~
+
+### Constant Names
+* We use [SCREAMING_SNAKE_CASE](http://en.wikipedia.org/wiki/Letter_case#Special_case_styles) for constant names (Google uses a `k` followed by mixed case, e.g. `kDaysInAWeek`).
+
+### Function Names
+* We use [lowerCamelCase](http://en.wikipedia.org/wiki/CamelCase#Variations_and_synonyms) for function names (Google uses mixed case for regular functions; and their accessors and mutators match the name of the variable).
+
+## Strings
+* Strings used in log and error messages should end without a period.
+
+## Comments
+* End each sentence within a comment with a punctuation mark (please note that we generally prefer periods); this applies to incomplete sentences as well.
+* For trailing comments, leave one space.
+
+## Breaks
+* Break before braces on enum, function, and record (i.e. struct, class, union) definitions.
+
+## Indentation
+
+### Class Format
+* Access modifiers are not indented (Google uses one space indentation).
+* Constructor initializers are indented by 2 spaces (Google indents by 4).
+
+### Templates
+* Leave one space after the `template` keyword, e.g. `template <typename T>` rather than `template<typename T>`.
+
+### Function Definition/Invocation
+* Newline when calling or defining a function: indent with 4 spaces.
+* We do not follow Google's style of wrapping on the open parenthesis, the general goal is to reduce visual "jaggedness" in the code. Prefer (1), (4), (5), sometimes (3), never (2):
+
+~~~{.cpp}
+// 1: OK.
+allocator->resourcesRecovered(frameworkId, slaveId, resources, filters);
+
+// 2: Don't use.
+allocator->resourcesRecovered(frameworkId, slaveId,
+ resources, filters);
+
+// 3: Don't use in this case due to "jaggedness".
+allocator->resourcesRecovered(frameworkId,
+ slaveId,
+ resources,
+ filters);
+
+// 3: In this case, 3 is OK.
+foobar(someArgument,
+ someOtherArgument,
+ theLastArgument);
+
+// 4: OK.
+allocator->resourcesRecovered(
+ frameworkId,
+ slaveId,
+ resources,
+ filters);
+
+// 5: OK.
+allocator->resourcesRecovered(
+ frameworkId, slaveId, resources, filters);
+~~~
+
+### Continuation
+* Newline for an assignment statement: indent with 2 spaces.
+
+~~~{.cpp}
+Try<Duration> failoverTimeout =
+ Duration::create(FrameworkInfo().failover_timeout());
+~~~
+
+## Empty Lines
+* 1 blank line at the end of the file.
+* Elements outside classes (classes, structs, global functions, etc.) should be spaced apart by 2 blank lines.
+* Elements inside classes (member variables and functions) should not be spaced apart by more than 1 blank line.
+
+## Capture by Reference
+
+We disallow capturing **temporaries** by reference. See [MESOS-2629](https://issues.apache.org/jira/browse/MESOS-2629) for the rationale.
+
+~~~{.cpp}
+Future<Nothing> f() { return Nothing(); }
+Future<bool> g() { return false; }
+
+struct T
+{
+ T(const char* data) : data(data) {}
+ const T& member() const { return *this; }
+ const char* data;
+};
+
+// 1: Don't use.
+const Future<Nothing>& future = f();
+
+// 1: Instead use.
+const Future<Nothing> future = f();
+
+// 2: Don't use.
+const Future<Nothing>& future = Future<Nothing>(Nothing());
+
+// 2: Instead use.
+const Future<Nothing> future = Future<Nothing>(Nothing());
+
+// 3: Don't use.
+const Future<bool>& future = f().then(lambda::bind(g));
+
+// 3: Instead use.
+const Future<bool> future = f().then(lambda::bind(g));
+
+// 4: Don't use (since the T that got constructed is a temporary!).
+const T& t = T("Hello").member();
+
+// 4: Preferred alias pattern (see below).
+const T t("Hello");
+const T& t_ = t.member();
+
+// 4: Can also use.
+const T t = T("Hello").member();
+~~~
+
+We allow capturing non-temporaries by *constant reference* when the intent is to **alias**.
+
+The goal is to make code more concise and improve readability. Use this if an expression referencing a field by `const` is:
+
+* Used repeatedly.
+* Would benefit from a concise name to provide context for readability.
+* Will **not** be invalidated during the lifetime of the alias. Otherwise document this explicitly.
+
+~~~{.cpp}
+hashmap<string, hashset<int>> index;
+
+// 1: Ok.
+const hashset<int>& values = index[2];
+
+// 2: Ok.
+for (auto iterator = index.begin(); iterator != index.end(); ++iterator) {
+ const hashset<int>& values = iterator->second;
+}
+
+// 3: Ok.
+foreachpair (const string& key, const hashset<int>& values, index) {}
+foreachvalue (const hashset<int>& values, index) {}
+foreachkey (const string& key, index) {}
+
+// 4: Avoid aliases in most circumstances as they can be dangerous.
+// This is an example of a dangling alias!
+vector<string> strings{"hello"};
+
+string& s = strings[0];
+
+strings.erase(strings.begin());
+
+s += "world"; // THIS IS A DANGLING REFERENCE!
+~~~
+
+## File Headers
+
+* Mesos source files must contain the "ASF" header:
+
+ /**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+* Stout and libprocess source files must contain the "Apache License Version 2.0" header:
+
+ /**
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+
+## C++11
+
+We support C++11 and require GCC 4.8+ or Clang 3.5+ compilers. The whitelist of supported C++11 features is:
+
+* Static assertions.
+* Multiple right angle brackets.
+* Type inference (`auto` and `decltype`). The main goal is to increase code readability. This is safely the case if the exact same type omitted on the left is already fully stated on the right. Here are several examples:
+
+~~~{.cpp}
+// 1: OK.
+const auto i = values.find(keys.front());
+// Compare with
+const typename map::iterator i = values.find(keys.front());
+
+// 2: OK.
+auto names = shared_ptr<list<string>>(new list<string>());
+// Compare with
+shared_ptr<list<string>> names = shared_ptr<list<string>>(new list<string>());
+
+// 3: Don't use.
+auto authorizer = LocalAuthorizer::create(acls);
+// Compare with
+Try<Owned<LocalAuthorizer>> authorizer = LocalAuthorizer::create();
+~~~
+
+* Rvalue references.
+* Explicitly-defaulted functions.
+* Variadic templates.
+* Delegating constructors.
+* Mutexes.
+ * `std::mutex`
+ * `std::lock_guard<std::mutex>`
+ * `std::unique_lock<std::mutex>`
+* Atomics (`std::atomic`)
+ * The standard defines a number of predefined typedefs for atomic types (e.g., `std::atomic_int`), in addition to `std::atomic<T>`. When a typedef is available, it should be preferred over explicit template specialization of `std::atomic<T>`.
+ * When reading from and writing to atomic values, the `load` and `store` member functions should be used instead of the overloads of `operator T()` and `operator=`. Being explicit helps to draw the reader's attention to the fact that atomic values are being manipulated.
+* Shared from this.
+ * `class T : public std::enable_shared_from_this<T>`
+ * `shared_from_this()`
+* Lambdas!
+ * Don't put a space between the capture list and the parameter list:
+
+~~~{.cpp}
+// 1: OK.
+[]() { ...; };
+
+// 2: Don't use.
+[] () { ...; };
+~~~
+
+ * Prefer default capture by value, explicit capture by value, then capture by reference. To avoid dangling-pointer bugs, *never* use default capture by reference:
+
+~~~{.cpp}
+// 1: OK.
+[=]() { ... }; // Default capture by value.
+[n]() { ... }; // Explicit capture by value.
+[&n]() { ... }; // Explicit capture by reference.
+[=, &n]() { ... }; // Default capture by value, explicit capture by reference.
+
+// 2: Don't use.
+[&]() { ... }; // Default capture by reference.
+~~~
+
+ * Use `mutable` only when absolutely necessary.
+
+~~~{.cpp}
+// 1: OK.
+[]() mutable { ...; };
+~~~
+
+ * Feel free to ignore the return type by default, adding it as necessary to appease the compiler or be more explicit for the reader.
+
+~~~{.cpp}
+// 1: OK.
+[]() { return true; };
+[]() -> bool { return ambiguous(); };
+~~~
+
+ * Feel free to use `auto` when naming a lambda expression:
+
+~~~{.cpp}
+// 1: OK.
+auto lambda = []() { ...; };
+~~~
+
+ * Format lambdas similar to how we format functions and methods. Feel free to let lambdas be one-liners:
+
+~~~{.cpp}
+// 1: OK.
+auto lambda = []() {
+ ...;
+};
+
+// 2: OK.
+auto lambda = []() { ...; };
+~~~
+
+ * Feel free to inline lambdas within function arguments:
+
+~~~{.cpp}
+instance.method([]() {
+ ...;
+});
+~~~
+
+ * Chain function calls on a newline after the closing brace of the lambda and the closing parenthesis of function call:
+
+~~~{.cpp}
+// 1: OK.
+instance
+ .method([]() {
+ ...;
+ })
+ .then([]() { ...; })
+ .then([]() {
+ ...;
+ });
+
+// 2: OK (when no chaining, compare to 1).
+instance.method([]() {
+ ...;
+});
+
+// 3: OK (if no 'instance.method').
+function([]() {
+ ...;
+})
+.then([]() { ...; })
+.then([]() {
+ ...;
+});
+
+// 3: OK (but prefer 1).
+instance.method([]() {
+ ...;
+})
+.then([]() { ...; })
+.then([]() {
+ ...;
+});
+~~~
+
+ * Wrap capture lists independently of parameters, *use the same formatting as if the capture list were template parameters*:
+
+~~~{.cpp}
+// 1: OK.
+function([&capture1, &capture2, &capture3](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+});
+
+function(
+ [&capture1, &capture2, &capture3](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+});
+
+auto lambda = [&capture1, &capture2, &capture3](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+};
+
+auto lambda =
+ [&capture1, &capture2, &capture3](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+};
+
+// 2: OK (when capture list is longer than 80 characters).
+function([
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1, const T2& p2) {
+ ...;
+});
+
+auto lambda = [
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1, const T2& p2) {
+ ...;
+};
+
+// 3: OK (but prefer 2).
+function([
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](const T1& p1, const T2& t2) {
+ ...;
+});
+
+auto lambda = [
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](const T1& p1, const T2& p2) {
+ ...;
+};
+
+// 3: Don't use.
+function([&capture1,
+ &capture2,
+ &capture3,
+ &capture4](const T1& p1, const T2& p2) {
+ ...;
+});
+
+auto lambda = [&capture1,
+ &capture2,
+ &capture3,
+ &capture4](const T1& p1, const T2& p2) {
+ ...;
+ };
+
+// 4: Don't use.
+function([&capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+});
+
+auto lambda = [&capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1, const T2& p2, const T3& p3) {
+ ...;
+};
+
+// 5: Don't use.
+function([&capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3) {
+ ...;
+ });
+
+auto lambda = [&capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3) {
+ ...;
+};
+
+// 6: OK (parameter list longer than 80 characters).
+function([&capture1, &capture2, &capture3](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3,
+ const T4& p4) {
+ ...;
+});
+
+auto lambda = [&capture1, &capture2, &capture3](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3,
+ const T4& p4) {
+ ...;
+};
+
+// 7: OK (capture and parameter lists longer than 80 characters).
+function([
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3,
+ const T4& p4) {
+ ...;
+});
+
+auto lambda = [
+ &capture1,
+ &capture2,
+ &capture3,
+ &capture4](
+ const T1& p1,
+ const T2& p2,
+ const T3& p3,
+ const T4& p4) {
+ ...;
+};
+~~~
+
+* Unrestricted Union.
+
+ Like the pre-existing `union`, we can overlap storage allocation for objects that never exist simultaneously. However, with C++11 we are no longer *restricted to having only non-POD types in unions*. Adding non-POD types to unions complicates things, however, because we need to make sure to properly call constructors and destructors. Therefore, only use unrestricted unions (i.e., unions with non-POD types) when the union has only a single field. What does this buy us? Now we can avoid dynamic memory allocations for "container" like types, e.g., `Option`, `Try`, `Result`, etc. In effect, we treat the union like a dynamic allocation, calling *placement new*, `new (&t) T(...)` anyplace we would have just called `new T(...)` and the destructor `t.~T()` anyplace we would have called `delete t`.
+
+* Constant expressions.
+
+ Constant expressions allow the declaration of static non-POD objects while eliminating the unpredictable runtime initialization and destruction issues normally encountered, helping eliminate macros and hard-coded literals without sacrificing performance and type safety. Changes which require converting from `constexpr` to `const` can propagate through the dependency tree requiring that dependent `constexpr` uses also be converted to `const`, hence we avoid using `constexpr` in complex functions.
+
+ `constexpr` behaves as a combination of `inline` and `const` and hence must be defined before use in another `constexpr`.
+
+ Prefer `constexpr` to `const` for all constant POD declarations, `constexpr` `char` arrays are preferred to `const` `string` literals.
+
+~~~{.cpp}
+ // OK
+ constexpr char LITERAL[] = "value";
+
+ // Not OK - not available at compile time for optimization and
+ // definition required in a separate compilation module.
+ const char LITERAL[];
+
+ // Not OK - uncertain initialization order, cannot be used in other
+ // constexpr statements.
+ const string LITERAL("value");
+~~~
+
+ `constexpr` functions are evaluated at compile time if all their arguments are constant expressions. Otherwise they default to initialization at runtime. However `constexpr` functions are limited in that they cannot perform dynamic casts, memory allocation or calls to non-constexpr functions. Prefer `constexpr` over const inline functions.
+
+~~~{.cpp}
+ constexpr size_t MIN = 200;
+ constexpr size_t MAX = 1000;
+ constexpr size_t SPAN() { return MAX-MIN; }
+ int array[SPAN()];
+~~~
+
+Const expression constructors allow object initialization at compile time provided that all the constructor arguments are `constexpr` and the constructor body is empty, i.e. all initialization is performed in the initialization list. Classes which provide `constexpr` constructors should normally also provide `constexpr` copy constructors to allow the class to be used in the return value from a `constexpr` function.
+
+~~~{.cpp}
+ class C
+ {
+ public:
+ constexpr C(int _i) : i(_i) {};
+ constexpr C(const C& c) : i(c.i) {}
+ private:
+ const int i;
+ };
+~~~
+
+ C++11 does not provide `constexpr string` or `constexpr` containers in the STL and hence `constexpr` cannot be used for any class using stout's Error() class.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/clang-format.md
----------------------------------------------------------------------
diff --git a/docs/clang-format.md b/docs/clang-format.md
index e8f593f..0ad8dc3 100644
--- a/docs/clang-format.md
+++ b/docs/clang-format.md
@@ -6,7 +6,7 @@ layout: documentation
[ClangFormat](http://llvm.org/releases/3.5.1/tools/clang/docs/ClangFormat.html) is an automatic source code formatting tool which helps us focus on the code rather than the formatting.
-> The provided configurations try to honor the [Mesos C++ Style Guide](http://mesos.apache.org/documentation/latest/mesos-c++-style-guide/) as much as possible, but there are some limitations which require manual attention. Even with these limitations however, ClangFormat will be extremely useful for your workflow!
+> The provided configurations try to honor the [Mesos C++ Style Guide](http://mesos.apache.org/documentation/latest/c++-style-guide/) as much as possible, but there are some limitations which require manual attention. Even with these limitations however, ClangFormat will be extremely useful for your workflow!
## Setup
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/containerizer.md
----------------------------------------------------------------------
diff --git a/docs/containerizer.md b/docs/containerizer.md
new file mode 100644
index 0000000..ea19558
--- /dev/null
+++ b/docs/containerizer.md
@@ -0,0 +1,82 @@
+---
+layout: documentation
+---
+
+# Mesos Containerizer
+
+The MesosContainerizer provides lightweight containerization and
+resource isolation of executors using Linux-specific functionality
+such as control cgroups and namespaces. It is composable so operators
+can selectively enable different isolators.
+
+It also provides basic support for POSIX systems (e.g., OSX) but
+without any actual isolation, only resource usage reporting.
+
+### Shared Filesystem
+
+The SharedFilesystem isolator can optionally be used on Linux hosts to
+enable modifications to each container's view of the shared
+filesystem.
+
+The modifications are specified in the ContainerInfo included in the
+ExecutorInfo, either by a framework or by using the
+--default\_container\_info slave flag.
+
+ContainerInfo specifies Volumes which map parts of the shared
+filesystem (host\_path) into the container's view of the filesystem
+(container\_path), as read-write or read-only. The host\_path can be
+absolute, in which case it will make the filesystem subtree rooted at
+host\_path also accessible under container\_path for each container.
+If host\_path is relative then it is considered as a directory
+relative to the executor's work directory. The directory will be
+created and permissions copied from the corresponding directory (which
+must exist) in the shared filesystem.
+
+The primary use-case for this isolator is to selectively make parts of
+the shared filesystem private to each container. For example, a
+private "/tmp" directory can be achieved with host\_path="tmp" and
+container\_path="/tmp" which will create a directory "tmp" inside the
+executor's work directory (mode 1777) and simultaneously mount it as
+/tmp inside the container. This is transparent to processes running
+inside the container. Containers will not be able to see the host's
+/tmp or any other container's /tmp.
+
+### Pid Namespace
+
+The Pid Namespace isolator can be used to isolate each container in
+a separate pid namespace with two main benefits:
+1. Visibility: Processes running in the container (executor and
+ descendants) are unable to see or signal processes outside the
+ namespace.
+2. Clean termination: Termination of the leading process in a pid
+ namespace will result in the kernel terminating all other processes
+ in the namespace.
+
+The Launcher will use (2) during destruction of a container in
+preference to the freezer cgroup, avoiding known kernel issues related
+to freezing cgroups under OOM conditions.
+
+/proc will be mounted for containers so tools such as 'ps' will work
+correctly.
+
+
+### Posix Disk Isolator
+
+The Posix Disk isolator provides basic disk isolation. It is able to
+report the disk usage for each sandbox and optionally enforce the disk
+quota. It can be used on both Linux and OS X.
+
+To enable the Posix Disk isolator, append `posix/disk` to the
+`--isolation` flag when starting the slave.
+
+By default, the disk quota enforcement is disabled. To enable it,
+specify `--enforce_container_disk_quota` when starting the slave.
+
+The Posix Disk isolator reports disk usage for each sandbox by
+periodically running the `du` command. The disk usage can be retrieved
+from the resource statistics endpoint (`/monitor/statistics.json`).
+
+The interval between two `du`s can be controlled by the slave flag
+`--container_disk_watch_interval`. For example,
+`--container_disk_watch_interval=1mins` sets the interval to be 1
+minute. The default interval is 15 seconds.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/documentation-guide.md
----------------------------------------------------------------------
diff --git a/docs/documentation-guide.md b/docs/documentation-guide.md
new file mode 100644
index 0000000..9548bca
--- /dev/null
+++ b/docs/documentation-guide.md
@@ -0,0 +1,37 @@
+# Mesos Documentation Guide
+
+Documentation is an integral part of every good feature. It describes the intended usage and enables new users to start using and understanding the feature.
+
+We have three different kinds of documentation:
+
+
+1. [MarkDown User Guides](/documentation/latest/markdown-style-guide/)
+
+ User guides and non-code technical documentation are stored in markdown files in the `docs/` folder. These files get rendered for the [online documentation](http://mesos.apache.org/documentation/latest/).
+
+ We will accept small documentation changes on [Github via a pull request](https://github.com/apache/mesos), larger documentation changes should go through the (reviewboard)[https://reviews.apache.org/groups/mesos/]
+
+
+2. [Doxygen API Documentation and Developer Guides as part of source code](/documentation/latest/doxygen-style-guide/)
+
+ Doxygen API documentation needs only to be applied to source code parts that
+ constitute an interface for which we want to generate Mesos API documentation
+ files. Implementation code that does not participate in this should still be
+ enhanced by source code comments as appropriate, but these comments should not follow the doxygen style.
+
+ Substantial libraries, components, and subcomponents of the Mesos system such as
+ stout, libprocess, master, slave, containerizer, allocator, and others
+ should have an overview page in markdown format that explains their
+ purpose, overall structure, and general use. This can even be a complete developer guide.
+
+
+3. Regular source code documentation
+
+ All other source code comments must follow the [Google Style Guide](https://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Comments).
+
+
+## Conventions
+
+We follow the [IETF RFC 2119](https://www.ietf.org/rfc/rfc2119.txt)
+on how to use words such as "must", "should", "can",
+and other requirement-related notions.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/doxygen-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/doxygen-style-guide.md b/docs/doxygen-style-guide.md
new file mode 100644
index 0000000..1bd3b3d
--- /dev/null
+++ b/docs/doxygen-style-guide.md
@@ -0,0 +1,196 @@
+# Apache Mesos Doxygen Style Guide
+
+This guide introduces a consistent style
+for [documenting Mesos source code](http://mesos.apache.org/api/latest/c++)
+using [Doxygen](http://www.doxygen.org).
+There is an ongoing, incremental effort with the goal to document all public Mesos, libprocess, and stout APIs this way.
+For now, existing code may not follow these guidelines, but new code should.
+
+
+## Source Code Documentation Syntax
+
+Doxygen documentation needs only to be applied to source code parts that
+constitute an interface for which we want to generate Mesos API documentation
+files. Implementation code that does not participate in this should still be
+enhanced by source code comments as appropriate, but these comments should not follow the doxygen style.
+
+We follow the [Javadoc syntax](http://en.wikipedia.org/wiki/Javadoc) to mark comment blocks.
+These have the general form:
+
+
+<!-- NOTE: In order to show the comments as part of the code blocks in
+this we must use indentation instead of fenced code blocks (i.e.,
+~~~{.cpp}) because doxygen will remove all of the comments and just
+render the code. -->
+
+
+ /**
+ * Brief summary.
+ *
+ * Detailed description. More detail.
+ * @see Some reference
+ *
+ * @param <name> Parameter description.
+ * @return Return value description.
+ */
+
+Example:
+
+ /**
+ * Returns a compressed version of a string.
+ *
+ * Compresses an input string using the foobar algorithm.
+ *
+ * @param uncompressed The input string.
+ * @return A compressed version of the input string.
+ */
+ std::string compress(const std::string& uncompressed);
+
+
+### Doxygen Tags
+
+This is the allowed set of doxygen tags that can be used.
+
+ * [\@param](http://doxygen.org/manual/commands.html#cmdparam) Describes function parameters.
+ * [\@return](http://doxygen.org/manual/commands.html#cmdreturn) Describes return values.
+ * [\@see](http://doxygen.org/manual/commands.html#cmdsa) Describes a cross-reference to classes, functions, methods, variables, files or URL.
+
+Example:
+
+ /**
+ * Available kinds of implementations.
+ *
+ * @see process::network::PollSocketImpl
+ */
+
+ * [\@file](http://doxygen.org/manual/commands.html#cmdfile) Describes a refence to a file. It is required when documenting global functions, variables, typedefs, or enums in separate files.
+ * [\@link](http://doxygen.org/manual/commands.html#cmdlink) and [\@endlink](http://doxygen.org/manual/commands.html#cmdendlink) Describes a link to a file, class, or member.
+ * [\@example](http://doxygen.org/manual/commands.html#cmdexample) Describes source code examples.
+ * [\@image](http://doxygen.org/manual/commands.html#cmdimage) Describes an image.
+
+ * When following these links be aware that the doxygen documentation is using another syntax in that \@param is explained as \\param.
+
+
+### Wrapping
+
+We wrap long descriptions using four spaces on the next line.
+
+ @param uncompressed The input string that requires
+ a very long description and an even longer
+ description on this line as well.
+
+
+### Constants and Variables
+
+Example:
+
+ /**
+ * Prefix used to name Docker containers in order to distinguish
+ * those created by Mesos from those created manually.
+ */
+ extern const std::string DOCKER_NAME_PREFIX;
+
+
+#### Fields
+
+Example:
+
+ /**
+ * The parent side of the pipe for stdin.
+ * If the mode is not PIPE, None will be stored.
+ * **NOTE**: stdin is a macro on some systems, hence this name instead.
+ */
+ Option<int> in;
+
+
+### Functions and Methods
+
+Example:
+
+ /**
+ * Forks a subprocess and execs the specified 'path' with the
+ * specified 'argv', redirecting stdin, stdout, and stderr as
+ * specified by 'in', 'out', and 'err' respectively.
+ *
+ * If 'setup' is not None, runs the specified function after forking
+ * but before exec'ing. If the return value of 'setup' is non-zero
+ * then that gets returned in 'status()' and we will not exec.
+ *
+ * @param path Relative or absolute path in the filesytem to the
+ * executable.
+ * @param argv Argument vector to pass to exec.
+ * @param in Redirection specification for stdin.
+ * @param out Redirection specification for stdout.
+ * @param err Redirection specification for stderr.
+ * @param flags Flags to be stringified and appended to 'argv'.
+ * @param environment Environment variables to use for the new
+ * subprocess or if None (the default) then the new subprocess
+ * will inherit the environment of the current process.
+ * @param setup Function to be invoked after forking but before
+ * exec'ing. NOTE: Take extra care not to invoke any
+ * async unsafe code in the body of this function.
+ * @param clone Function to be invoked in order to fork/clone the
+ * subprocess.
+ * @return The subprocess or an error if one occurred.
+ */
+ Try<Subprocess> subprocess(
+ const std::string& path,
+ std::vector<std::string> argv,
+ const Subprocess::IO& in = Subprocess::FD(STDIN_FILENO),
+ const Subprocess::IO& out = Subprocess::FD(STDOUT_FILENO),
+ const Subprocess::IO& err = Subprocess::FD(STDERR_FILENO),
+ const Option<flags::FlagsBase>& flags = None(),
+ const Option<std::map<std::string, std::string>>& environment = None(),
+ const Option<lambda::function<int()>>& setup = None(),
+ const Option<lambda::function<
+ pid_t(const lambda::function<int()>&)>>& clone = None());
+
+
+### Classes and Structs
+
+Example:
+
+ /**
+ * Represents a fork() exec()ed subprocess. Access is provided to the
+ * input / output of the process, as well as the exit status. The
+ * input / output file descriptors are only closed after:
+ * 1. The subprocess has terminated.
+ * 2. There are no longer any references to the associated
+ * Subprocess object.
+ */
+ class Subprocess
+ {
+ public:
+
+
+## Library and Component Overview Pages and Developer Guides
+
+Substantial libraries, components, and subcomponents of the Mesos system such as
+stout, libprocess, master, slave, containerizer, allocator, and others
+should have an overview page in markdown format that explains their
+purpose, overall structure, and general use. This can even be a complete developer guide.
+
+This page must be located in the top directory of the library/component and named "README.md".
+
+The first line in such a document must be a section heading bearing the title which will appear in the generated Doxygen index.
+Example: "# Libprocess Developer Guide"
+
+### Example Code
+
+C++ code examples should be enclosed by '~~~{.cpp}'.
+
+Example:
+
+~~~{.cpp}
+int main(int argc, char** argv)
+{
+ ...
+}
+~~~
+
+**NOTE**: Because of shortcomings of Doxygen's markdown parser we currently use indentation for wrapping all non C++ code blocks.
+
+
+## Building Doxygen Documentation
+
+As of right now, the Doxygen documentation should be built from the *build* subdirectory using *doxygen ../Doxyfile* . The documentation will then be generated into the *./doxygen* subdirectory.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/effective-code-reviewing.md
----------------------------------------------------------------------
diff --git a/docs/effective-code-reviewing.md b/docs/effective-code-reviewing.md
index ff6507c..25d7126 100644
--- a/docs/effective-code-reviewing.md
+++ b/docs/effective-code-reviewing.md
@@ -22,7 +22,7 @@ to consider before sending review requests:
change clear in the review request, so the reviewer is not left
guessing. It is highly recommended to attach a JIRA issue with your
review for additional context.
-4. **Follow the [style guide](http://mesos.apache.org/documentation/latest/mesos-c++-style-guide/)
+4. **Follow the [style guide](http://mesos.apache.org/documentation/latest/c++-style-guide/)
and the style of code around you**.
5. **Do a self-review of your changes before publishing**: Approach it
from the perspective of a reviewer with no context. Is it easy to figure
@@ -88,4 +88,4 @@ review process.
6. **For reviewers, be thorough when giving a 'ship it'**: understand that
reviewing requires a substantial investment of time and focus:
a. You are expected to understand and support code that you're giving a 'ship it' to.
- b. You are expected to be accountable for the quality of the code you've given a 'ship it' to.
\ No newline at end of file
+ b. You are expected to be accountable for the quality of the code you've given a 'ship it' to.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/frameworks.md
----------------------------------------------------------------------
diff --git a/docs/frameworks.md b/docs/frameworks.md
new file mode 100644
index 0000000..60405b1
--- /dev/null
+++ b/docs/frameworks.md
@@ -0,0 +1,34 @@
+---
+layout: documentation
+---
+# Software projects built on Mesos
+
+## Long Running Services
+
+* [Aurora](http://aurora.incubator.apache.org) is a service scheduler that runs on top of Mesos, enabling you to run long-running services that take advantage of Mesos' scalability, fault-tolerance, and resource isolation.
+* [Marathon](https://github.com/mesosphere/marathon) is a private PaaS built on Mesos. It automatically handles hardware or software failures and ensures that an app is "always on".
+* [Singularity](https://github.com/HubSpot/Singularity) is a scheduler (HTTP API and web interface) for running Mesos tasks: long running processes, one-off tasks, and scheduled jobs.
+* [SSSP](https://github.com/mesosphere/sssp) is a simple web application that provides a white-label "Megaupload" for storing and sharing files in S3.
+
+## Big Data Processing
+
+* [Cray Chapel](https://github.com/nqn/mesos-chapel) is a productive parallel programming language. The Chapel Mesos scheduler lets you run Chapel programs on Mesos.
+* [Dpark](https://github.com/douban/dpark) is a Python clone of Spark, a MapReduce-like framework written in Python, running on Mesos.
+* [Exelixi](https://github.com/mesosphere/exelixi) is a distributed framework for running genetic algorithms at scale.
+* [Hadoop](https://github.com/mesos/hadoop) Running Hadoop on Mesos distributes MapReduce jobs efficiently across an entire cluster.
+* [Hama](http://wiki.apache.org/hama/GettingStartedMesos) is a distributed computing framework based on Bulk Synchronous Parallel computing techniques for massive scientific computations e.g., matrix, graph and network algorithms.
+* [MPI](https://github.com/mesosphere/mesos-hydra) is a message-passing system designed to function on a wide variety of parallel computers.
+* [Spark](http://spark.incubator.apache.org/) is a fast and general-purpose cluster computing system which makes parallel jobs easy to write.
+* [Storm](https://github.com/mesosphere/storm-mesos) is a distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing.
+
+## Batch Scheduling
+
+* [Chronos](https://github.com/airbnb/chronos) is a distributed job scheduler that supports complex job topologies. It can be used as a more fault-tolerant replacement for Cron.
+* [Jenkins](https://github.com/jenkinsci/mesos-plugin) is a continuous integration server. The mesos-jenkins plugin allows it to dynamically launch workers on a Mesos cluster depending on the workload.
+* [JobServer](http://www.grandlogic.com/content/html_docs/jobserver.html) is a distributed job scheduler and processor which allows developers to build custom batch processing Tasklets using point and click web UI.
+
+## Data Storage
+
+* [Cassandra](https://github.com/mesosphere/cassandra-mesos) is a performant and highly available distributed database. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data.
+* [ElasticSearch](https://github.com/mesosphere/elasticsearch-mesos) is a distributed search engine. Mesos makes it easy to run and scale.
+* [Hypertable](https://code.google.com/p/hypertable/wiki/Mesos) is a high performance, scalable, distributed storage and processing system for structured and unstructured data.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/home.md
----------------------------------------------------------------------
diff --git a/docs/home.md b/docs/home.md
index e7d6930..af88625 100644
--- a/docs/home.md
+++ b/docs/home.md
@@ -6,14 +6,14 @@ layout: documentation
## Mesos Fundamentals
-* [Mesos Architecture](/documentation/latest/mesos-architecture/) providing an overview of Mesos concepts.
-* [Video and Slides of Mesos Presentations](/documentation/latest/mesos-presentations/)
+* [Mesos Architecture](/documentation/latest/architecture/) providing an overview of Mesos concepts.
+* [Video and Slides of Mesos Presentations](/documentation/latest/presentations/)
## Running Mesos
* [Getting Started](/documentation/latest/getting-started/) for basic instructions on compiling and installing Mesos.
* [Configuration](/documentation/latest/configuration/) for command-line arguments.
-* [Mesos Containerizer](/documentation/latest/mesos-containerizer/) default containerizer, supports both Linux and POSIX systems.
+* [Mesos Containerizer](/documentation/latest/containerizer/) default containerizer, supports both Linux and POSIX systems.
* [Docker Containerizer](/documentation/latest/docker-containerizer/) for launching a Docker image as a Task, or as an Executor.
* [External Containerizer](/documentation/latest/external-containerizer/) for custom containerization implementations.
* [Framework Authentication](/documentation/latest/authentication/)
@@ -27,7 +27,7 @@ layout: documentation
* [Slave Recovery](/documentation/latest/slave-recovery/) for doing seamless upgrades.
* [Maintenance](/documentation/latest/maintenance/) for performing maintenance on a Mesos cluster.
* [Tools](/documentation/latest/tools/) for setting up and running a Mesos cluster.
-* [SSL](/documentation/latest/mesos-ssl/) for enabling and enforcing SSL communication.
+* [SSL](/documentation/latest/ssl/) for enabling and enforcing SSL communication.
## Advanced Features
@@ -40,13 +40,13 @@ layout: documentation
## Running Mesos Frameworks
- * [Mesos frameworks](/documentation/latest/mesos-frameworks/) for a list of apps built on top of Mesos and instructions on how to run them.
+ * [Mesos frameworks](/documentation/latest/frameworks/) for a list of apps built on top of Mesos and instructions on how to run them.
## Developing Mesos Frameworks
* [Framework Development Guide](/documentation/latest/app-framework-development-guide/) describes how to build applications on top of Mesos.
* [Reconciliation](/documentation/latest/reconciliation/) for ensuring a framework's state remains eventually consistent in the face of failures.
-* [Scheduler HTTP API](/documentation/latest/scheduler_http_api/) describes the new HTTP API for communication between schedulers and the Mesos master.
+* [Scheduler HTTP API](/documentation/latest/scheduler-http-api/) describes the new HTTP API for communication between schedulers and the Mesos master.
* [Javadoc](/api/latest/java/) documents the Mesos Java API.
* [Doxygen](/api/latest/c++/namespacemesos.html) documents the Mesos C++ API.
* [Developer Tools](/documentation/latest/tools/) for hacking on Mesos or writing frameworks.
@@ -60,17 +60,17 @@ layout: documentation
* [Reporting an Issue, Improvement, or Feature](/documentation/latest/reporting-a-bug/) for getting started with JIRA.
* [Submitting a Patch](/documentation/latest/submitting-a-patch/) for getting started with ReviewBoard and our tooling around it.
-* [Testing Patterns](/documentation/latest/mesos-testing-patterns/) for tips and tricks used in Mesos tests.
+* [Testing Patterns](/documentation/latest/testing-patterns/) for tips and tricks used in Mesos tests.
* [Effective Code Reviewing](/documentation/latest/effective-code-reviewing/) guidelines, tips, and learnings for how to do effective code reviews.
* [Engineering Principles and Practices](/documentation/latest/engineering-principles-and-practices/) to serve as a shared set of project-level values for the community.
* [Committing](/documentation/latest/committing/) guidelines for committing changes.
* [Committers and Maintainers](/documentation/latest/committers/) a listing of project committers and component maintainers; useful when seeking feedback.
* [Doxygen](/api/latest/c++/) documents the internal Mesos APIs.
-* [Documentation Guide](/documentation/latest/mesos-documentation-guide/)
- * [C++ Style Guide](/documentation/latest/mesos-c++-style-guide/)
- * [Doxygen Style Guide](/documentation/latest/mesos-doxygen-style-guide/)
- * [Markdown Style Guide](/documentation/latest/mesos-markdown-style-guide/)
-* [Development Roadmap](/documentation/latest/mesos-roadmap/)
+* [Documentation Guide](/documentation/latest/documentation-guide/)
+ * [C++ Style Guide](/documentation/latest/c++-style-guide/)
+ * [Doxygen Style Guide](/documentation/latest/doxygen-style-guide/)
+ * [Markdown Style Guide](/documentation/latest/markdown-style-guide/)
+* [Development Roadmap](/documentation/latest/roadmap/)
* [Release Guide](/documentation/latest/release-guide/)
## More Info about Mesos
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/markdown-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/markdown-style-guide.md b/docs/markdown-style-guide.md
new file mode 100644
index 0000000..fe9ffef
--- /dev/null
+++ b/docs/markdown-style-guide.md
@@ -0,0 +1,118 @@
+# Mesos Markdown Style Guide
+
+This guide introduces a consistent documentation style to be used across the entire non-code documentation.
+User guides and non-code technical documentation are stored in markdown files in the `docs/` folder. These files get rendered for the [online documentation](http://mesos.apache.org/documentation/latest/).
+
+**NOTE:** As of right now this is work in progress and the existing documentation might not yet comply to this style.
+
+
+## What to document?
+
+Any new substantial feature should be documented in its own markdown file.
+If the link between source code and documentation is not obvious, consider inserting a short code comment stating that there is non-code documentation that needs to be kept in sync and indicating where it is located.
+
+
+## Keep documentation and style-guides in sync with code.
+
+When changing code consider whether you need to update the documentation.
+This is especially relevant when introducing new or updating existing command line flags.
+These should be reflected in `configuration.md`!
+
+
+## Code Examples
+
+Code examples should be specified as follows:
+
+ ~~~{.cpp}
+ int main(int argc, char** argv)
+ {
+ ....
+ }
+ ~~~
+
+**NOTE:** Because of shortcomings of Doxygen's markdown parser we currently use indentation for wrapping all non C++ code blocks.
+
+## Notes/Emphasis
+
+Notes are used to highlight important parts of the text and should be specified as follows.
+
+~~~{.txt}
+**NOTE:** Short note.
+Continued longer note.
+~~~
+
+We use single backticks to highlight individual words in a sentence such as certain identifiers:
+
+~~~{.txt}
+Use the default `HierarchicalDRF` allocator....
+~~~
+
+
+## Commands
+
+We use single backticks to highlight sample commands as follows:
+
+~~~{.txt}
+`mesos-master --help`
+~~~
+
+
+## Files/Path
+
+Files and path references should be specified as follows:
+
+~~~{.text}
+Remember you can also use the `file:///path/to/file` or `/path/to/file`
+~~~
+
+
+## Tables
+
+In order to avoid problems with markdown formatting we should specify tables in html directly:
+
+~~~{.html}
+<table class="table table-striped">
+ <thead>
+ <tr>
+ <th width="30%">
+ Flag
+ </th>
+ <th>
+ Explanation
+ </th>
+ </thead>
+ <tr>
+ <td>
+ --ip=VALUE
+ </td>
+ <td>
+ IP address to listen on
+ </td>
+ </tr>
+ <tr>
+ <td>
+ --[no-]help
+ </td>
+ <td>
+ Prints this help message (default: false)
+
+ </td>
+ </tr>
+</table>
+~~~
+
+
+## Indendation and Whitespace
+
+We use no extra indentation in markdown files.
+We have one new line after section headings and two blank lines
+in between sections.
+
+~~~{.txt}
+... end of previous section.
+
+
+## New Section
+
+Beginning of new section ....
+~~~
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-architecture.md
----------------------------------------------------------------------
diff --git a/docs/mesos-architecture.md b/docs/mesos-architecture.md
deleted file mode 100644
index 439a56d..0000000
--- a/docs/mesos-architecture.md
+++ /dev/null
@@ -1,32 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Architecture
-
-
-
-The above figure shows the main components of Mesos. Mesos consists of a *master* daemon that manages *slave* daemons running on each cluster node, and *mesos applications* (also called *frameworks*) that run *tasks* on these slaves.
-
-The master enables fine-grained sharing of resources (cpu, ram, ...) across applications by making them *resource offers*. Each resource offer contains a list of <slave ID, resource1: amount1, resource2, amount2, ...>. The master decides *how many* resources to offer to each framework according to a given organizational policy, such as fair sharing, or strict priority. To support a diverse set of policies, the master employs a modular architecture that makes it easy to add new allocation modules via a plugin mechanism.
-
-A framework running on top of Mesos consists of two components: a *scheduler* that registers with the master to be offered resources, and an *executor* process that is launched on slave nodes to run the framework's tasks (see the [App/Framework development guide](app-framework-development-guide.md) for more details about application schedulers and executors). While the master determines **how many** resources are offered to each framework, the frameworks' schedulers select **which** of the offered resources to use. When a frameworks accepts offered resources, it passes to Mesos a description of the tasks it wants to run on them. In turn, Mesos launches the tasks on the corresponding slaves.
-
-## Example of resource offer
-
-The figure below shows an example of how a framework gets scheduled to run a task.
-
-
-
-Let's walk through the events in the figure.
-
-1. Slave 1 reports to the master that it has 4 CPUs and 4 GB of memory free. The master then invokes the allocation policy module, which tells it that framework 1 should be offered all available resources.
-1. The master sends a resource offer describing what is available on slave 1 to framework 1.
-1. The framework's scheduler replies to the master with information about two tasks to run on the slave, using <2 CPUs, 1 GB RAM> for the first task, and <1 CPUs, 2 GB RAM> for the second task.
-1. Finally, the master sends the tasks to the slave, which allocates appropriate resources to the framework's executor, which in turn launches the two tasks (depicted with dotted-line borders in the figure). Because 1 CPU and 1 GB of RAM are still unallocated, the allocation module may now offer them to framework 2.
-
-In addition, this resource offer process repeats when tasks finish and new resources become free.
-
-While the thin interface provided by Mesos allows it to scale and allows the frameworks to evolve independently, one question remains: how can the constraints of a framework be satisfied without Mesos knowing about these constraints? For example, how can a framework achieve data locality without Mesos knowing which nodes store the data required by the framework? Mesos answers these questions by simply giving frameworks the ability to **reject** offers. A framework will reject the offers that do not satisfy its constraints and accept the ones that do. In particular, we have found that a simple policy called delay scheduling, in which frameworks wait for a limited time to acquire nodes storing the input data, yields nearly optimal data locality.
-
-You can also read much more about the Mesos architecture in this [technical paper](http://mesos.berkeley.edu/mesos_tech_report.pdf).
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-c++-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/mesos-c++-style-guide.md b/docs/mesos-c++-style-guide.md
deleted file mode 100644
index 0b61891..0000000
--- a/docs/mesos-c++-style-guide.md
+++ /dev/null
@@ -1,579 +0,0 @@
----
-layout: documentation
----
-
-# Mesos C++ Style Guide
-
-The Mesos codebase follows the [Google C++ Style Guide](http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml) with some notable differences, as described below. Note that the [clang-format](/documentation/latest/clang-format/) tool can be helpful to ensure that some of the mechanical style rules are obeyed.
-
-## Scoping
-
-### Namespaces
-* We avoid `using namespace foo` statements as it is not explicit about which symbols are pulled in, and it can often pull in a lot of symbols, which sometimes lead to conflicts.
-* It is OK to use namespace aliases to help pull in sub-namespaces, such as `namespace http = process::http;`. These should only be present at the top of the .cpp file.
-
-## Naming
-
-### Variable Names
-* We use [lowerCamelCase](http://en.wikipedia.org/wiki/CamelCase#Variations_and_synonyms) for variable names (Google uses snake_case, and their class member variables have trailing underscores).
-* We prepend constructor and function arguments with a leading underscore to avoid ambiguity and / or shadowing:
-
-~~~{.cpp}
-Try(State _state, T* _t = NULL, const std::string& _message = "")
- : state(_state), t(_t), message(_message) {}
-~~~
-
-* Prefer trailing underscores for use as member fields (but not required). Some trailing underscores are used to distinguish between similar variables in the same scope (think prime symbols), *but this should be avoided as much as possible, including removing existing instances in the code base.*
-
-* If you find yourself creating a copy of an argument passed by const reference, consider passing it by value instead (if you don't want to use a leading underscore and copy in the body of the function):
-
-~~~{.cpp}
-// You can pass-by-value in ProtobufProcess::install() handlers.
-void Slave::statusUpdate(StatusUpdate update, const UPID& pid)
-{
- ...
- update.mutable_status()->set_source(
- pid == UPID() ? TaskStatus::SOURCE_SLAVE : TaskStatus::SOURCE_EXECUTOR);
- ...
-}
-~~~
-
-### Constant Names
-* We use [SCREAMING_SNAKE_CASE](http://en.wikipedia.org/wiki/Letter_case#Special_case_styles) for constant names (Google uses a `k` followed by mixed case, e.g. `kDaysInAWeek`).
-
-### Function Names
-* We use [lowerCamelCase](http://en.wikipedia.org/wiki/CamelCase#Variations_and_synonyms) for function names (Google uses mixed case for regular functions; and their accessors and mutators match the name of the variable).
-
-## Strings
-* Strings used in log and error messages should end without a period.
-
-## Comments
-* End each sentence within a comment with a punctuation mark (please note that we generally prefer periods); this applies to incomplete sentences as well.
-* For trailing comments, leave one space.
-
-## Breaks
-* Break before braces on enum, function, and record (i.e. struct, class, union) definitions.
-
-## Indentation
-
-### Class Format
-* Access modifiers are not indented (Google uses one space indentation).
-* Constructor initializers are indented by 2 spaces (Google indents by 4).
-
-### Templates
-* Leave one space after the `template` keyword, e.g. `template <typename T>` rather than `template<typename T>`.
-
-### Function Definition/Invocation
-* Newline when calling or defining a function: indent with 4 spaces.
-* We do not follow Google's style of wrapping on the open parenthesis, the general goal is to reduce visual "jaggedness" in the code. Prefer (1), (4), (5), sometimes (3), never (2):
-
-~~~{.cpp}
-// 1: OK.
-allocator->resourcesRecovered(frameworkId, slaveId, resources, filters);
-
-// 2: Don't use.
-allocator->resourcesRecovered(frameworkId, slaveId,
- resources, filters);
-
-// 3: Don't use in this case due to "jaggedness".
-allocator->resourcesRecovered(frameworkId,
- slaveId,
- resources,
- filters);
-
-// 3: In this case, 3 is OK.
-foobar(someArgument,
- someOtherArgument,
- theLastArgument);
-
-// 4: OK.
-allocator->resourcesRecovered(
- frameworkId,
- slaveId,
- resources,
- filters);
-
-// 5: OK.
-allocator->resourcesRecovered(
- frameworkId, slaveId, resources, filters);
-~~~
-
-### Continuation
-* Newline for an assignment statement: indent with 2 spaces.
-
-~~~{.cpp}
-Try<Duration> failoverTimeout =
- Duration::create(FrameworkInfo().failover_timeout());
-~~~
-
-## Empty Lines
-* 1 blank line at the end of the file.
-* Elements outside classes (classes, structs, global functions, etc.) should be spaced apart by 2 blank lines.
-* Elements inside classes (member variables and functions) should not be spaced apart by more than 1 blank line.
-
-## Capture by Reference
-
-We disallow capturing **temporaries** by reference. See [MESOS-2629](https://issues.apache.org/jira/browse/MESOS-2629) for the rationale.
-
-~~~{.cpp}
-Future<Nothing> f() { return Nothing(); }
-Future<bool> g() { return false; }
-
-struct T
-{
- T(const char* data) : data(data) {}
- const T& member() const { return *this; }
- const char* data;
-};
-
-// 1: Don't use.
-const Future<Nothing>& future = f();
-
-// 1: Instead use.
-const Future<Nothing> future = f();
-
-// 2: Don't use.
-const Future<Nothing>& future = Future<Nothing>(Nothing());
-
-// 2: Instead use.
-const Future<Nothing> future = Future<Nothing>(Nothing());
-
-// 3: Don't use.
-const Future<bool>& future = f().then(lambda::bind(g));
-
-// 3: Instead use.
-const Future<bool> future = f().then(lambda::bind(g));
-
-// 4: Don't use (since the T that got constructed is a temporary!).
-const T& t = T("Hello").member();
-
-// 4: Preferred alias pattern (see below).
-const T t("Hello");
-const T& t_ = t.member();
-
-// 4: Can also use.
-const T t = T("Hello").member();
-~~~
-
-We allow capturing non-temporaries by *constant reference* when the intent is to **alias**.
-
-The goal is to make code more concise and improve readability. Use this if an expression referencing a field by `const` is:
-
-* Used repeatedly.
-* Would benefit from a concise name to provide context for readability.
-* Will **not** be invalidated during the lifetime of the alias. Otherwise document this explicitly.
-
-~~~{.cpp}
-hashmap<string, hashset<int>> index;
-
-// 1: Ok.
-const hashset<int>& values = index[2];
-
-// 2: Ok.
-for (auto iterator = index.begin(); iterator != index.end(); ++iterator) {
- const hashset<int>& values = iterator->second;
-}
-
-// 3: Ok.
-foreachpair (const string& key, const hashset<int>& values, index) {}
-foreachvalue (const hashset<int>& values, index) {}
-foreachkey (const string& key, index) {}
-
-// 4: Avoid aliases in most circumstances as they can be dangerous.
-// This is an example of a dangling alias!
-vector<string> strings{"hello"};
-
-string& s = strings[0];
-
-strings.erase(strings.begin());
-
-s += "world"; // THIS IS A DANGLING REFERENCE!
-~~~
-
-## File Headers
-
-* Mesos source files must contain the "ASF" header:
-
- /**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-* Stout and libprocess source files must contain the "Apache License Version 2.0" header:
-
- /**
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License
- */
-
-## C++11
-
-We support C++11 and require GCC 4.8+ or Clang 3.5+ compilers. The whitelist of supported C++11 features is:
-
-* Static assertions.
-* Multiple right angle brackets.
-* Type inference (`auto` and `decltype`). The main goal is to increase code readability. This is safely the case if the exact same type omitted on the left is already fully stated on the right. Here are several examples:
-
-~~~{.cpp}
-// 1: OK.
-const auto i = values.find(keys.front());
-// Compare with
-const typename map::iterator i = values.find(keys.front());
-
-// 2: OK.
-auto names = shared_ptr<list<string>>(new list<string>());
-// Compare with
-shared_ptr<list<string>> names = shared_ptr<list<string>>(new list<string>());
-
-// 3: Don't use.
-auto authorizer = LocalAuthorizer::create(acls);
-// Compare with
-Try<Owned<LocalAuthorizer>> authorizer = LocalAuthorizer::create();
-~~~
-
-* Rvalue references.
-* Explicitly-defaulted functions.
-* Variadic templates.
-* Delegating constructors.
-* Mutexes.
- * `std::mutex`
- * `std::lock_guard<std::mutex>`
- * `std::unique_lock<std::mutex>`
-* Atomics (`std::atomic`)
- * The standard defines a number of predefined typedefs for atomic types (e.g., `std::atomic_int`), in addition to `std::atomic<T>`. When a typedef is available, it should be preferred over explicit template specialization of `std::atomic<T>`.
- * When reading from and writing to atomic values, the `load` and `store` member functions should be used instead of the overloads of `operator T()` and `operator=`. Being explicit helps to draw the reader's attention to the fact that atomic values are being manipulated.
-* Shared from this.
- * `class T : public std::enable_shared_from_this<T>`
- * `shared_from_this()`
-* Lambdas!
- * Don't put a space between the capture list and the parameter list:
-
-~~~{.cpp}
-// 1: OK.
-[]() { ...; };
-
-// 2: Don't use.
-[] () { ...; };
-~~~
-
- * Prefer default capture by value, explicit capture by value, then capture by reference. To avoid dangling-pointer bugs, *never* use default capture by reference:
-
-~~~{.cpp}
-// 1: OK.
-[=]() { ... }; // Default capture by value.
-[n]() { ... }; // Explicit capture by value.
-[&n]() { ... }; // Explicit capture by reference.
-[=, &n]() { ... }; // Default capture by value, explicit capture by reference.
-
-// 2: Don't use.
-[&]() { ... }; // Default capture by reference.
-~~~
-
- * Use `mutable` only when absolutely necessary.
-
-~~~{.cpp}
-// 1: OK.
-[]() mutable { ...; };
-~~~
-
- * Feel free to ignore the return type by default, adding it as necessary to appease the compiler or be more explicit for the reader.
-
-~~~{.cpp}
-// 1: OK.
-[]() { return true; };
-[]() -> bool { return ambiguous(); };
-~~~
-
- * Feel free to use `auto` when naming a lambda expression:
-
-~~~{.cpp}
-// 1: OK.
-auto lambda = []() { ...; };
-~~~
-
- * Format lambdas similar to how we format functions and methods. Feel free to let lambdas be one-liners:
-
-~~~{.cpp}
-// 1: OK.
-auto lambda = []() {
- ...;
-};
-
-// 2: OK.
-auto lambda = []() { ...; };
-~~~
-
- * Feel free to inline lambdas within function arguments:
-
-~~~{.cpp}
-instance.method([]() {
- ...;
-});
-~~~
-
- * Chain function calls on a newline after the closing brace of the lambda and the closing parenthesis of function call:
-
-~~~{.cpp}
-// 1: OK.
-instance
- .method([]() {
- ...;
- })
- .then([]() { ...; })
- .then([]() {
- ...;
- });
-
-// 2: OK (when no chaining, compare to 1).
-instance.method([]() {
- ...;
-});
-
-// 3: OK (if no 'instance.method').
-function([]() {
- ...;
-})
-.then([]() { ...; })
-.then([]() {
- ...;
-});
-
-// 3: OK (but prefer 1).
-instance.method([]() {
- ...;
-})
-.then([]() { ...; })
-.then([]() {
- ...;
-});
-~~~
-
- * Wrap capture lists independently of parameters, *use the same formatting as if the capture list were template parameters*:
-
-~~~{.cpp}
-// 1: OK.
-function([&capture1, &capture2, &capture3](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-});
-
-function(
- [&capture1, &capture2, &capture3](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-});
-
-auto lambda = [&capture1, &capture2, &capture3](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-};
-
-auto lambda =
- [&capture1, &capture2, &capture3](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-};
-
-// 2: OK (when capture list is longer than 80 characters).
-function([
- &capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1, const T2& p2) {
- ...;
-});
-
-auto lambda = [
- &capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1, const T2& p2) {
- ...;
-};
-
-// 3: OK (but prefer 2).
-function([
- &capture1,
- &capture2,
- &capture3,
- &capture4](const T1& p1, const T2& t2) {
- ...;
-});
-
-auto lambda = [
- &capture1,
- &capture2,
- &capture3,
- &capture4](const T1& p1, const T2& p2) {
- ...;
-};
-
-// 3: Don't use.
-function([&capture1,
- &capture2,
- &capture3,
- &capture4](const T1& p1, const T2& p2) {
- ...;
-});
-
-auto lambda = [&capture1,
- &capture2,
- &capture3,
- &capture4](const T1& p1, const T2& p2) {
- ...;
- };
-
-// 4: Don't use.
-function([&capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-});
-
-auto lambda = [&capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1, const T2& p2, const T3& p3) {
- ...;
-};
-
-// 5: Don't use.
-function([&capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1,
- const T2& p2,
- const T3& p3) {
- ...;
- });
-
-auto lambda = [&capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1,
- const T2& p2,
- const T3& p3) {
- ...;
-};
-
-// 6: OK (parameter list longer than 80 characters).
-function([&capture1, &capture2, &capture3](
- const T1& p1,
- const T2& p2,
- const T3& p3,
- const T4& p4) {
- ...;
-});
-
-auto lambda = [&capture1, &capture2, &capture3](
- const T1& p1,
- const T2& p2,
- const T3& p3,
- const T4& p4) {
- ...;
-};
-
-// 7: OK (capture and parameter lists longer than 80 characters).
-function([
- &capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1,
- const T2& p2,
- const T3& p3,
- const T4& p4) {
- ...;
-});
-
-auto lambda = [
- &capture1,
- &capture2,
- &capture3,
- &capture4](
- const T1& p1,
- const T2& p2,
- const T3& p3,
- const T4& p4) {
- ...;
-};
-~~~
-
-* Unrestricted Union.
-
- Like the pre-existing `union`, we can overlap storage allocation for objects that never exist simultaneously. However, with C++11 we are no longer *restricted to having only non-POD types in unions*. Adding non-POD types to unions complicates things, however, because we need to make sure to properly call constructors and destructors. Therefore, only use unrestricted unions (i.e., unions with non-POD types) when the union has only a single field. What does this buy us? Now we can avoid dynamic memory allocations for "container" like types, e.g., `Option`, `Try`, `Result`, etc. In effect, we treat the union like a dynamic allocation, calling *placement new*, `new (&t) T(...)` anyplace we would have just called `new T(...)` and the destructor `t.~T()` anyplace we would have called `delete t`.
-
-* Constant expressions.
-
- Constant expressions allow the declaration of static non-POD objects while eliminating the unpredictable runtime initialization and destruction issues normally encountered, helping eliminate macros and hard-coded literals without sacrificing performance and type safety. Changes which require converting from `constexpr` to `const` can propagate through the dependency tree requiring that dependent `constexpr` uses also be converted to `const`, hence we avoid using `constexpr` in complex functions.
-
- `constexpr` behaves as a combination of `inline` and `const` and hence must be defined before use in another `constexpr`.
-
- Prefer `constexpr` to `const` for all constant POD declarations, `constexpr` `char` arrays are preferred to `const` `string` literals.
-
-~~~{.cpp}
- // OK
- constexpr char LITERAL[] = "value";
-
- // Not OK - not available at compile time for optimization and
- // definition required in a separate compilation module.
- const char LITERAL[];
-
- // Not OK - uncertain initialization order, cannot be used in other
- // constexpr statements.
- const string LITERAL("value");
-~~~
-
- `constexpr` functions are evaluated at compile time if all their arguments are constant expressions. Otherwise they default to initialization at runtime. However `constexpr` functions are limited in that they cannot perform dynamic casts, memory allocation or calls to non-constexpr functions. Prefer `constexpr` over const inline functions.
-
-~~~{.cpp}
- constexpr size_t MIN = 200;
- constexpr size_t MAX = 1000;
- constexpr size_t SPAN() { return MAX-MIN; }
- int array[SPAN()];
-~~~
-
-Const expression constructors allow object initialization at compile time provided that all the constructor arguments are `constexpr` and the constructor body is empty, i.e. all initialization is performed in the initialization list. Classes which provide `constexpr` constructors should normally also provide `constexpr` copy constructors to allow the class to be used in the return value from a `constexpr` function.
-
-~~~{.cpp}
- class C
- {
- public:
- constexpr C(int _i) : i(_i) {};
- constexpr C(const C& c) : i(c.i) {}
- private:
- const int i;
- };
-~~~
-
- C++11 does not provide `constexpr string` or `constexpr` containers in the STL and hence `constexpr` cannot be used for any class using stout's Error() class.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-containerizer.md
----------------------------------------------------------------------
diff --git a/docs/mesos-containerizer.md b/docs/mesos-containerizer.md
deleted file mode 100644
index ea19558..0000000
--- a/docs/mesos-containerizer.md
+++ /dev/null
@@ -1,82 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Containerizer
-
-The MesosContainerizer provides lightweight containerization and
-resource isolation of executors using Linux-specific functionality
-such as control cgroups and namespaces. It is composable so operators
-can selectively enable different isolators.
-
-It also provides basic support for POSIX systems (e.g., OSX) but
-without any actual isolation, only resource usage reporting.
-
-### Shared Filesystem
-
-The SharedFilesystem isolator can optionally be used on Linux hosts to
-enable modifications to each container's view of the shared
-filesystem.
-
-The modifications are specified in the ContainerInfo included in the
-ExecutorInfo, either by a framework or by using the
---default\_container\_info slave flag.
-
-ContainerInfo specifies Volumes which map parts of the shared
-filesystem (host\_path) into the container's view of the filesystem
-(container\_path), as read-write or read-only. The host\_path can be
-absolute, in which case it will make the filesystem subtree rooted at
-host\_path also accessible under container\_path for each container.
-If host\_path is relative then it is considered as a directory
-relative to the executor's work directory. The directory will be
-created and permissions copied from the corresponding directory (which
-must exist) in the shared filesystem.
-
-The primary use-case for this isolator is to selectively make parts of
-the shared filesystem private to each container. For example, a
-private "/tmp" directory can be achieved with host\_path="tmp" and
-container\_path="/tmp" which will create a directory "tmp" inside the
-executor's work directory (mode 1777) and simultaneously mount it as
-/tmp inside the container. This is transparent to processes running
-inside the container. Containers will not be able to see the host's
-/tmp or any other container's /tmp.
-
-### Pid Namespace
-
-The Pid Namespace isolator can be used to isolate each container in
-a separate pid namespace with two main benefits:
-1. Visibility: Processes running in the container (executor and
- descendants) are unable to see or signal processes outside the
- namespace.
-2. Clean termination: Termination of the leading process in a pid
- namespace will result in the kernel terminating all other processes
- in the namespace.
-
-The Launcher will use (2) during destruction of a container in
-preference to the freezer cgroup, avoiding known kernel issues related
-to freezing cgroups under OOM conditions.
-
-/proc will be mounted for containers so tools such as 'ps' will work
-correctly.
-
-
-### Posix Disk Isolator
-
-The Posix Disk isolator provides basic disk isolation. It is able to
-report the disk usage for each sandbox and optionally enforce the disk
-quota. It can be used on both Linux and OS X.
-
-To enable the Posix Disk isolator, append `posix/disk` to the
-`--isolation` flag when starting the slave.
-
-By default, the disk quota enforcement is disabled. To enable it,
-specify `--enforce_container_disk_quota` when starting the slave.
-
-The Posix Disk isolator reports disk usage for each sandbox by
-periodically running the `du` command. The disk usage can be retrieved
-from the resource statistics endpoint (`/monitor/statistics.json`).
-
-The interval between two `du`s can be controlled by the slave flag
-`--container_disk_watch_interval`. For example,
-`--container_disk_watch_interval=1mins` sets the interval to be 1
-minute. The default interval is 15 seconds.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-documentation-guide.md
----------------------------------------------------------------------
diff --git a/docs/mesos-documentation-guide.md b/docs/mesos-documentation-guide.md
index 42ed95f..2a4151b 100644
--- a/docs/mesos-documentation-guide.md
+++ b/docs/mesos-documentation-guide.md
@@ -9,6 +9,8 @@ We have three different kinds of documentation:
User guides and non-code technical documentation are stored in markdown files in the `docs/` folder. These files get rendered for the [online documentation](http://mesos.apache.org/documentation/latest/).
+ We will accept small documentation changes on [Github via a pull request](https://github.com/apache/mesos), larger documentation changes should go through the (reviewboard)[https://reviews.apache.org/groups/mesos/]
+
2. [Doxygen API Documentation and Developer Guides as part of source code](/documentation/latest/mesos-doxygen-style-guide/)
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-doxygen-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/mesos-doxygen-style-guide.md b/docs/mesos-doxygen-style-guide.md
deleted file mode 100644
index 1bd3b3d..0000000
--- a/docs/mesos-doxygen-style-guide.md
+++ /dev/null
@@ -1,196 +0,0 @@
-# Apache Mesos Doxygen Style Guide
-
-This guide introduces a consistent style
-for [documenting Mesos source code](http://mesos.apache.org/api/latest/c++)
-using [Doxygen](http://www.doxygen.org).
-There is an ongoing, incremental effort with the goal to document all public Mesos, libprocess, and stout APIs this way.
-For now, existing code may not follow these guidelines, but new code should.
-
-
-## Source Code Documentation Syntax
-
-Doxygen documentation needs only to be applied to source code parts that
-constitute an interface for which we want to generate Mesos API documentation
-files. Implementation code that does not participate in this should still be
-enhanced by source code comments as appropriate, but these comments should not follow the doxygen style.
-
-We follow the [Javadoc syntax](http://en.wikipedia.org/wiki/Javadoc) to mark comment blocks.
-These have the general form:
-
-
-<!-- NOTE: In order to show the comments as part of the code blocks in
-this we must use indentation instead of fenced code blocks (i.e.,
-~~~{.cpp}) because doxygen will remove all of the comments and just
-render the code. -->
-
-
- /**
- * Brief summary.
- *
- * Detailed description. More detail.
- * @see Some reference
- *
- * @param <name> Parameter description.
- * @return Return value description.
- */
-
-Example:
-
- /**
- * Returns a compressed version of a string.
- *
- * Compresses an input string using the foobar algorithm.
- *
- * @param uncompressed The input string.
- * @return A compressed version of the input string.
- */
- std::string compress(const std::string& uncompressed);
-
-
-### Doxygen Tags
-
-This is the allowed set of doxygen tags that can be used.
-
- * [\@param](http://doxygen.org/manual/commands.html#cmdparam) Describes function parameters.
- * [\@return](http://doxygen.org/manual/commands.html#cmdreturn) Describes return values.
- * [\@see](http://doxygen.org/manual/commands.html#cmdsa) Describes a cross-reference to classes, functions, methods, variables, files or URL.
-
-Example:
-
- /**
- * Available kinds of implementations.
- *
- * @see process::network::PollSocketImpl
- */
-
- * [\@file](http://doxygen.org/manual/commands.html#cmdfile) Describes a refence to a file. It is required when documenting global functions, variables, typedefs, or enums in separate files.
- * [\@link](http://doxygen.org/manual/commands.html#cmdlink) and [\@endlink](http://doxygen.org/manual/commands.html#cmdendlink) Describes a link to a file, class, or member.
- * [\@example](http://doxygen.org/manual/commands.html#cmdexample) Describes source code examples.
- * [\@image](http://doxygen.org/manual/commands.html#cmdimage) Describes an image.
-
- * When following these links be aware that the doxygen documentation is using another syntax in that \@param is explained as \\param.
-
-
-### Wrapping
-
-We wrap long descriptions using four spaces on the next line.
-
- @param uncompressed The input string that requires
- a very long description and an even longer
- description on this line as well.
-
-
-### Constants and Variables
-
-Example:
-
- /**
- * Prefix used to name Docker containers in order to distinguish
- * those created by Mesos from those created manually.
- */
- extern const std::string DOCKER_NAME_PREFIX;
-
-
-#### Fields
-
-Example:
-
- /**
- * The parent side of the pipe for stdin.
- * If the mode is not PIPE, None will be stored.
- * **NOTE**: stdin is a macro on some systems, hence this name instead.
- */
- Option<int> in;
-
-
-### Functions and Methods
-
-Example:
-
- /**
- * Forks a subprocess and execs the specified 'path' with the
- * specified 'argv', redirecting stdin, stdout, and stderr as
- * specified by 'in', 'out', and 'err' respectively.
- *
- * If 'setup' is not None, runs the specified function after forking
- * but before exec'ing. If the return value of 'setup' is non-zero
- * then that gets returned in 'status()' and we will not exec.
- *
- * @param path Relative or absolute path in the filesytem to the
- * executable.
- * @param argv Argument vector to pass to exec.
- * @param in Redirection specification for stdin.
- * @param out Redirection specification for stdout.
- * @param err Redirection specification for stderr.
- * @param flags Flags to be stringified and appended to 'argv'.
- * @param environment Environment variables to use for the new
- * subprocess or if None (the default) then the new subprocess
- * will inherit the environment of the current process.
- * @param setup Function to be invoked after forking but before
- * exec'ing. NOTE: Take extra care not to invoke any
- * async unsafe code in the body of this function.
- * @param clone Function to be invoked in order to fork/clone the
- * subprocess.
- * @return The subprocess or an error if one occurred.
- */
- Try<Subprocess> subprocess(
- const std::string& path,
- std::vector<std::string> argv,
- const Subprocess::IO& in = Subprocess::FD(STDIN_FILENO),
- const Subprocess::IO& out = Subprocess::FD(STDOUT_FILENO),
- const Subprocess::IO& err = Subprocess::FD(STDERR_FILENO),
- const Option<flags::FlagsBase>& flags = None(),
- const Option<std::map<std::string, std::string>>& environment = None(),
- const Option<lambda::function<int()>>& setup = None(),
- const Option<lambda::function<
- pid_t(const lambda::function<int()>&)>>& clone = None());
-
-
-### Classes and Structs
-
-Example:
-
- /**
- * Represents a fork() exec()ed subprocess. Access is provided to the
- * input / output of the process, as well as the exit status. The
- * input / output file descriptors are only closed after:
- * 1. The subprocess has terminated.
- * 2. There are no longer any references to the associated
- * Subprocess object.
- */
- class Subprocess
- {
- public:
-
-
-## Library and Component Overview Pages and Developer Guides
-
-Substantial libraries, components, and subcomponents of the Mesos system such as
-stout, libprocess, master, slave, containerizer, allocator, and others
-should have an overview page in markdown format that explains their
-purpose, overall structure, and general use. This can even be a complete developer guide.
-
-This page must be located in the top directory of the library/component and named "README.md".
-
-The first line in such a document must be a section heading bearing the title which will appear in the generated Doxygen index.
-Example: "# Libprocess Developer Guide"
-
-### Example Code
-
-C++ code examples should be enclosed by '~~~{.cpp}'.
-
-Example:
-
-~~~{.cpp}
-int main(int argc, char** argv)
-{
- ...
-}
-~~~
-
-**NOTE**: Because of shortcomings of Doxygen's markdown parser we currently use indentation for wrapping all non C++ code blocks.
-
-
-## Building Doxygen Documentation
-
-As of right now, the Doxygen documentation should be built from the *build* subdirectory using *doxygen ../Doxyfile* . The documentation will then be generated into the *./doxygen* subdirectory.
[2/3] mesos git commit: Documentation file naming cleanup
Posted by dl...@apache.org.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-frameworks.md
----------------------------------------------------------------------
diff --git a/docs/mesos-frameworks.md b/docs/mesos-frameworks.md
deleted file mode 100644
index 60405b1..0000000
--- a/docs/mesos-frameworks.md
+++ /dev/null
@@ -1,34 +0,0 @@
----
-layout: documentation
----
-# Software projects built on Mesos
-
-## Long Running Services
-
-* [Aurora](http://aurora.incubator.apache.org) is a service scheduler that runs on top of Mesos, enabling you to run long-running services that take advantage of Mesos' scalability, fault-tolerance, and resource isolation.
-* [Marathon](https://github.com/mesosphere/marathon) is a private PaaS built on Mesos. It automatically handles hardware or software failures and ensures that an app is "always on".
-* [Singularity](https://github.com/HubSpot/Singularity) is a scheduler (HTTP API and web interface) for running Mesos tasks: long running processes, one-off tasks, and scheduled jobs.
-* [SSSP](https://github.com/mesosphere/sssp) is a simple web application that provides a white-label "Megaupload" for storing and sharing files in S3.
-
-## Big Data Processing
-
-* [Cray Chapel](https://github.com/nqn/mesos-chapel) is a productive parallel programming language. The Chapel Mesos scheduler lets you run Chapel programs on Mesos.
-* [Dpark](https://github.com/douban/dpark) is a Python clone of Spark, a MapReduce-like framework written in Python, running on Mesos.
-* [Exelixi](https://github.com/mesosphere/exelixi) is a distributed framework for running genetic algorithms at scale.
-* [Hadoop](https://github.com/mesos/hadoop) Running Hadoop on Mesos distributes MapReduce jobs efficiently across an entire cluster.
-* [Hama](http://wiki.apache.org/hama/GettingStartedMesos) is a distributed computing framework based on Bulk Synchronous Parallel computing techniques for massive scientific computations e.g., matrix, graph and network algorithms.
-* [MPI](https://github.com/mesosphere/mesos-hydra) is a message-passing system designed to function on a wide variety of parallel computers.
-* [Spark](http://spark.incubator.apache.org/) is a fast and general-purpose cluster computing system which makes parallel jobs easy to write.
-* [Storm](https://github.com/mesosphere/storm-mesos) is a distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing.
-
-## Batch Scheduling
-
-* [Chronos](https://github.com/airbnb/chronos) is a distributed job scheduler that supports complex job topologies. It can be used as a more fault-tolerant replacement for Cron.
-* [Jenkins](https://github.com/jenkinsci/mesos-plugin) is a continuous integration server. The mesos-jenkins plugin allows it to dynamically launch workers on a Mesos cluster depending on the workload.
-* [JobServer](http://www.grandlogic.com/content/html_docs/jobserver.html) is a distributed job scheduler and processor which allows developers to build custom batch processing Tasklets using point and click web UI.
-
-## Data Storage
-
-* [Cassandra](https://github.com/mesosphere/cassandra-mesos) is a performant and highly available distributed database. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data.
-* [ElasticSearch](https://github.com/mesosphere/elasticsearch-mesos) is a distributed search engine. Mesos makes it easy to run and scale.
-* [Hypertable](https://code.google.com/p/hypertable/wiki/Mesos) is a high performance, scalable, distributed storage and processing system for structured and unstructured data.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-markdown-style-guide.md
----------------------------------------------------------------------
diff --git a/docs/mesos-markdown-style-guide.md b/docs/mesos-markdown-style-guide.md
deleted file mode 100644
index fe9ffef..0000000
--- a/docs/mesos-markdown-style-guide.md
+++ /dev/null
@@ -1,118 +0,0 @@
-# Mesos Markdown Style Guide
-
-This guide introduces a consistent documentation style to be used across the entire non-code documentation.
-User guides and non-code technical documentation are stored in markdown files in the `docs/` folder. These files get rendered for the [online documentation](http://mesos.apache.org/documentation/latest/).
-
-**NOTE:** As of right now this is work in progress and the existing documentation might not yet comply to this style.
-
-
-## What to document?
-
-Any new substantial feature should be documented in its own markdown file.
-If the link between source code and documentation is not obvious, consider inserting a short code comment stating that there is non-code documentation that needs to be kept in sync and indicating where it is located.
-
-
-## Keep documentation and style-guides in sync with code.
-
-When changing code consider whether you need to update the documentation.
-This is especially relevant when introducing new or updating existing command line flags.
-These should be reflected in `configuration.md`!
-
-
-## Code Examples
-
-Code examples should be specified as follows:
-
- ~~~{.cpp}
- int main(int argc, char** argv)
- {
- ....
- }
- ~~~
-
-**NOTE:** Because of shortcomings of Doxygen's markdown parser we currently use indentation for wrapping all non C++ code blocks.
-
-## Notes/Emphasis
-
-Notes are used to highlight important parts of the text and should be specified as follows.
-
-~~~{.txt}
-**NOTE:** Short note.
-Continued longer note.
-~~~
-
-We use single backticks to highlight individual words in a sentence such as certain identifiers:
-
-~~~{.txt}
-Use the default `HierarchicalDRF` allocator....
-~~~
-
-
-## Commands
-
-We use single backticks to highlight sample commands as follows:
-
-~~~{.txt}
-`mesos-master --help`
-~~~
-
-
-## Files/Path
-
-Files and path references should be specified as follows:
-
-~~~{.text}
-Remember you can also use the `file:///path/to/file` or `/path/to/file`
-~~~
-
-
-## Tables
-
-In order to avoid problems with markdown formatting we should specify tables in html directly:
-
-~~~{.html}
-<table class="table table-striped">
- <thead>
- <tr>
- <th width="30%">
- Flag
- </th>
- <th>
- Explanation
- </th>
- </thead>
- <tr>
- <td>
- --ip=VALUE
- </td>
- <td>
- IP address to listen on
- </td>
- </tr>
- <tr>
- <td>
- --[no-]help
- </td>
- <td>
- Prints this help message (default: false)
-
- </td>
- </tr>
-</table>
-~~~
-
-
-## Indendation and Whitespace
-
-We use no extra indentation in markdown files.
-We have one new line after section headings and two blank lines
-in between sections.
-
-~~~{.txt}
-... end of previous section.
-
-
-## New Section
-
-Beginning of new section ....
-~~~
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-presentations.md
----------------------------------------------------------------------
diff --git a/docs/mesos-presentations.md b/docs/mesos-presentations.md
deleted file mode 100644
index 2ce4b12..0000000
--- a/docs/mesos-presentations.md
+++ /dev/null
@@ -1,92 +0,0 @@
----
-layout: documentation
----
-
-# Video and Slides of Mesos Presentations
-_(Listed in date descending order)_
-
-## Building and Deploying Applications to Apache Mesos
-([Slides](https://www.slideshare.net/charmalloc/buildingdeployingapplicationsmesos))
-Joe Stein
-Presented February 26, 2015 at [DigitalOcean Community Meetup](http://www.meetup.com/DigitalOcean_Community/events/220580767/)
-
-## MesosCon 2014
- [Video playlist](https://www.youtube.com/playlist?list=PLDVc2EaAVPg9kp8cFzjR1Yxj96I4U5EGN)
-
-## Datacenter Computing with Apache Mesos
-([Slides](http://www.slideshare.net/pacoid/datacenter-computing-with-apache-mesos))
-Paco Nathan
-Presented April 15, 2014 at [Big Data DC Meetup](http://www.meetup.com/bigdatadc/events/172610652/)
-
-## Apache Spark at Viadeo (Running on Mesos)
-([Slides](https://speakerdeck.com/ecepoi/apache-spark-at-viadeo), [Video](http://www.youtube.com/watch?v=shaZslr49vQ&t=16m55s))
-Eugen Cepoi
-Presented April 9, 2014 at Paris Hadoop User Group
-
-## Mesos, HubSpot, and Singularity
-([Video](https://www.youtube.com/watch?v=ROn14csiikw))
-Tom Petr
-Presented April 3rd, 2014 at @TwitterOSS #conf
-
-## Building Distributed Frameworks on Mesos
-([Video](https://www.youtube.com/watch?v=n5GT7OFSh58))
-Benjamin Hindman
-Presented March 25th, 2014 at [Aurora and Mesos Frameworks Meetup](https://www.eventbrite.com/e/aurora-and-mesosframeworksmeetup-tickets-10850994617)
-
-## Introduction to Apache Aurora
-([Video](https://www.youtube.com/watch?v=asd_h6VzaJc))
-Bill Farner
-Presented March 25th, 2014 at [Aurora and Mesos Frameworks Meetup](https://www.eventbrite.com/e/aurora-and-mesosframeworksmeetup-tickets-10850994617)
-
-## Improving Resource Efficiency with Apache Mesos
-([Video](https://www.youtube.com/watch?v=YpmElyi94AA))
-Christina Delimitrou
-Presented April 3rd, 2014 at @TwitterOSS #conf
-
-### Apache Mesos as an SDK for Building Distributed Frameworks ([Slides](http://www.slideshare.net/pacoid/strata-sc-2014-apache-mesos-as-an-sdk-for-building-distributed-frameworks))
-Paco Nathan
-Presented February 13th, 2014 at [Strata](http://strataconf.com/)
-
-### Run your data center like Google's with Apache Mesos ([Video + Demo](https://www.youtube.com/watch?v=2YWVGMuMTrg))
-Abhishek Parolkar
-Presented November 14th, 2013 at [Cloud Expo Asia 2013](http://www.cloudexpoasia.com/)
-
-### Datacenter Management with Mesos ([Video](http://www.youtube.com/watch?v=YB1VW0LKzJ4))
-Benjamin Hindman
-Presented August 29th, 2013 at [AMP Camp](http://ampcamp.berkeley.edu/3/)
-
-### Building a Framework on Mesos: A Case Study with Jenkins ([Video](http://www.youtube.com/watch?v=TPXw_lMTJVk))
-Vinod Kone
-Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
-
-### Hadoop on Mesos ([Video](http://www.youtube.com/watch?v=SFj5EMw8THk))
-Brenden Matthews
-Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
-
-### Introduction to Apache Mesos ([Slides](https://speakerdeck.com/benh/apache-mesos-nyc-meetup))
-Benjamin Hindman
-Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
-
-### Chronos: A distributed, fault-tolerant and highly available job orchestration framework for Mesos ([Slides](https://speakerdeck.com/mesos/chronos-august-2013-nyc-meetup))
-Florian Leibert
-Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
-
-### Airbnb tech talk ([Video](http://www.youtube.com/watch?v=Hal00g8o1iY))
-Benjamin Hindman
-Presented September 6, 2012 at [Airbnb](http://airbnb.com)
-
-### Managing Twitter clusters with Mesos ([Video](http://www.youtube.com/watch?v=37OMbAjnJn0))
-Benjamin Hindman
-Presented August 22, 2012 at [AMP Camp](http://ampcamp.berkeley.edu)
-
-### Mesos: A Platform for Fine-Grained Resource Sharing in Datacenters ([Video](http://www.youtube.com/watch?v=dB8IDu7g9Nc))
-Matei Zaharia
-Presented March 2011 at [UC Berkeley](http://berkeley.edu)
-
-### Mesos: Efficiently Sharing the Datacenter ([Video](http://vimeo.com/17821090))
-Benjamin Hindman
-Presented November 8, 2010 at [LinkedIn](http://linkedin.com)
-
-### Mesos: A Resource Management Platform for Hadoop and Big Data Clusters ([Video](http://www.youtube.com/watch?v=lE3jR6nM3bw))
-Matei Zaharia
-Presented Summer 2010 at [Yahoo](http://yahoo.com)
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-roadmap.md
----------------------------------------------------------------------
diff --git a/docs/mesos-roadmap.md b/docs/mesos-roadmap.md
deleted file mode 100644
index 1f68ba0..0000000
--- a/docs/mesos-roadmap.md
+++ /dev/null
@@ -1,13 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Roadmap
-
-The [Mesos JIRA Road Map](https://issues.apache.org/jira/browse/MESOS/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel) provides a view of which tickets are targeted for upcoming releases.
-
-The ['Epic' issues in JIRA](https://issues.apache.org/jira/issues/?jql=project%20%3D%20MESOS%20AND%20resolution%20%3D%20Unresolved%20AND%20issuetype%20%3D%20Epic%20ORDER%20BY%20priority%20DESC) provide a view into the larger projects planned to be worked on.
-
-There is also an ad-hoc [Release Planning](https://cwiki.apache.org/confluence/display/MESOS/Mesos+Release+Planning) document that attempts to capture times and features for upcoming releases.
-
-If you have comments or suggestions, feel free to reach out to the Mesos dev list (dev@mesos.apache.org).
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-ssl.md
----------------------------------------------------------------------
diff --git a/docs/mesos-ssl.md b/docs/mesos-ssl.md
deleted file mode 100644
index 9df34db..0000000
--- a/docs/mesos-ssl.md
+++ /dev/null
@@ -1,92 +0,0 @@
----
-layout: documentation
----
-
-# Configuration
-There is currently only one implementation of the [libprocess socket interface](https://github.com/apache/mesos/blob/master/3rdparty/libprocess/include/process/socket.hpp) that supports SSL. This implementation uses [libevent](https://github.com/libevent/libevent). Specifically it relies on the `libevent-openssl` library that wraps `openssl`.
-
-After building Mesos 0.23.0 from source, assuming you have installed the required [Dependencies](#Dependencies), you can modify your configure line to enable SSL as follows:
-
-~~~
-../configure --enable-libevent --enable-ssl
-~~~
-
-# Running
-Once you have successfully built and installed your new binaries, here are the environment variables that are applicable to the `Master`, `Slave`, `Framework Scheduler/Executor`, or any `libprocess process`:
-
-#### SSL_ENABLED=(false|0,true|1) [default=false|0]
-Turn on or off SSL. When it is turned off it is the equivalent of default mesos with libevent as the backing for events. All sockets default to the non-SSL implementation. When it is turned on, the default configuration for sockets is SSL. This means outgoing connections will use SSL, and incoming connections will be expected to speak SSL as well. None of the below flags are relevant if SSL is not enabled.
-
-#### SSL_SUPPORT_DOWNGRADE=(false|0,true|1) [default=false|0]
-Control whether or not non-SSL connections can be established. If this is enabled __on the accepting side__, then the accepting side will downgrade to a non-SSL socket if the connecting side is attempting to communicate via non-SSL. (e.g. HTTP). See [Upgrading Your Cluster](#Upgrading) for more details.
-
-#### SSL_CERT_FILE=(path to certificate)
-The location of the certificate that will be presented.
-
-#### SSL_KEY_FILE=(path to key)
-The location of the private key used by OpenSSL.
-
-#### SSL_VERIFY_CERT=(false|0,true|1) [default=false|0]
-Control whether certificates are verified when presented. If this is false, even when a certificate is presented, it will not be verified. When `SSL_REQUIRE_CERT` is true, `SSL_VERIFY_CERT` is overridden and all certificates will be verified _and_ required.
-
-#### SSL_REQUIRE_CERT=(false|0,true|1) [default=false|0]
-Enforce that certificates must be presented by connecting clients. This means all connections (including tools hitting endpoints) must present valid certificates in order to establish a connection.
-
-#### SSL_VERIFY_DEPTH=(N) [default=4]
-The maximum depth used to verify certificates. The default is 4. See the OpenSSL documentation or contact your system administrator to learn why you may want to change this.
-
-#### SSL_CA_DIR=(path to CA directory)
-The directory used to find the certificate authority / authorities. You can specify `SSL_CA_DIR` or `SSL_CA_FILE` depending on how you want to restrict your certificate authorization.
-
-#### SSL_CA_FILE=(path to CA file)
-The file used to find the certificate authority. You can specify `SSL_CA_DIR` or `SSL_CA_FILE` depending on how you want to restrict your certificate authorization.
-
-#### SSL_CIPHERS=(accepted ciphers separated by ':') [default=AES128-SHA:AES256-SHA:RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA]
-A list of `:`-separated ciphers. Use these if you want to restrict or open up the accepted ciphers for OpenSSL. Read the OpenSSL documentation or contact your system administrators to see whether you want to override the default values.
-
-#### SSL_ENABLE_SSL_V3=(false|0,true|1) [default=false|0]
-#### SSL_ENABLE_TLS_V1_0=(false|0,true|1) [default=false|0]
-#### SSL_ENABLE_TLS_V1_1=(false|0,true|1) [default=false|0]
-#### SSL_ENABLE_TLS_V1_2=(false|0,true|1) [default=true|1]
-The above switches enable / disable the specified protocols. By default only TLS V1.2 is enabled. SSL V2 is always disabled; there is no switch to enable it. The mentality here is to restrict security by default, and force users to open it up explicitly. Many older version of the protocols have known vulnerabilities, so only enable these if you fully understand the risks.
-_SSLv2 is disabled completely because modern versions of OpenSSL disable it using multiple compile time configuration options._
-#<a name="Dependencies"></a>Dependencies
-
-### libevent
-We require the OpenSSL support from libevent. The suggested version of libevent is [`2.0.22-stable`](https://github.com/libevent/libevent/releases/tag/release-2.0.22-stable). As new releases come out we will try to maintain compatibility.
-
-~~~
-// For example, on OSX:
-brew install libevent
-~~~
-
-### OpenSSL
-We require [OpenSSL](https://github.com/openssl/openssl). There are multiple branches of OpenSSL that are being maintained by the community. Since security requires being vigilant, we recommend reading the release notes for the current releases of OpenSSL and deciding on a version within your organization based on your security needs. Mesos is not too deeply dependent on specific OpenSSL versions, so there is room for you to make security decisions as an organization.
-Please ensure the `event2` and `openssl` headers are available for building mesos.
-
-~~~
-// For example, on OSX:
-brew install openssl
-~~~
-
-# <a name="Upgrading"></a>Upgrading Your Cluster
-_There is no SSL specific requirement for upgrading different components in a specific order._
-
-The recommended strategy is to restart all your components to enable SSL with downgrades support enabled. Once all components have SSL enabled, then do a second restart of all your components to disable downgrades. This strategy will allow each component to be restarted independently at your own convenience with no time restrictions. It will also allow you to try SSL in a subset of your cluster. __NOTE:__ While different components in your cluster are serving SSL vs non-SSL traffic, any relative links in the WebUI may be broken. Please see the [WebUI](#WebUI) section for details. Here are sample commands for upgrading your cluster:
-
-~~~
-// Restart each component with downgrade support (master, slave, framework):
-SSL_ENABLED=true SSL_SUPPORT_DOWNGRADE=true SSL_KEY_FILE=<path-to-your-private-key> SSL_CERT_FILE=<path-to-your-certificate> <Any other SSL_* environment variables you may choose> <your-component (e.g. bin/master.sh)> <your-flags>
-
-// Restart each component WITHOUT downgrade support (master, slave, framework):
-SSL_ENABLED=true SSL_SUPPORT_DOWNGRADE=false SSL_KEY_FILE=<path-to-your-private-key> SSL_CERT_FILE=<path-to-your-certificate> <Any other SSL_* environment variables you may choose> <your-component (e.g. bin/master.sh)> <your-flags>
-~~~
-The end state is a cluster that is only communicating with SSL.
-
-__NOTE:__ Any tools you may use that communicate with your components must be able to speak SSL, or they will be denied. You may choose to maintain `SSL_SUPPORT_DOWNGRADE=true` for some time as you upgrade your internal tooling. The advantage of `SSL_SUPPORT_DOWNGRADE=true` is that all components that speak SSL will do so, while other components may still communicate over insecure channels.
-
-# <a name="WebUI"></a>WebUI
-The default Mesos WebUI uses relative links. Some of these links transition between endpoints served by the master and slaves. The WebUI currently does not have enough information to change the 'http' vs 'https' links based on whether the target endpoint is currently being served by an SSL-enabled binary. This may cause certain links in the WebUI to be broken when a cluster is in a transition state between SSL and non-SSL. Any tools that hit these endpoints will still be able to access them as long as they hit the endpoint using the right protocol, or the `SSL_SUPPORT_DOWNGRADE` option is set to true.
-
-### Certificates
-Most browsers have built in protection that guard transitions between pages served using different certificates. For this reason you may choose to serve both the master and slave endpoints using a common certificate that covers multiple hostnames. If you do not do this, certain links, such as those to slave sandboxes, may seem broken as the browser treats the transition between differing certificates transition as unsafe.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/mesos-testing-patterns.md
----------------------------------------------------------------------
diff --git a/docs/mesos-testing-patterns.md b/docs/mesos-testing-patterns.md
deleted file mode 100644
index d0d92ff..0000000
--- a/docs/mesos-testing-patterns.md
+++ /dev/null
@@ -1,81 +0,0 @@
----
-layout: documentation
----
-
-# Mesos Testing Patterns
-
-A collection of common testing patterns used in Mesos tests. If you have found a good way to test a certain condition that you think may be useful for other cases, please document it here together with motivation and background.
-
-## Using `Clock` magic to ensure an event is processed
-Scheduling a sequence of events in an asynchronous environment is not easy: a function call usually initiates an action and returns immediately, while the action runs in background. A simple, obvious, and bad solution is to use `os::sleep()` to wait for action completion. The time the action needs to finish may vary on different machines, while increasing sleep duration increases the test execution time and slows down `make check`. One of the right ways to do it is to wait for an action to finish and proceed right after. This is possible using libprocess' `Clock` routines.
-
-
-Every message enqueued in a libprocess process' (or actor's, to avoid ambiguity with OS processes) mailbox is processed by `ProcessManager` (right now there is a single instance of `ProcessManager` per OS process, but this may change in the future). `ProcessManager` fetches actors from the runnable actors list and services all events from the actor's mailbox. Using `Clock::settle()` call we can block the calling thread until `ProcessManager` empties mailboxes of all actors. Here is the example of this pattern:
-
-~~~{.cpp}
-// As Master::killTask isn't doing anything, we shouldn't get a status update.
-EXPECT_CALL(sched, statusUpdate(&driver, _))
- .Times(0);
-
-// Set expectation that Master receives killTask message.
-Future<KillTaskMessage> killTaskMessage =
- FUTURE_PROTOBUF(KillTaskMessage(), _, master.get());
-
-// Attempt to kill unknown task while slave is transitioning.
-TaskID unknownTaskId;
-unknownTaskId.set_value("2");
-
-// Stop the clock.
-Clock::pause();
-
-// Initiate an action.
-driver.killTask(unknownTaskId);
-
-// Make sure the event associated with the action has been queued.
-AWAIT_READY(killTaskMessage);
-
-// Wait for all messages to be dispatched and processed completely to satisfy
-// the expectation that we didn't receive a status update.
-Clock::settle();
-
-Clock::resume();
-~~~
-
-## Intercepting a message sent to a different OS process
-Intercepting messages sent between libprocess processes (let's call them actors to avoid ambiguity with OS processes) that live in the same OS process is easy, e.g.:
-
-~~~{.cpp}
-Future<SlaveReregisteredMessage> slaveReregisteredMessage =
- FUTURE_PROTOBUF(SlaveReregisteredMessage(), _, _);
-...
-AWAIT_READY(slaveReregisteredMessage);
-~~~
-
-However, this won't work if we want to intercept a message sent to an actor (technically a `UPID`) that lives in another OS process. For example, `CommandExecutor` spawned by a slave will live in a separate OS process, though master and slave instances live in the same OS process together with our test (see `mesos/src/tests/cluster.hpp`). The wait in this code will fail:
-
-~~~{.cpp}
-Future<ExecutorRegisteredMessage> executorRegisteredMessage =
- FUTURE_PROTOBUF(ExecutorRegisteredMessage(), _, _);
-...
-AWAIT_READY(executorRegisteredMessage);
-~~~
-
-### Why messages sent outside the OS process are not intercepted?
-Libprocess events may be filtered (see `libprocess/include/process/filter.hpp`). `FUTURE_PROTOBUF` uses this ability and sets an expectation on a `filter` method of `TestsFilter` class with a `MessageMatcher`, that matches the message we want to intercept. The actual filtering happens in `ProcessManager::resume()`, which fetches messages from the queue of the received events.
-
-*No* filtering happens when sending, encoding, or transporting the message (see e.g. `ProcessManager::deliver()` or `SocketManager::send()`). Therefore in the aforementioned example, `ExecutorRegisteredMessage` leaves the slave undetected by the filter, reaches another OS process where executor lives, gets enqueued into the `CommandExecutorProcess`' mailbox and can be filtered there, but remember our expectation is set in another OS process!
-
-### How to workaround
-Consider setting expectations on corresponding incoming messages ensuring they are processed and therefore ACK message is sent.
-
-For the aforementioned example, instead of intercepting `ExecutorRegisteredMessage`, we can intercept `RegisterExecutorMessage` and wait until its processed, which includes sending `ExecutorRegisteredMessage` (see `Slave::registerExecutor()`):
-
-~~~{.cpp}
-Future<RegisterExecutorMessage> registerExecutorMessage =
- FUTURE_PROTOBUF(RegisterExecutorMessage(), _, _);
-...
-AWAIT_READY(registerExecutorMessage);
-Clock::pause();
-Clock::settle();
-Clock::resume();
-~~~
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/powered-by-mesos.md
----------------------------------------------------------------------
diff --git a/docs/powered-by-mesos.md b/docs/powered-by-mesos.md
index ed2cd37..bbcac44 100644
--- a/docs/powered-by-mesos.md
+++ b/docs/powered-by-mesos.md
@@ -95,6 +95,6 @@ layout: documentation
## Software projects built on Mesos
-See the [Mesos frameworks page](mesos-frameworks.md) for a list of apps built on top of Mesos, and instructions on how to use them.
+See the [Mesos frameworks page](frameworks.md) for a list of apps built on top of Mesos, and instructions on how to use them.
If you're using Mesos, please add yourself to the list above, or email user@mesos.apache.org and we'll add you!
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/presentations.md
----------------------------------------------------------------------
diff --git a/docs/presentations.md b/docs/presentations.md
new file mode 100644
index 0000000..2ce4b12
--- /dev/null
+++ b/docs/presentations.md
@@ -0,0 +1,92 @@
+---
+layout: documentation
+---
+
+# Video and Slides of Mesos Presentations
+_(Listed in date descending order)_
+
+## Building and Deploying Applications to Apache Mesos
+([Slides](https://www.slideshare.net/charmalloc/buildingdeployingapplicationsmesos))
+Joe Stein
+Presented February 26, 2015 at [DigitalOcean Community Meetup](http://www.meetup.com/DigitalOcean_Community/events/220580767/)
+
+## MesosCon 2014
+ [Video playlist](https://www.youtube.com/playlist?list=PLDVc2EaAVPg9kp8cFzjR1Yxj96I4U5EGN)
+
+## Datacenter Computing with Apache Mesos
+([Slides](http://www.slideshare.net/pacoid/datacenter-computing-with-apache-mesos))
+Paco Nathan
+Presented April 15, 2014 at [Big Data DC Meetup](http://www.meetup.com/bigdatadc/events/172610652/)
+
+## Apache Spark at Viadeo (Running on Mesos)
+([Slides](https://speakerdeck.com/ecepoi/apache-spark-at-viadeo), [Video](http://www.youtube.com/watch?v=shaZslr49vQ&t=16m55s))
+Eugen Cepoi
+Presented April 9, 2014 at Paris Hadoop User Group
+
+## Mesos, HubSpot, and Singularity
+([Video](https://www.youtube.com/watch?v=ROn14csiikw))
+Tom Petr
+Presented April 3rd, 2014 at @TwitterOSS #conf
+
+## Building Distributed Frameworks on Mesos
+([Video](https://www.youtube.com/watch?v=n5GT7OFSh58))
+Benjamin Hindman
+Presented March 25th, 2014 at [Aurora and Mesos Frameworks Meetup](https://www.eventbrite.com/e/aurora-and-mesosframeworksmeetup-tickets-10850994617)
+
+## Introduction to Apache Aurora
+([Video](https://www.youtube.com/watch?v=asd_h6VzaJc))
+Bill Farner
+Presented March 25th, 2014 at [Aurora and Mesos Frameworks Meetup](https://www.eventbrite.com/e/aurora-and-mesosframeworksmeetup-tickets-10850994617)
+
+## Improving Resource Efficiency with Apache Mesos
+([Video](https://www.youtube.com/watch?v=YpmElyi94AA))
+Christina Delimitrou
+Presented April 3rd, 2014 at @TwitterOSS #conf
+
+### Apache Mesos as an SDK for Building Distributed Frameworks ([Slides](http://www.slideshare.net/pacoid/strata-sc-2014-apache-mesos-as-an-sdk-for-building-distributed-frameworks))
+Paco Nathan
+Presented February 13th, 2014 at [Strata](http://strataconf.com/)
+
+### Run your data center like Google's with Apache Mesos ([Video + Demo](https://www.youtube.com/watch?v=2YWVGMuMTrg))
+Abhishek Parolkar
+Presented November 14th, 2013 at [Cloud Expo Asia 2013](http://www.cloudexpoasia.com/)
+
+### Datacenter Management with Mesos ([Video](http://www.youtube.com/watch?v=YB1VW0LKzJ4))
+Benjamin Hindman
+Presented August 29th, 2013 at [AMP Camp](http://ampcamp.berkeley.edu/3/)
+
+### Building a Framework on Mesos: A Case Study with Jenkins ([Video](http://www.youtube.com/watch?v=TPXw_lMTJVk))
+Vinod Kone
+Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
+
+### Hadoop on Mesos ([Video](http://www.youtube.com/watch?v=SFj5EMw8THk))
+Brenden Matthews
+Presented July 25, 2013 at [SF Mesos Meetup](http://www.meetup.com/Distributed-data-processing-with-Mesos/events/128585772/)
+
+### Introduction to Apache Mesos ([Slides](https://speakerdeck.com/benh/apache-mesos-nyc-meetup))
+Benjamin Hindman
+Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
+
+### Chronos: A distributed, fault-tolerant and highly available job orchestration framework for Mesos ([Slides](https://speakerdeck.com/mesos/chronos-august-2013-nyc-meetup))
+Florian Leibert
+Presented August 20, 2013 at [NYC Mesos Meetup](https://mesos-nyc-aug2013.eventbrite.com/)
+
+### Airbnb tech talk ([Video](http://www.youtube.com/watch?v=Hal00g8o1iY))
+Benjamin Hindman
+Presented September 6, 2012 at [Airbnb](http://airbnb.com)
+
+### Managing Twitter clusters with Mesos ([Video](http://www.youtube.com/watch?v=37OMbAjnJn0))
+Benjamin Hindman
+Presented August 22, 2012 at [AMP Camp](http://ampcamp.berkeley.edu)
+
+### Mesos: A Platform for Fine-Grained Resource Sharing in Datacenters ([Video](http://www.youtube.com/watch?v=dB8IDu7g9Nc))
+Matei Zaharia
+Presented March 2011 at [UC Berkeley](http://berkeley.edu)
+
+### Mesos: Efficiently Sharing the Datacenter ([Video](http://vimeo.com/17821090))
+Benjamin Hindman
+Presented November 8, 2010 at [LinkedIn](http://linkedin.com)
+
+### Mesos: A Resource Management Platform for Hadoop and Big Data Clusters ([Video](http://www.youtube.com/watch?v=lE3jR6nM3bw))
+Matei Zaharia
+Presented Summer 2010 at [Yahoo](http://yahoo.com)
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/roadmap.md
----------------------------------------------------------------------
diff --git a/docs/roadmap.md b/docs/roadmap.md
new file mode 100644
index 0000000..1f68ba0
--- /dev/null
+++ b/docs/roadmap.md
@@ -0,0 +1,13 @@
+---
+layout: documentation
+---
+
+# Mesos Roadmap
+
+The [Mesos JIRA Road Map](https://issues.apache.org/jira/browse/MESOS/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel) provides a view of which tickets are targeted for upcoming releases.
+
+The ['Epic' issues in JIRA](https://issues.apache.org/jira/issues/?jql=project%20%3D%20MESOS%20AND%20resolution%20%3D%20Unresolved%20AND%20issuetype%20%3D%20Epic%20ORDER%20BY%20priority%20DESC) provide a view into the larger projects planned to be worked on.
+
+There is also an ad-hoc [Release Planning](https://cwiki.apache.org/confluence/display/MESOS/Mesos+Release+Planning) document that attempts to capture times and features for upcoming releases.
+
+If you have comments or suggestions, feel free to reach out to the Mesos dev list (dev@mesos.apache.org).
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/scheduler-http-api.md
----------------------------------------------------------------------
diff --git a/docs/scheduler-http-api.md b/docs/scheduler-http-api.md
new file mode 100644
index 0000000..de6cfc9
--- /dev/null
+++ b/docs/scheduler-http-api.md
@@ -0,0 +1,519 @@
+---
+layout: documentation
+---
+
+# Scheduler HTTP API
+
+Mesos 0.24.0 added **experimental** support for v1 Scheduler HTTP API.
+
+
+## Overview
+
+The scheduler interacts with Mesos via “/api/v1/scheduler” endpoint hosted by the Mesos master. The fully qualified URL of the endpoint might look like:
+
+ http://masterhost:5050/api/v1/scheduler
+
+Note that we refer to this endpoint with its suffix "/scheduler" in the rest of this document. This endpoint accepts HTTP POST requests with data encoded as JSON (Content-Type: application/json) or binary Protobuf (Content-Type: application/x-protobuf). The first request that a scheduler sends to “/scheduler” endpoint is called SUBSCRIBE and results in a streaming response (“200 OK” status code with Transfer-Encoding: chunked). **Schedulers are expected to keep the subscription connection open as long as possible (barring errors in network, software, hardware etc.) and incrementally process the response** (NOTE: HTTP client libraries that can only parse the response after the connection is closed cannot be used). For the encoding used, please refer to **Events** section below.
+
+All the subsequent (non subscribe) requests to “/scheduler” endpoint (see details below in **Calls** section) must be sent using a different connection(s) than the one being used for subscription. Master responds to these HTTP POST requests with “202 Accepted” status codes (or, for unsuccessful requests, with 4xx or 5xx status codes; details in later sections). The “202 Accepted” response means that a request has been accepted for processing, not that the processing of the request has been completed. The request might or might not be acted upon by Mesos (e.g., master fails during the processing of the request). Any asynchronous responses from these requests will be streamed on long-lived subscription connection.
+
+
+## Calls
+
+The following calls are currently accepted by the master. The canonical source of this information is [scheduler.proto](https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto) (NOTE: The protobuf definitions are subject to change before the beta API is finalized). Note that when sending JSON encoded Calls, schedulers should encode raw bytes in Base64 and strings in UTF-8.
+
+### SUBSCRIBE
+
+This is the first step in the communication process between the scheduler and the master. This is also to be considered as subscription to the “/scheduler” events stream.
+
+To subscribe with the master, the scheduler sends a HTTP POST request with encoded `SUBSCRIBE` message with the required FrameworkInfo. Note that if "subscribe.framework_info.id" is not set, master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream with RecordIO encoding, with the first event being `SUBSCRIBED` event (see details in **Events** section).
+
+```
+SUBSCRIBE Request (JSON):
+
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Accept: application/json
+Connection: close
+
+{
+ “type” : “SUBSCRIBE”,
+
+ “subscribe” : {
+ “framework_info” : {
+ “user” : “foo”,
+ “name” : “Example HTTP Framework”
+ },
+
+ “force” : true
+ }
+}
+
+SUBSCRIBE Response Event (JSON):
+HTTP/1.1 200 OK
+
+Content-Type: application/json
+Transfer-Encoding: chunked
+
+<event length>
+{
+ “type” : “SUBSCRIBED”,
+ “subscribed” : {
+ “framework_id” : {“value”:“12220-3440-12532-2345”},
+ "heartbeat_interval_seconds" : 15
+ }
+}
+<more events>
+```
+
+Alternatively, if “subscribe.framework_info.id” is set, master considers this a request from an already subscribed scheduler reconnecting after a disconnection (e.g., due to failover or network disconnection) and responds with `SUBSCRIBED` event containing the same FrameworkID. The “subscribe.force” field describes how the master reacts when multiple scheduler instances (with the same framework id) attempt to subscribe with the master at the same time (e.g., due to network partition). See the semantics in **Disconnections** section below.
+
+NOTE: In the old version of the API, (re-)registered callbacks also included MasterInfo, which contained information about the master the driver currently connected to. With the new API, since schedulers explicitly subscribe with the leading master (see details below in **Master Detection** section), it’s not relevant anymore.
+
+If subscription fails for whatever reason (e.g., invalid request), a HTTP 4xx response is returned with the error message as part of the body and the connection is closed.
+
+Scheduler must make additional HTTP requests to the “/scheduler” endpoint only after it has opened a persistent connection to it by sending a `SUBSCRIBE` request and received a `SUBSCRIBED` response. Calls made without subscription will result in a “403 Forbidden“ instead of a “202 Accepted“ response. A scheduler might also receive a “400 Bad Request” response if the HTTP request is malformed (e.g., malformed HTTP headers).
+
+### TEARDOWN
+Sent by the scheduler when it wants to tear itself down. When Mesos receives this request it will shut down all executors (and consequently kill tasks) and remove persistent volumes (if requested). It then removes the framework and closes all open connections from this scheduler to the Master.
+
+```
+TEARDOWN Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “TEARDOWN”,
+}
+
+TEARDOWN Response:
+HTTP/1.1 202 Accepted
+```
+
+### ACCEPT
+Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, its resources are considered allocated to the framework. Also, any of the offer’s resources not used in the `ACCEPT` call (e.g., to launch a task) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.).
+
+```
+ACCEPT Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “ACCEPT”,
+ “accept” : {
+ “offer_ids” : [
+ {“value” : “12220-3440-12532-O12”},
+ {“value” : “12220-3440-12532-O12”}
+ ],
+ “operations” : [ {“type” : “LAUNCH”, “launch” : {...}} ],
+ “filters” : {...}
+ }
+}
+
+ACCEPT Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### DECLINE
+Sent by the scheduler to explicitly decline offer(s) received. Note that this is same as sending an `ACCEPT` call with no operations.
+
+```
+DECLINE Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “DECLINE”,
+ “decline” : {
+ “offer_ids” : [
+ {“value” : “12220-3440-12532-O12”},
+ {“value” : “12220-3440-12532-O13”}
+ ],
+ “filters” : {...}
+ }
+}
+
+DECLINE Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### REVIVE
+Sent by the scheduler to remove any/all filters that it has previously set via `ACCEPT` or `DECLINE` calls.
+
+```
+REVIVE Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “REVIVE”,
+}
+
+REVIVE Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### KILL
+Sent by the scheduler to kill a specific task. If the scheduler has a custom executor, the kill is forwarded to the executor and it is up to the executor to kill the task and send a `TASK_KILLED` (or `TASK_FAILED`) update. Mesos releases the resources for a task once it receives a terminal update for the task. If the task is unknown to the master, a `TASK_LOST` will be generated.
+
+```
+KILL Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “KILL”,
+ “kill” : {
+ “task_id” : {“value” : “12220-3440-12532-my-task”},
+ “agent_id” : {“value” : “12220-3440-12532-S1233”}
+ }
+}
+
+KILL Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### SHUTDOWN
+Sent by the scheduler to shutdown a specific custom executor (NOTE: This is a new call that was not present in the old API). When an executor gets a shutdown event, it is expected to kill all its tasks (and send `TASK_KILLED` updates) and terminate. If an executor doesn’t terminate within a certain timeout (configurable via “--executor_shutdown_grace_period” agent flag), the agent will forcefully destroy the container (executor and its tasks) and transitions its active tasks to `TASK_LOST`.
+
+```
+SHUTDOWN Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “SHUTDOWN”,
+ “shutdown” : {
+ “executor_id” : {“value” : “123450-2340-1232-my-executor”},
+ “agent_id” : {“value” : “12220-3440-12532-S1233”}
+ }
+}
+
+SHUTDOWN Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### ACKNOWLEDGE
+Sent by the scheduler to acknowledge a status update. Note that with the new API, schedulers are responsible for explicitly acknowledging the receipt of status updates that have “status.uuid()” set. These status updates are reliably retried until they are acknowledged by the scheduler. The scheduler must not acknowledge status updates that do not have “status.uuid()” set as they are not retried. "uuid" is raw bytes encoded in Base64.
+
+```
+ACKNOWLEDGE Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “ACKNOWLEDGE”,
+ “acknowledge” : {
+ “agent_id” : {“value” : “12220-3440-12532-S1233”},
+ “task_id” : {“value” : “12220-3440-12532-my-task”},
+ “uuid” : “jhadf73jhakdlfha723adf”
+ }
+}
+
+ACKNOWLEDGE Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### RECONCILE
+Sent by the scheduler to query the status of non-terminal tasks. This causes the master to send back `UPDATE` events for each task in the list. Tasks that are no longer known to Mesos will result in `TASK_LOST` updates. If the list of tasks is empty, master will send `UPDATE` events for all currently known tasks of the framework.
+
+```
+RECONCILE Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “RECONCILE”,
+ “reconcile” : {
+ “tasks” : [
+ { “task_id” : { “value” : “312325” },
+ “agent_id” : { “value” : “123535” }
+ }
+ ]
+ }
+}
+
+RECONCILE Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### MESSAGE
+Sent by the scheduler to send arbitrary binary data to the executor. Note that Mesos neither interprets this data nor makes any guarantees about the delivery of this message to the executor. "data" is raw bytes encoded in Base64.
+
+```
+MESSAGE Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “MESSAGE”,
+ “message” : {
+ “agent_id” : {“value” : “12220-3440-12532-S1233”},
+ “executor_id” : {“value” : “my-framework-executor”},
+ “data” : “adaf838jahd748jnaldf”
+ }
+}
+
+MESSAGE Response:
+HTTP/1.1 202 Accepted
+
+```
+
+### REQUEST
+Sent by the scheduler to request resources from the master/allocator. The built-in hierarchical allocator simply ignores this request but other allocators (modules) can interpret this in a customizable fashion.
+
+```
+Request (JSON):
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+
+{
+ “framework_id” : {“value” : “12220-3440-12532-2345”},
+ “type” : “REQUEST”,
+ “requests” : [
+ {
+ “agent_id” : {“value” : “12220-3440-12532-S1233”},
+ “resources” : {}
+ },
+ ]
+}
+
+REQUEST Response:
+HTTP/1.1 202 Accepted
+
+```
+
+## Events
+
+Scheduler is expected to keep a **persistent** connection open to “/scheduler” endpoint even after getting a SUBSCRIBED HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with *no* “Content-Length” header set. All subsequent events that are relevant to this framework generated by Mesos are streamed on this connection. Master encodes each Event in RecordIO format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf (possibly compressed) encoded event. Note that the value of length will never be ‘0’ and the size of the length will be the size of unsigned integer (i.e., 64 bits). Also, note that the RecordIO encoding should be decoded by the scheduler whereas the underlying HTTP chunked encoding is typically invisible at the application (scheduler) layer. The type of content encoding used for the events will be determined by the accept header of the POST request (e
.g., Accept: application/json).
+
+The following events are currently sent by the master. The canonical source of this information is at [scheduler.proto](include/mesos/v1/scheduler/scheduler.proto). Note that when sending JSON encoded events, master encodes raw bytes in Base64 and strings in UTF-8.
+
+### SUBSCRIBED
+The first event sent by the master when the scheduler sends a `SUBSCRIBE` request on the persistent connection. See `SUBSCRIBE` in Calls section for the format.
+
+
+### OFFERS
+Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on a agent. Until the scheduler 'Accept's or 'Decline's an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g. due to a lost agent or `--offer_timeout`.
+
+```
+OFFERS Event (JSON)
+<event-length>
+{
+ “type” : “OFFERS”,
+ “offers” : [
+ {
+ “offer_id”:{“value”: “12214-23523-O235235”},
+ “framework_id”:{“value”: “12124-235325-32425”},
+ “agent_id”:{“value”: “12325-23523-S23523”},
+ “hostname”:“agent.host”,
+ “resources”:[...],
+ “attributes”:[...],
+ “executor_ids”:[]
+ }
+ ]
+}
+```
+
+### RESCIND
+Sent by the master when a particular offer is no longer valid (e.g., the agent corresponding to the offer has been removed) and hence needs to be rescinded. Any future calls (`ACCEPT` / `DECLINE`) made by the scheduler regarding this offer will be invalid.
+
+```
+RESCIND Event (JSON)
+<event-length>
+{
+ “type” : “RESCIND”,
+ “rescind” : {
+ “offer_id” : { “value” : “12214-23523-O235235”}
+ }
+}
+```
+
+### UPDATE
+Sent by the master whenever there is a status update that is generated by the executor, agent or master. Status updates should be used by executors to reliably communicate the status of the tasks that they manage. It is crucial that a terminal update (e.g., `TASK_FINISHED`, `TASK_KILLED`, `TASK_FAILED`) is sent by the executor as soon as the task terminates, in order for Mesos to release the resources allocated to the task. It is also the responsibility of the scheduler to explicitly acknowledge the receipt of status updates that are reliably retried. See `ACKNOWLEDGE` in the Calls section above for the semantics. Note that `uuid` and `data` are raw bytes encoded in Base64.
+
+
+```
+UPDATE Event (JSON)
+
+<event-length>
+{
+ “type” : “UPDATE”,
+ “update” : {
+ “status” : {
+ “task_id” : { “value” : “12344-my-task”},
+ “state” : “TASK_RUNNING”,
+ “source” : “SOURCE_EXECUTOR”,
+ “uuid” : “adfadfadbhgvjayd23r2uahj”,
+ "bytes" : "uhdjfhuagdj63d7hadkf"
+
+ }
+ }
+}
+```
+
+### MESSAGE
+A custom message generated by the executor that is forwarded to the scheduler by the master. Note that this message is not interpreted by Mesos and is only forwarded (without reliability guarantees) to the scheduler. It is up to the executor to retry if the message is dropped for any reason. Note that `data` is raw bytes encoded as Base64.
+
+```
+MESSAGE Event (JSON)
+
+<event-length>
+{
+ “type” : “MESSAGE”,
+ “message” : {
+ “agent_id” : { “value” : “12214-23523-S235235”},
+ “executor_id” : { “value” : “12214-23523-my-executor”},
+ “data” : “adfadf3t2wa3353dfadf”
+ }
+}
+```
+
+
+### FAILURE
+Sent by the master when a agent is removed from the cluster (e.g., failed health checks) or when an executor is terminated. Note that, this event coincides with receipt of terminal `UPDATE` events for any active tasks belonging to the agent or executor and receipt of `RESCIND` events for any outstanding offers belonging to the agent. Note that there is no guaranteed order between the `FAILURE`, `UPDATE` and `RESCIND` events.
+
+```
+FAILURE Event (JSON)
+
+<event-length>
+{
+ “type” : “FAILURE”,
+ “failure” : {
+ “agent_id” : { “value” : “12214-23523-S235235”},
+ “executor_id” : { “value” : “12214-23523-my-executor”},
+ “status” : 1
+ }
+}
+```
+
+### ERROR
+Sent by the master when an asynchronous error event is generated (e.g., a framework is not authorized to subscribe with the given role). It is recommended that the framework abort when it receives an error and retry subscription as necessary.
+
+```
+ERROR Event (JSON)
+
+<event-length>
+{
+ “type” : “ERROR”,
+ “message” : “Framework is not authorized”
+}
+```
+
+### HEARTBEAT
+This event is periodically sent by the master to inform the scheduler that a connection is alive. This also helps ensure that network intermediates do not close the persistent subscription connection due to lack of data flow. See the next section on how a scheduler can use this event to deal with network partitions.
+
+```
+HEARTBEAT Event (JSON)
+
+<event-length>
+{
+ “type” : “HEARTBEAT”,
+}
+```
+
+## Disconnections
+
+Master considers a scheduler disconnected if the persistent subscription connection (opened via `SUBSCRIBE` request) to “/scheduler” breaks. The connection could break for several reasons, e.g., scheduler restart, scheduler failover, network error. Note that the master doesn’t keep track of non-subscription connection(s) to
+“/scheduler” because it is not expected to be a persistent connection.
+
+If master realizes that the subscription connection is broken, it marks the scheduler as “disconnected” and starts a failover timeout (failover timeout is part of FrameworkInfo). It also drops any pending events in its queue. Additionally, it rejects subsequent non-subscribe HTTP requests to “/scheduler” with “403 Forbidden”, until the scheduler subscribes again with “/scheduler”. If the scheduler *does not* re-subscribe within the failover timeout, the master considers the scheduler gone forever and shuts down all its executors, thus killing all its tasks. Therefore, all production schedulers are recommended to use a high value (e.g., 4 weeks) for the failover timeout.
+
+NOTE: To force shutdown a framework before the framework timeout elapses (e.g., during framework development and testing), either the framework can send `TEARDOWN` call (part of Scheduler API) or an operator can use the “/master/teardown” endpoint (part of Operator API).
+
+If the scheduler realizes that its subscription connection to “/scheduler” is broken, it should attempt to open a new persistent connection to the
+“/scheduler” (on possibly new master based on the result of master detection) and resubscribe. It should not send new non-subscribe HTTP requests to “/scheduler” unless it gets a `SUBSCRIBED` event; such requests will result in “403 Forbidden”.
+
+If the master does not realize that the subscription connection is broken, but the scheduler realizes it, the scheduler might open a new persistent connection to
+“/scheduler” via `SUBSCRIBE`. In this case, the semantics depend on the value of `subscribe.force`. If set to true, master closes the existing subscription connection and allows subscription on the new connection. If set to false, the new connection attempt is disallowed in favor of the existing connection. The invariant here is that, only one persistent subscription connection for a given FrameworkID is allowed on the master. For HA schedulers, it is recommended that a scheduler instance set `subscribe.force` to true only when it just got elected and set it to false for all subsequent reconnection attempts (e.g, due to disconnection or master failover).
+
+### Network partitions
+
+In the case of a network partition, the subscription connection between the scheduler and master might not necessarily break. To be able to detect this scenario, master periodically (e.g., 15s) sends `HEARTBEAT` events (similar in vein to Twitter’s Streaming API). If a scheduler doesn’t receive a bunch (e.g., 5) of these heartbeats within a time window, it should immediately disconnect and try to re-subscribe. It is highly recommended for schedulers to use an exponential backoff strategy (e.g., upto a maximum of 15s) to avoid overwhelming the master while reconnecting. Schedulers can use a similar timeout (e.g., 75s) for receiving responses to any HTTP requests.
+
+## Master detection
+
+Mesos has a high-availability mode that uses multiple Mesos masters; one active master (called the leader or leading master) and several standbys in case it fails. The masters elect the leader, with ZooKeeper coordinating the election. For more details please refer to the [documentation](/documentation/latest/high-availability.md).
+
+Schedulers are expected to make HTTP requests to the leading master. If requests are made to a non-leading master a “HTTP 307 Temporary Redirect” will be received with the “Location” header pointing to the leading master.
+
+Example subscription workflow with redirection when the scheduler hits a non-leading master.
+
+```
+Scheduler → Master
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost1:5050
+Content-Type: application/json
+Accept: application/json
+Connection: keep-alive
+
+{
+ “framework_info” : {
+ “user” : “foo”,
+ “name” : “Example HTTP Framework”
+ },
+ “type” : “SUBSCRIBE”
+}
+
+Master → Scheduler
+HTTP/1.1 307 Temporary Redirect
+Location: masterhost2:5050
+
+
+Scheduler → Master
+POST /api/v1/scheduler HTTP/1.1
+
+Host: masterhost2:5050
+Content-Type: application/json
+Accept: application/json
+Connection: keep-alive
+
+{
+ “framework_info” : {
+ “user” : “foo”,
+ “name” : “Example HTTP Framework”
+ },
+ “type” : “SUBSCRIBE”
+}
+```
+
+If the scheduler knows the list of master’s hostnames for a cluster, it could use this mechanism to find the leading master to subscribe with. Alternatively, the scheduler could use a pure language library that detects the leading master given a ZooKeeper (or etcd) URL. For a `C++` library that does ZooKeeper based master detection please look at `src/scheduler/scheduler.cpp`.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b29ec4f1/docs/scheduler_http_api.md
----------------------------------------------------------------------
diff --git a/docs/scheduler_http_api.md b/docs/scheduler_http_api.md
deleted file mode 100644
index de6cfc9..0000000
--- a/docs/scheduler_http_api.md
+++ /dev/null
@@ -1,519 +0,0 @@
----
-layout: documentation
----
-
-# Scheduler HTTP API
-
-Mesos 0.24.0 added **experimental** support for v1 Scheduler HTTP API.
-
-
-## Overview
-
-The scheduler interacts with Mesos via “/api/v1/scheduler” endpoint hosted by the Mesos master. The fully qualified URL of the endpoint might look like:
-
- http://masterhost:5050/api/v1/scheduler
-
-Note that we refer to this endpoint with its suffix "/scheduler" in the rest of this document. This endpoint accepts HTTP POST requests with data encoded as JSON (Content-Type: application/json) or binary Protobuf (Content-Type: application/x-protobuf). The first request that a scheduler sends to “/scheduler” endpoint is called SUBSCRIBE and results in a streaming response (“200 OK” status code with Transfer-Encoding: chunked). **Schedulers are expected to keep the subscription connection open as long as possible (barring errors in network, software, hardware etc.) and incrementally process the response** (NOTE: HTTP client libraries that can only parse the response after the connection is closed cannot be used). For the encoding used, please refer to **Events** section below.
-
-All the subsequent (non subscribe) requests to “/scheduler” endpoint (see details below in **Calls** section) must be sent using a different connection(s) than the one being used for subscription. Master responds to these HTTP POST requests with “202 Accepted” status codes (or, for unsuccessful requests, with 4xx or 5xx status codes; details in later sections). The “202 Accepted” response means that a request has been accepted for processing, not that the processing of the request has been completed. The request might or might not be acted upon by Mesos (e.g., master fails during the processing of the request). Any asynchronous responses from these requests will be streamed on long-lived subscription connection.
-
-
-## Calls
-
-The following calls are currently accepted by the master. The canonical source of this information is [scheduler.proto](https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto) (NOTE: The protobuf definitions are subject to change before the beta API is finalized). Note that when sending JSON encoded Calls, schedulers should encode raw bytes in Base64 and strings in UTF-8.
-
-### SUBSCRIBE
-
-This is the first step in the communication process between the scheduler and the master. This is also to be considered as subscription to the “/scheduler” events stream.
-
-To subscribe with the master, the scheduler sends a HTTP POST request with encoded `SUBSCRIBE` message with the required FrameworkInfo. Note that if "subscribe.framework_info.id" is not set, master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream with RecordIO encoding, with the first event being `SUBSCRIBED` event (see details in **Events** section).
-
-```
-SUBSCRIBE Request (JSON):
-
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-Accept: application/json
-Connection: close
-
-{
- “type” : “SUBSCRIBE”,
-
- “subscribe” : {
- “framework_info” : {
- “user” : “foo”,
- “name” : “Example HTTP Framework”
- },
-
- “force” : true
- }
-}
-
-SUBSCRIBE Response Event (JSON):
-HTTP/1.1 200 OK
-
-Content-Type: application/json
-Transfer-Encoding: chunked
-
-<event length>
-{
- “type” : “SUBSCRIBED”,
- “subscribed” : {
- “framework_id” : {“value”:“12220-3440-12532-2345”},
- "heartbeat_interval_seconds" : 15
- }
-}
-<more events>
-```
-
-Alternatively, if “subscribe.framework_info.id” is set, master considers this a request from an already subscribed scheduler reconnecting after a disconnection (e.g., due to failover or network disconnection) and responds with `SUBSCRIBED` event containing the same FrameworkID. The “subscribe.force” field describes how the master reacts when multiple scheduler instances (with the same framework id) attempt to subscribe with the master at the same time (e.g., due to network partition). See the semantics in **Disconnections** section below.
-
-NOTE: In the old version of the API, (re-)registered callbacks also included MasterInfo, which contained information about the master the driver currently connected to. With the new API, since schedulers explicitly subscribe with the leading master (see details below in **Master Detection** section), it’s not relevant anymore.
-
-If subscription fails for whatever reason (e.g., invalid request), a HTTP 4xx response is returned with the error message as part of the body and the connection is closed.
-
-Scheduler must make additional HTTP requests to the “/scheduler” endpoint only after it has opened a persistent connection to it by sending a `SUBSCRIBE` request and received a `SUBSCRIBED` response. Calls made without subscription will result in a “403 Forbidden“ instead of a “202 Accepted“ response. A scheduler might also receive a “400 Bad Request” response if the HTTP request is malformed (e.g., malformed HTTP headers).
-
-### TEARDOWN
-Sent by the scheduler when it wants to tear itself down. When Mesos receives this request it will shut down all executors (and consequently kill tasks) and remove persistent volumes (if requested). It then removes the framework and closes all open connections from this scheduler to the Master.
-
-```
-TEARDOWN Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “TEARDOWN”,
-}
-
-TEARDOWN Response:
-HTTP/1.1 202 Accepted
-```
-
-### ACCEPT
-Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, its resources are considered allocated to the framework. Also, any of the offer’s resources not used in the `ACCEPT` call (e.g., to launch a task) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.).
-
-```
-ACCEPT Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “ACCEPT”,
- “accept” : {
- “offer_ids” : [
- {“value” : “12220-3440-12532-O12”},
- {“value” : “12220-3440-12532-O12”}
- ],
- “operations” : [ {“type” : “LAUNCH”, “launch” : {...}} ],
- “filters” : {...}
- }
-}
-
-ACCEPT Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### DECLINE
-Sent by the scheduler to explicitly decline offer(s) received. Note that this is same as sending an `ACCEPT` call with no operations.
-
-```
-DECLINE Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “DECLINE”,
- “decline” : {
- “offer_ids” : [
- {“value” : “12220-3440-12532-O12”},
- {“value” : “12220-3440-12532-O13”}
- ],
- “filters” : {...}
- }
-}
-
-DECLINE Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### REVIVE
-Sent by the scheduler to remove any/all filters that it has previously set via `ACCEPT` or `DECLINE` calls.
-
-```
-REVIVE Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “REVIVE”,
-}
-
-REVIVE Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### KILL
-Sent by the scheduler to kill a specific task. If the scheduler has a custom executor, the kill is forwarded to the executor and it is up to the executor to kill the task and send a `TASK_KILLED` (or `TASK_FAILED`) update. Mesos releases the resources for a task once it receives a terminal update for the task. If the task is unknown to the master, a `TASK_LOST` will be generated.
-
-```
-KILL Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “KILL”,
- “kill” : {
- “task_id” : {“value” : “12220-3440-12532-my-task”},
- “agent_id” : {“value” : “12220-3440-12532-S1233”}
- }
-}
-
-KILL Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### SHUTDOWN
-Sent by the scheduler to shutdown a specific custom executor (NOTE: This is a new call that was not present in the old API). When an executor gets a shutdown event, it is expected to kill all its tasks (and send `TASK_KILLED` updates) and terminate. If an executor doesn’t terminate within a certain timeout (configurable via “--executor_shutdown_grace_period” agent flag), the agent will forcefully destroy the container (executor and its tasks) and transitions its active tasks to `TASK_LOST`.
-
-```
-SHUTDOWN Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “SHUTDOWN”,
- “shutdown” : {
- “executor_id” : {“value” : “123450-2340-1232-my-executor”},
- “agent_id” : {“value” : “12220-3440-12532-S1233”}
- }
-}
-
-SHUTDOWN Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### ACKNOWLEDGE
-Sent by the scheduler to acknowledge a status update. Note that with the new API, schedulers are responsible for explicitly acknowledging the receipt of status updates that have “status.uuid()” set. These status updates are reliably retried until they are acknowledged by the scheduler. The scheduler must not acknowledge status updates that do not have “status.uuid()” set as they are not retried. "uuid" is raw bytes encoded in Base64.
-
-```
-ACKNOWLEDGE Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “ACKNOWLEDGE”,
- “acknowledge” : {
- “agent_id” : {“value” : “12220-3440-12532-S1233”},
- “task_id” : {“value” : “12220-3440-12532-my-task”},
- “uuid” : “jhadf73jhakdlfha723adf”
- }
-}
-
-ACKNOWLEDGE Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### RECONCILE
-Sent by the scheduler to query the status of non-terminal tasks. This causes the master to send back `UPDATE` events for each task in the list. Tasks that are no longer known to Mesos will result in `TASK_LOST` updates. If the list of tasks is empty, master will send `UPDATE` events for all currently known tasks of the framework.
-
-```
-RECONCILE Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “RECONCILE”,
- “reconcile” : {
- “tasks” : [
- { “task_id” : { “value” : “312325” },
- “agent_id” : { “value” : “123535” }
- }
- ]
- }
-}
-
-RECONCILE Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### MESSAGE
-Sent by the scheduler to send arbitrary binary data to the executor. Note that Mesos neither interprets this data nor makes any guarantees about the delivery of this message to the executor. "data" is raw bytes encoded in Base64.
-
-```
-MESSAGE Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “MESSAGE”,
- “message” : {
- “agent_id” : {“value” : “12220-3440-12532-S1233”},
- “executor_id” : {“value” : “my-framework-executor”},
- “data” : “adaf838jahd748jnaldf”
- }
-}
-
-MESSAGE Response:
-HTTP/1.1 202 Accepted
-
-```
-
-### REQUEST
-Sent by the scheduler to request resources from the master/allocator. The built-in hierarchical allocator simply ignores this request but other allocators (modules) can interpret this in a customizable fashion.
-
-```
-Request (JSON):
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost:5050
-Content-Type: application/json
-
-{
- “framework_id” : {“value” : “12220-3440-12532-2345”},
- “type” : “REQUEST”,
- “requests” : [
- {
- “agent_id” : {“value” : “12220-3440-12532-S1233”},
- “resources” : {}
- },
- ]
-}
-
-REQUEST Response:
-HTTP/1.1 202 Accepted
-
-```
-
-## Events
-
-Scheduler is expected to keep a **persistent** connection open to “/scheduler” endpoint even after getting a SUBSCRIBED HTTP Response event. This is indicated by “Connection: keep-alive” and “Transfer-Encoding: chunked” headers with *no* “Content-Length” header set. All subsequent events that are relevant to this framework generated by Mesos are streamed on this connection. Master encodes each Event in RecordIO format, i.e., string representation of length of the event in bytes followed by JSON or binary Protobuf (possibly compressed) encoded event. Note that the value of length will never be ‘0’ and the size of the length will be the size of unsigned integer (i.e., 64 bits). Also, note that the RecordIO encoding should be decoded by the scheduler whereas the underlying HTTP chunked encoding is typically invisible at the application (scheduler) layer. The type of content encoding used for the events will be determined by the accept header of the POST request (e
.g., Accept: application/json).
-
-The following events are currently sent by the master. The canonical source of this information is at [scheduler.proto](include/mesos/v1/scheduler/scheduler.proto). Note that when sending JSON encoded events, master encodes raw bytes in Base64 and strings in UTF-8.
-
-### SUBSCRIBED
-The first event sent by the master when the scheduler sends a `SUBSCRIBE` request on the persistent connection. See `SUBSCRIBE` in Calls section for the format.
-
-
-### OFFERS
-Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on a agent. Until the scheduler 'Accept's or 'Decline's an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g. due to a lost agent or `--offer_timeout`.
-
-```
-OFFERS Event (JSON)
-<event-length>
-{
- “type” : “OFFERS”,
- “offers” : [
- {
- “offer_id”:{“value”: “12214-23523-O235235”},
- “framework_id”:{“value”: “12124-235325-32425”},
- “agent_id”:{“value”: “12325-23523-S23523”},
- “hostname”:“agent.host”,
- “resources”:[...],
- “attributes”:[...],
- “executor_ids”:[]
- }
- ]
-}
-```
-
-### RESCIND
-Sent by the master when a particular offer is no longer valid (e.g., the agent corresponding to the offer has been removed) and hence needs to be rescinded. Any future calls (`ACCEPT` / `DECLINE`) made by the scheduler regarding this offer will be invalid.
-
-```
-RESCIND Event (JSON)
-<event-length>
-{
- “type” : “RESCIND”,
- “rescind” : {
- “offer_id” : { “value” : “12214-23523-O235235”}
- }
-}
-```
-
-### UPDATE
-Sent by the master whenever there is a status update that is generated by the executor, agent or master. Status updates should be used by executors to reliably communicate the status of the tasks that they manage. It is crucial that a terminal update (e.g., `TASK_FINISHED`, `TASK_KILLED`, `TASK_FAILED`) is sent by the executor as soon as the task terminates, in order for Mesos to release the resources allocated to the task. It is also the responsibility of the scheduler to explicitly acknowledge the receipt of status updates that are reliably retried. See `ACKNOWLEDGE` in the Calls section above for the semantics. Note that `uuid` and `data` are raw bytes encoded in Base64.
-
-
-```
-UPDATE Event (JSON)
-
-<event-length>
-{
- “type” : “UPDATE”,
- “update” : {
- “status” : {
- “task_id” : { “value” : “12344-my-task”},
- “state” : “TASK_RUNNING”,
- “source” : “SOURCE_EXECUTOR”,
- “uuid” : “adfadfadbhgvjayd23r2uahj”,
- "bytes" : "uhdjfhuagdj63d7hadkf"
-
- }
- }
-}
-```
-
-### MESSAGE
-A custom message generated by the executor that is forwarded to the scheduler by the master. Note that this message is not interpreted by Mesos and is only forwarded (without reliability guarantees) to the scheduler. It is up to the executor to retry if the message is dropped for any reason. Note that `data` is raw bytes encoded as Base64.
-
-```
-MESSAGE Event (JSON)
-
-<event-length>
-{
- “type” : “MESSAGE”,
- “message” : {
- “agent_id” : { “value” : “12214-23523-S235235”},
- “executor_id” : { “value” : “12214-23523-my-executor”},
- “data” : “adfadf3t2wa3353dfadf”
- }
-}
-```
-
-
-### FAILURE
-Sent by the master when a agent is removed from the cluster (e.g., failed health checks) or when an executor is terminated. Note that, this event coincides with receipt of terminal `UPDATE` events for any active tasks belonging to the agent or executor and receipt of `RESCIND` events for any outstanding offers belonging to the agent. Note that there is no guaranteed order between the `FAILURE`, `UPDATE` and `RESCIND` events.
-
-```
-FAILURE Event (JSON)
-
-<event-length>
-{
- “type” : “FAILURE”,
- “failure” : {
- “agent_id” : { “value” : “12214-23523-S235235”},
- “executor_id” : { “value” : “12214-23523-my-executor”},
- “status” : 1
- }
-}
-```
-
-### ERROR
-Sent by the master when an asynchronous error event is generated (e.g., a framework is not authorized to subscribe with the given role). It is recommended that the framework abort when it receives an error and retry subscription as necessary.
-
-```
-ERROR Event (JSON)
-
-<event-length>
-{
- “type” : “ERROR”,
- “message” : “Framework is not authorized”
-}
-```
-
-### HEARTBEAT
-This event is periodically sent by the master to inform the scheduler that a connection is alive. This also helps ensure that network intermediates do not close the persistent subscription connection due to lack of data flow. See the next section on how a scheduler can use this event to deal with network partitions.
-
-```
-HEARTBEAT Event (JSON)
-
-<event-length>
-{
- “type” : “HEARTBEAT”,
-}
-```
-
-## Disconnections
-
-Master considers a scheduler disconnected if the persistent subscription connection (opened via `SUBSCRIBE` request) to “/scheduler” breaks. The connection could break for several reasons, e.g., scheduler restart, scheduler failover, network error. Note that the master doesn’t keep track of non-subscription connection(s) to
-“/scheduler” because it is not expected to be a persistent connection.
-
-If master realizes that the subscription connection is broken, it marks the scheduler as “disconnected” and starts a failover timeout (failover timeout is part of FrameworkInfo). It also drops any pending events in its queue. Additionally, it rejects subsequent non-subscribe HTTP requests to “/scheduler” with “403 Forbidden”, until the scheduler subscribes again with “/scheduler”. If the scheduler *does not* re-subscribe within the failover timeout, the master considers the scheduler gone forever and shuts down all its executors, thus killing all its tasks. Therefore, all production schedulers are recommended to use a high value (e.g., 4 weeks) for the failover timeout.
-
-NOTE: To force shutdown a framework before the framework timeout elapses (e.g., during framework development and testing), either the framework can send `TEARDOWN` call (part of Scheduler API) or an operator can use the “/master/teardown” endpoint (part of Operator API).
-
-If the scheduler realizes that its subscription connection to “/scheduler” is broken, it should attempt to open a new persistent connection to the
-“/scheduler” (on possibly new master based on the result of master detection) and resubscribe. It should not send new non-subscribe HTTP requests to “/scheduler” unless it gets a `SUBSCRIBED` event; such requests will result in “403 Forbidden”.
-
-If the master does not realize that the subscription connection is broken, but the scheduler realizes it, the scheduler might open a new persistent connection to
-“/scheduler” via `SUBSCRIBE`. In this case, the semantics depend on the value of `subscribe.force`. If set to true, master closes the existing subscription connection and allows subscription on the new connection. If set to false, the new connection attempt is disallowed in favor of the existing connection. The invariant here is that, only one persistent subscription connection for a given FrameworkID is allowed on the master. For HA schedulers, it is recommended that a scheduler instance set `subscribe.force` to true only when it just got elected and set it to false for all subsequent reconnection attempts (e.g, due to disconnection or master failover).
-
-### Network partitions
-
-In the case of a network partition, the subscription connection between the scheduler and master might not necessarily break. To be able to detect this scenario, master periodically (e.g., 15s) sends `HEARTBEAT` events (similar in vein to Twitter’s Streaming API). If a scheduler doesn’t receive a bunch (e.g., 5) of these heartbeats within a time window, it should immediately disconnect and try to re-subscribe. It is highly recommended for schedulers to use an exponential backoff strategy (e.g., upto a maximum of 15s) to avoid overwhelming the master while reconnecting. Schedulers can use a similar timeout (e.g., 75s) for receiving responses to any HTTP requests.
-
-## Master detection
-
-Mesos has a high-availability mode that uses multiple Mesos masters; one active master (called the leader or leading master) and several standbys in case it fails. The masters elect the leader, with ZooKeeper coordinating the election. For more details please refer to the [documentation](/documentation/latest/high-availability.md).
-
-Schedulers are expected to make HTTP requests to the leading master. If requests are made to a non-leading master a “HTTP 307 Temporary Redirect” will be received with the “Location” header pointing to the leading master.
-
-Example subscription workflow with redirection when the scheduler hits a non-leading master.
-
-```
-Scheduler → Master
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost1:5050
-Content-Type: application/json
-Accept: application/json
-Connection: keep-alive
-
-{
- “framework_info” : {
- “user” : “foo”,
- “name” : “Example HTTP Framework”
- },
- “type” : “SUBSCRIBE”
-}
-
-Master → Scheduler
-HTTP/1.1 307 Temporary Redirect
-Location: masterhost2:5050
-
-
-Scheduler → Master
-POST /api/v1/scheduler HTTP/1.1
-
-Host: masterhost2:5050
-Content-Type: application/json
-Accept: application/json
-Connection: keep-alive
-
-{
- “framework_info” : {
- “user” : “foo”,
- “name” : “Example HTTP Framework”
- },
- “type” : “SUBSCRIBE”
-}
-```
-
-If the scheduler knows the list of master’s hostnames for a cluster, it could use this mechanism to find the leading master to subscribe with. Alternatively, the scheduler could use a pure language library that detects the leading master given a ZooKeeper (or etcd) URL. For a `C++` library that does ZooKeeper based master detection please look at `src/scheduler/scheduler.cpp`.