You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by dl...@apache.org on 2015/02/24 06:22:08 UTC
svn commit: r1661841 [3/3] - in /mesos/site: publish/
publish/documentation/app-framework-development-guide/
publish/documentation/authorization/ publish/documentation/configuration/
publish/documentation/getting-started/ publish/documentation/latest/a...
Modified: mesos/site/source/documentation/latest/app-framework-development-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/app-framework-development-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/app-framework-development-guide.md (original)
+++ mesos/site/source/documentation/latest/app-framework-development-guide.md Tue Feb 24 05:22:07 2015
@@ -79,12 +79,15 @@ Declared in `MESOS_HOME/include/mesos/sc
/**
* Invoked when the status of a task has changed (e.g., a slave is
* lost and so the task is lost, a task finishes and an executor
- * sends a status update saying so, etc). Note that returning from
- * this callback _acknowledges_ receipt of this status update! If
- * for whatever reason the scheduler aborts during this callback (or
+ * sends a status update saying so, etc). If implicit
+ * acknowledgements are being used, then returning from this
+ * callback _acknowledges_ receipt of this status update! If for
+ * whatever reason the scheduler aborts during this callback (or
* the process exits) another status update will be delivered (note,
* however, that this is currently not true if the slave sending the
- * status update is lost/fails during that time).
+ * status update is lost/fails during that time). If explicit
+ * acknowledgements are in use, the scheduler must acknowledge this
+ * status on the driver.
*/
virtual void statusUpdate(SchedulerDriver* driver,
const TaskStatus& status) = 0;
Modified: mesos/site/source/documentation/latest/authorization.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/authorization.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/authorization.md (original)
+++ mesos/site/source/documentation/latest/authorization.md Tue Feb 24 05:22:07 2015
@@ -140,7 +140,7 @@ There are couple of important things to
7. Framework `foo` can only register with `analytics` role but no other roles. Also, no other framework can register with any roles.
{
- "permissive" : "false",
+ "permissive" : false,
"register_frameworks": [
{
@@ -154,7 +154,7 @@ There are couple of important things to
8. Only `ops` principal can shutdown any frameworks through "/shutdown" HTTP endpoint.
{
- "permissive" : "false",
+ "permissive" : false,
"shutdown_frameworks": [
{
Modified: mesos/site/source/documentation/latest/configuration.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/configuration.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/configuration.md (original)
+++ mesos/site/source/documentation/latest/configuration.md Tue Feb 24 05:22:07 2015
@@ -7,7 +7,7 @@ layout: documentation
The Mesos master and slave can take a variety of configuration options through command-line arguments, or environment variables. A list of the available options can be seen by running `mesos-master --help` or `mesos-slave --help`. Each option can be set in two ways:
-* By passing it to the binary using `--option_name=value`.
+* By passing it to the binary using `--option_name=value`, either specifying the value directly, or specifying a file in which the value resides (`--option_name=file://path/to/file`). The path can be absolute or relative to the current working directory.
* By setting the environment variable `MESOS_OPTION_NAME` (the option name with a `MESOS_` prefix added to it).
Configuration values are searched for first in the environment, then on the command-line.
@@ -158,8 +158,7 @@ If you have special compilation requirem
ZooKeeper URL (used for leader election amongst masters)
May be one of:
<pre><code>zk://host1:port1,host2:port2,.../path
-zk://username:password@host1:port1,host2:port2,.../path
-file://path/to/file (where file contains one of the above)</code></pre>
+zk://username:password@host1:port1,host2:port2,.../path</code></pre>
</td>
</tr>
</table>
@@ -181,10 +180,9 @@ file://path/to/file (where file contains
--acls=VALUE
</td>
<td>
- The value could be a JSON formatted string of ACLs
- or a file path containing the JSON formatted ACLs used
- for authorization. Path could be of the form <code>file:///path/to/file</code>
- or <code>/path/to/file</code>.
+ The value is a JSON formatted string of ACLs. Remember you can also use
+ the <code>file:///path/to/file</code> argument value format to write the
+ JSON in a file.
<p/>
See the ACLs protobuf in mesos.proto for the expected format.
<p/>
@@ -267,7 +265,7 @@ file://path/to/file (where file contains
Either a path to a text file with a list of credentials,
each line containing 'principal' and 'secret' separated by whitespace,
or, a path to a JSON-formatted file containing credentials.
- Path could be of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+ Path should be of the form <code>file:///path/to/file</code>
<p/>
JSON file Example:
<pre><code>{
@@ -324,8 +322,9 @@ file://path/to/file (where file contains
subsystems.
<p/>
Use <code>--modules=filepath</code> to specify the list of modules via a
- file containing a JSON formatted string. 'filepath' can be
- of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+ file containing a JSON formatted string. Remember you can also use
+ the <code>file:///path/to/file</code> argument value format to write the
+ JSON in a file.
<p/>
Use <code>--modules="{...}"</code> to specify the list of modules inline.
<p/>
@@ -382,8 +381,9 @@ file://path/to/file (where file contains
or a file path containing the JSON formatted rate limits used
for framework rate limiting.
<p/>
- Path could be of the form <code>file:///path/to/file</code>
- or <code>/path/to/file</code>.
+ Remember you can also use
+ the <code>file:///path/to/file</code> argument value format to write the
+ JSON in a file.
<p/>
See the RateLimits protobuf in mesos.proto for the expected format.
@@ -534,10 +534,15 @@ file://path/to/file (where file contains
--whitelist=VALUE
</td>
<td>
- Path to a file with a list of slaves
- (one per line) to advertise offers for.
+ A filename which contains a list of slaves (one per line) to advertise
+ offers for. The file is watched, and periodically re-read to refresh
+ the slave whitelist. By default there is no whitelist / all machines are
+ accepted. (default: None)
+ <p/>
+
+ Example:
+ <pre><code>file:///etc/mesos/slave_whitelist</code></pre>
<p/>
- Path could be of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>. (default: *)
</td>
</tr>
<tr>
@@ -613,7 +618,8 @@ file://path/to/file (where file contains
</li>
<li> a path to a file containing either one of the above options </li>
-<pre><code> --master=file://path/to/file (where file contains one of the above)</code></pre>
+ <pre><code>You can also use the <code>file:///path/to/file</code> syntax
+ to read the argument from a file which contains one of the above</code></pre>
</li>
</ol>
Examples:
@@ -724,13 +730,18 @@ file://path/to/file (where file contains
--credential=VALUE
</td>
<td>
- Either a path to a text with a single line
+ A path to a text file with a single line
containing 'principal' and 'secret' separated by whitespace.
+
<p/>
Or a path containing the JSON formatted information used for one credential.
<p/>
- Path could be of the form <code>file:///path/to/file< code> or <code>/path/to/file</code>.
+ Path should be of the form <code>file://path/to/file</code>.
<p/>
+ Remember you can also use
+ the <code>file:///path/to/file</code> argument value format to read the
+ value from a file.
+ </p>
JSON file example:
<pre><code>{
"principal": "username",
@@ -930,9 +941,8 @@ file://path/to/file (where file contains
List of modules to be loaded and be available to the internal
subsystems.
<p/>
- Use <code>--modules=filepath</code> to specify the list of modules via a
- file containing a JSON formatted string. 'filepath' can be
- of the form <code>file:///path/to/file</code> or <code>/path/to/file</code>.
+ Remember you can also use the <code>file:///path/to/file</code>
+ argument value format to have the value read from a file.
<p/>
Use <code>--modules="{...}"</code> to specify the list of modules inline.
<p/>
Modified: mesos/site/source/documentation/latest/getting-started.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/getting-started.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/getting-started.md (original)
+++ mesos/site/source/documentation/latest/getting-started.md Tue Feb 24 05:22:07 2015
@@ -13,9 +13,9 @@ There are different ways you can get Mes
$ wget http://www.apache.org/dist/mesos/0.20.1/mesos-0.20.1.tar.gz
$ tar -zxf mesos-0.20.1.tar.gz
-2. Clone the Mesos git [repository](http://git-wip-us.apache.org/repos/asf/mesos.git) (***Advanced Users Only***)
+2. Clone the Mesos git [repository](https://git-wip-us.apache.org/repos/asf/mesos.git) (***Advanced Users Only***)
- $ git clone http://git-wip-us.apache.org/repos/asf/mesos.git
+ $ git clone https://git-wip-us.apache.org/repos/asf/mesos.git
## System Requirements
@@ -77,7 +77,7 @@ There are different ways you can get Mes
$ sudo yum groupinstall -y "Development Tools"
- $ sudo yum install -y python-devel java-1.7.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel
+ $ sudo yum install -y python-devel java-1.7.0-openjdk-devel zlib-devel libcurl-devel openssl-devel cyrus-sasl-devel cyrus-sasl-md5 apr-devel subversion-devel apr-utils-devel
# Install maven.
$ wget http://mirror.nexcess.net/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
Modified: mesos/site/source/documentation/latest/mesos-c++-style-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/mesos-c%2B%2B-style-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/mesos-c++-style-guide.md (original)
+++ mesos/site/source/documentation/latest/mesos-c++-style-guide.md Tue Feb 24 05:22:07 2015
@@ -103,7 +103,7 @@ const typename map::iterator& i = values
// 2: Don't use.
auto authorizer = LocalAuthorizer::create(acls);
// Compare with
-Try<Owned<LocalAuthorizer>> authorizer = LocalAuthorizer::create();
+Try<Owned<LocalAuthorizer>> authorizer = LocalAuthorizer::create();
</pre>
* Rvalue references.
Modified: mesos/site/source/documentation/latest/modules.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/modules.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/modules.md (original)
+++ mesos/site/source/documentation/latest/modules.md Tue Feb 24 05:22:07 2015
@@ -8,9 +8,7 @@ Experimental support for Mesos modules w
### Disclaimer
-- Use and development of Mesos modules is at own risk! Only graced modules
-(modules that are part of Mesos distribution) are maintained by the Mesos
-project.
+- Use and development of Mesos modules is at own risk!
- Please direct all questions to the relevant module writer and/or write to
modules@mesos.apache.org. Questions related to modules sent to user and dev list
@@ -119,8 +117,39 @@ If both "file" and "name" are specified,
## What kinds of modules are supported?
-Mesos currently only provides Isolator and Authentication modules. Additional
-graced modules will be added in the near future.
+Here are the various module kinds currently available:
+
+### Anonymous
+
+Anonymous modules do __not__ receive any callbacks but simply
+coexists with their parent process.
+
+Unlike other named modules, an anonymous module does not directly
+replace or provide essential Mesos functionality (such as an
+Isolator module does). Unlike a decorator module it does not
+directly add or inject data into Mesos core either.
+
+Anonymous modules do not require any specific selectors (flags) as
+they are immediately instantiated when getting loaded via the
+`--modules` flag by the Mesos master or slave.
+
+### Authentication
+
+Authenticatee and Authenticator modules allow for third parties to quickly
+develop and plug-in new authentication methods. An example for such modules
+could be to support PAM (LDAP, MySQL, NIS, UNIX) backed authentication.
+
+### Hook
+
+Similar to Apache Webserver Modules, hooks allows module writers to tie into
+internal components which may not be suitable to be abstracted entirely behind
+modules but rather let's them define actions on so-called hooks.
+
+### Isolator
+
+Isolator modules enable experimenting with specialized isolation and monitoring
+capabilities. Examples of these could be 3rdparty resource isolation mechanisms
+for GPGPU hardware, networking, etc.
## Writing Mesos modules
Modified: mesos/site/source/documentation/latest/release-guide.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/release-guide.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/release-guide.md (original)
+++ mesos/site/source/documentation/latest/release-guide.md Tue Feb 24 05:22:07 2015
@@ -138,3 +138,7 @@ Per the guidelines [when to archive](htt
1. Checkout the mesos distribution folder: `svn co https://dist.apache.org/repos/dist/release/mesos`
2. Remove all minor versions that are no longer under development and commit the change.
+
+## Set the release date
+
+1. Find the released Mesos version on https://issues.apache.org/jira/plugins/servlet/project-config/MESOS/versions, and update the release date.
Modified: mesos/site/source/documentation/latest/slave-recovery.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/slave-recovery.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/slave-recovery.md (original)
+++ mesos/site/source/documentation/latest/slave-recovery.md Tue Feb 24 05:22:07 2015
@@ -15,18 +15,20 @@ Mesos slave could be restarted for an up
Slave recovery works by having the slave checkpoint enough information (e.g., Task Info, Executor Info, Status Updates) about the running tasks and executors to local disk. Once the slave ***and*** the framework(s) enable checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. Note that if the host running the slave process is rebooted all the executors/tasks are killed.
-> NOTE: To enable slave recovery both the slave and the framework should explicitly request checkpointing.
+> NOTE: To enable recovery the framework should explicitly request checkpointing.
> Alternatively, a framework that doesn't want the disk i/o overhead of checkpointing can opt out of checkpointing.
## Enabling slave checkpointing
+> NOTE: From Mesos 0.22.0 slave checkpointing will be automatically enabled for all slaves.
As part of this feature, 4 new flags were added to the slave.
* `checkpoint` : Whether to checkpoint slave and frameworks information
- to disk [Default: false].
+ to disk [Default: true].
- This enables a restarted slave to recover status updates and reconnect
with (--recover=reconnect) or kill (--recover=kill) old executors.
+ > NOTE: From Mesos 0.22.0 this flag will be removed as it will be enabled for all slaves.
* `strict` : Whether to do recovery in strict mode [Default: true].
* If strict=true, any and all recovery errors are considered fatal.
@@ -45,7 +47,7 @@ As part of this feature, 4 new flags wer
reconnect to the slave will self-terminate.
NOTE: This flag is only applicable when `--checkpoint` is enabled.
-> NOTE: If checkpointing is enabled on the slave, but none of the frameworks have enabled checkpointing,
+> NOTE: If none of the frameworks have enabled checkpointing,
> executors/tasks of frameworks die when the slave dies and are not recovered.
A restarted slave should re-register with master within a timeout (currently, 75s). If the slave takes longer
Modified: mesos/site/source/documentation/latest/upgrades.md
URL: http://svn.apache.org/viewvc/mesos/site/source/documentation/latest/upgrades.md?rev=1661841&r1=1661840&r2=1661841&view=diff
==============================================================================
--- mesos/site/source/documentation/latest/upgrades.md (original)
+++ mesos/site/source/documentation/latest/upgrades.md Tue Feb 24 05:22:07 2015
@@ -8,6 +8,8 @@ This document serves as a guide for user
## (WIP) Upgrading from 0.21.x to 0.22.x
+**NOTE** The C++/Java/Python scheduler bindings have been updated. In particular, the driver can be constructed with an additional argument that specifies whether to use implicit driver acknowledgements. In `statusUpdate`, the `TaskStatus` now includes a UUID to make explicit acknowledgements possible.
+
**NOTE**: The Authentication API has changed slightly in this release to support additional authentication mechanisms. The change from 'string' to 'bytes' for AuthenticationStartMessage.data has no impact on C++ or the over-the-wire representation, so it only impacts pure language bindings for languages like Java and Python that use different types for UTF-8 strings vs. byte arrays.
```
@@ -17,6 +19,17 @@ message AuthenticationStartMessage {
}
```
+**NOTE** All Mesos arguments can now be passed using file:// to read them out of a file (either an absolute or relative path). The --credentials, --whitelist, and any flags that expect JSON backed arguments (such as --modules) behave as before, although support for just passing a absolute path for any JSON flags rather than file:// has been deprecated and will produce a warning (and the absolute path behavior will be removed in a future release).
+
+In order to upgrade a running cluster:
+
+* Install the new master binaries and restart the masters.
+* Install the new slave binaries and restart the slaves.
+* Upgrade the schedulers:
+ * For Java schedulers, link the new native library against the new JAR. The JAR contains API changes per the **NOTE** above. A 0.21.0 JAR will work with a 0.22.0 libmesos. A 0.22.0 JAR will work with a 0.21.0 libmesos if explicit acks are not being used. 0.22.0 and 0.21.0 are inter-operable at the protocol level between the master and the scheduler.
+ * For Python schedulers, upgrade to use a 0.22.0 egg. If constructing `MesosSchedulerDriverImpl` with `Credentials`, your code must be updated to pass the `implicitAcknowledgements` argument before `Credentials`. You may run a 0.21.0 Python scheduler against a 0.22.0 master, and vice versa.
+* Restart the schedulers.
+* Upgrade the executors by linking the latest native library / jar / egg.
## Upgrading from 0.20.x to 0.21.x