You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by jc...@apache.org on 2016/09/28 18:23:59 UTC
svn commit: r1762695 [19/19] - in /aurora/site: data/ publish/
publish/documentation/0.10.0/ publish/documentation/0.10.0/build-system/
publish/documentation/0.10.0/client-cluster-configuration/
publish/documentation/0.10.0/client-commands/ publish/doc...
Added: aurora/site/source/documentation/0.16.0/reference/task-lifecycle.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/0.16.0/reference/task-lifecycle.md?rev=1762695&view=auto
==============================================================================
--- aurora/site/source/documentation/0.16.0/reference/task-lifecycle.md (added)
+++ aurora/site/source/documentation/0.16.0/reference/task-lifecycle.md Wed Sep 28 18:23:53 2016
@@ -0,0 +1,146 @@
+# Task Lifecycle
+
+When Aurora reads a configuration file and finds a `Job` definition, it:
+
+1. Evaluates the `Job` definition.
+2. Splits the `Job` into its constituent `Task`s.
+3. Sends those `Task`s to the scheduler.
+4. The scheduler puts the `Task`s into `PENDING` state, starting each
+ `Task`'s life cycle.
+
+
+
+
+Please note, a couple of task states described below are missing from
+this state diagram.
+
+
+## PENDING to RUNNING states
+
+When a `Task` is in the `PENDING` state, the scheduler constantly
+searches for machines satisfying that `Task`'s resource request
+requirements (RAM, disk space, CPU time) while maintaining configuration
+constraints such as "a `Task` must run on machines dedicated to a
+particular role" or attribute limit constraints such as "at most 2
+`Task`s from the same `Job` may run on each rack". When the scheduler
+finds a suitable match, it assigns the `Task` to a machine and puts the
+`Task` into the `ASSIGNED` state.
+
+From the `ASSIGNED` state, the scheduler sends an RPC to the agent
+machine containing `Task` configuration, which the agent uses to spawn
+an executor responsible for the `Task`'s lifecycle. When the scheduler
+receives an acknowledgment that the machine has accepted the `Task`,
+the `Task` goes into `STARTING` state.
+
+`STARTING` state initializes a `Task` sandbox. When the sandbox is fully
+initialized, Thermos begins to invoke `Process`es. Also, the agent
+machine sends an update to the scheduler that the `Task` is
+in `RUNNING` state.
+
+
+
+## RUNNING to terminal states
+
+There are various ways that an active `Task` can transition into a terminal
+state. By definition, it can never leave this state. However, depending on
+nature of the termination and the originating `Job` definition
+(e.g. `service`, `max_task_failures`), a replacement `Task` might be
+scheduled.
+
+### Natural Termination: FINISHED, FAILED
+
+A `RUNNING` `Task` can terminate without direct user interaction. For
+example, it may be a finite computation that finishes, even something as
+simple as `echo hello world.`, or it could be an exceptional condition in
+a long-lived service. If the `Task` is successful (its underlying
+processes have succeeded with exit status `0` or finished without
+reaching failure limits) it moves into `FINISHED` state. If it finished
+after reaching a set of failure limits, it goes into `FAILED` state.
+
+A terminated `TASK` which is subject to rescheduling will be temporarily
+`THROTTLED`, if it is considered to be flapping. A task is flapping, if its
+previous invocation was terminated after less than 5 minutes (scheduler
+default). The time penalty a task has to remain in the `THROTTLED` state,
+before it is eligible for rescheduling, increases with each consecutive
+failure.
+
+### Forceful Termination: KILLING, RESTARTING
+
+You can terminate a `Task` by issuing an `aurora job kill` command, which
+moves it into `KILLING` state. The scheduler then sends the agent a
+request to terminate the `Task`. If the scheduler receives a successful
+response, it moves the Task into `KILLED` state and never restarts it.
+
+If a `Task` is forced into the `RESTARTING` state via the `aurora job restart`
+command, the scheduler kills the underlying task but in parallel schedules
+an identical replacement for it.
+
+In any case, the responsible executor on the agent follows an escalation
+sequence when killing a running task:
+
+ 1. If a `HttpLifecycleConfig` is not present, skip to (4).
+ 2. Send a POST to the `graceful_shutdown_endpoint` and wait 5 seconds.
+ 3. Send a POST to the `shutdown_endpoint` and wait 5 seconds.
+ 4. Send SIGTERM (`kill`) and wait at most `finalization_wait` seconds.
+ 5. Send SIGKILL (`kill -9`).
+
+If the executor notices that all `Process`es in a `Task` have aborted
+during this sequence, it will not proceed with subsequent steps.
+Note that graceful shutdown is best-effort, and due to the many
+inevitable realities of distributed systems, it may not be performed.
+
+### Unexpected Termination: LOST
+
+If a `Task` stays in a transient task state for too long (such as `ASSIGNED`
+or `STARTING`), the scheduler forces it into `LOST` state, creating a new
+`Task` in its place that's sent into `PENDING` state.
+
+In addition, if the Mesos core tells the scheduler that a agent has
+become unhealthy (or outright disappeared), the `Task`s assigned to that
+agent go into `LOST` state and new `Task`s are created in their place.
+From `PENDING` state, there is no guarantee a `Task` will be reassigned
+to the same machine unless job constraints explicitly force it there.
+
+### Giving Priority to Production Tasks: PREEMPTING
+
+Sometimes a Task needs to be interrupted, such as when a non-production
+Task's resources are needed by a higher priority production Task. This
+type of interruption is called a *pre-emption*. When this happens in
+Aurora, the non-production Task is killed and moved into
+the `PREEMPTING` state when both the following are true:
+
+- The task being killed is a non-production task.
+- The other task is a `PENDING` production task that hasn't been
+ scheduled due to a lack of resources.
+
+The scheduler UI shows the non-production task was preempted in favor of
+the production task. At some point, tasks in `PREEMPTING` move to `KILLED`.
+
+Note that non-production tasks consuming many resources are likely to be
+preempted in favor of production tasks.
+
+### Making Room for Maintenance: DRAINING
+
+Cluster operators can set agent into maintenance mode. This will transition
+all `Task` running on this agent into `DRAINING` and eventually to `KILLED`.
+Drained `Task`s will be restarted on other agents for which no maintenance
+has been announced yet.
+
+
+
+## State Reconciliation
+
+Due to the many inevitable realities of distributed systems, there might
+be a mismatch of perceived and actual cluster state (e.g. a machine returns
+from a `netsplit` but the scheduler has already marked all its `Task`s as
+`LOST` and rescheduled them).
+
+Aurora regularly runs a state reconciliation process in order to detect
+and correct such issues (e.g. by killing the errant `RUNNING` tasks).
+By default, the proper detection of all failure scenarios and inconsistencies
+may take up to an hour.
+
+To emphasize this point: there is no uniqueness guarantee for a single
+instance of a job in the presence of network partitions. If the `Task`
+requires that, it should be baked in at the application level using a
+distributed coordination service such as Zookeeper.
Modified: aurora/site/source/documentation/latest/additional-resources/tools.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/additional-resources/tools.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/additional-resources/tools.md (original)
+++ aurora/site/source/documentation/latest/additional-resources/tools.md Wed Sep 28 18:23:53 2016
@@ -19,3 +19,6 @@ Various tools integrate with Aurora. Is
* Packaging and deployment:
- [aurora-packaging](https://github.com/apache/aurora-packaging), the source of the official Aurora packages
+
+* Thrift Clients:
+ - [gorealis](https://github.com/rdelval/gorealis) for communicating with the scheduler using Go
Modified: aurora/site/source/documentation/latest/contributing.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/contributing.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/contributing.md (original)
+++ aurora/site/source/documentation/latest/contributing.md Wed Sep 28 18:23:53 2016
@@ -16,8 +16,10 @@ both languages.
There are issues in [Jira](https://issues.apache.org/jira/browse/AURORA) with the
["newbie" label](https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20labels%20%3D%20newbie%20and%20resolution%3Dunresolved)
-that are good starting places for new Aurora contributors; pick one of these and dive in! Once
-you've got a patch, the next step is to post a review.
+that are good starting places for new Aurora contributors; pick one of these and dive in! To assign
+a task to yourself, first ask for your JIRA id to be whitelisted by either asking in IRC/Slack or by
+emailing dev@apache.aurora.org. Once your JIRA account has been whitelisted you can assign tickets
+to yourself. The next step is to prepare your patch and finally post it for review.
## Getting your ReviewBoard Account
Modified: aurora/site/source/documentation/latest/development/committers-guide.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/development/committers-guide.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/development/committers-guide.md (original)
+++ aurora/site/source/documentation/latest/development/committers-guide.md Wed Sep 28 18:23:53 2016
@@ -56,6 +56,16 @@ must have access to commit to the dist.a
1. Ensure that all issues resolved for this release candidate are tagged with the correct Fix
Version in JIRA, the changelog script will use this to generate the CHANGELOG in step #2.
+To assign the fix version:
+
+ * Look up the [previous release date](https://issues.apache.org/jira/browse/aurora/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel).
+ * Query all issues resolved after that release date: `project = AURORA AND status in (resolved, Closed) and fixVersion is empty and resolutiondate >= "YYYY/MM/DD"`
+ * In the upper right corner of the query result, select Tools > Bulk Edit.
+ * Select all issues > edit issue > set 'Change Fix Version/s' to the release version.
+ * Make sure to uncheck 'Send mail for this update' at the bottom.
+
+2. Prepare RELEASE-NOTES.md for the release. This just boils down to removing the "(Not yet
+released)" suffix from the impending release.
2. Create a release candidate. This will automatically update the CHANGELOG and commit it, create a
branch and update the current version within the trunk. To create a minor version update and publish
@@ -71,14 +81,20 @@ by running
4. Wait for the vote to complete. If the vote fails close the vote by replying to the initial [VOTE]
email sent in step #3 by editing the subject to [RESULT][VOTE] ... and noting the failure reason
-(example [here](http://markmail.org/message/d4d6xtvj7vgwi76f)). Now address any issues and go back to
-step #1 and run again, this time you will use the -r flag to increment the release candidate
-version. This will automatically clean up the release candidate rc0 branch and source distribution.
+(example [here](http://markmail.org/message/d4d6xtvj7vgwi76f)). You'll also need to manually revert
+the commits generated by the release candidate script that incremented the snapshot version and
+updated the changelog. Once that is done, now address any issues and go back to step #1 and run
+again, this time you will use the -r flag to increment the release candidate version. This will
+automatically clean up the release candidate rc0 branch and source distribution.
./build-support/release/release-candidate -l m -r 1 -p
5. Once the vote has successfully passed create the release
+**IMPORTANT: make sure to use the correct release at this final step (e.g.: `-r 1` if rc1 candidate
+has been voted for). Once the release tag is pushed it will be very hard to undo due to remote
+git pre-receive hook explicitly forbidding release tag manipulations.**
+
./build-support/release/release
6. Update the draft email created fom the `release` script in step #5 to include the Apache ID's for
Modified: aurora/site/source/documentation/latest/development/db-migration.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/development/db-migration.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/development/db-migration.md (original)
+++ aurora/site/source/documentation/latest/development/db-migration.md Wed Sep 28 18:23:53 2016
@@ -14,7 +14,7 @@ When adding or altering tables or changi
[schema.sql](../../src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql), a new
migration class should be created under the org.apache.aurora.scheduler.storage.db.migration
package. The class should implement the [MigrationScript](https://github.com/mybatis/migrations/blob/master/src/main/java/org/apache/ibatis/migration/MigrationScript.java)
-interface (see [V001_TestMigration](https://github.com/apache/aurora/blob/rel/0.15.0/src/test/java/org/apache/aurora/scheduler/storage/db/testmigration/V001_TestMigration.java)
+interface (see [V001_TestMigration](https://github.com/apache/aurora/blob/rel/0.16.0/src/test/java/org/apache/aurora/scheduler/storage/db/testmigration/V001_TestMigration.java)
as an example). The upgrade and downgrade scripts are defined in this class. When restoring a
snapshot the list of migrations on the classpath is compared to the list of applied changes in the
DB. Any changes that have not yet been applied are executed and their downgrade script is stored
Modified: aurora/site/source/documentation/latest/development/design-documents.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/development/design-documents.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/development/design-documents.md (original)
+++ aurora/site/source/documentation/latest/development/design-documents.md Wed Sep 28 18:23:53 2016
@@ -9,6 +9,7 @@ in the proposed form.
Current and past documents:
* [Command Hooks for the Aurora Client](../design/command-hooks/)
+* [Dynamic Reservations](https://docs.google.com/document/d/19gV8Po6DIHO14tOC7Qouk8RnboY8UCfRTninwn_5-7c/edit)
* [GPU Resources in Aurora](https://docs.google.com/document/d/1J9SIswRMpVKQpnlvJAMAJtKfPP7ZARFknuyXl-2aZ-M/edit)
* [Health Checks for Updates](https://docs.google.com/document/d/1ZdgW8S4xMhvKW7iQUX99xZm10NXSxEWR0a-21FP5d94/edit)
* [JobUpdateDiff thrift API](https://docs.google.com/document/d/1Fc_YhhV7fc4D9Xv6gJzpfooxbK4YWZcvzw6Bd3qVTL8/edit)
Modified: aurora/site/source/documentation/latest/development/thrift.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/development/thrift.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/development/thrift.md (original)
+++ aurora/site/source/documentation/latest/development/thrift.md Wed Sep 28 18:23:53 2016
@@ -6,7 +6,7 @@ client/server RPC protocol as well as fo
correctly handling additions and renames of the existing members, field removals must be done
carefully to ensure backwards compatibility and provide predictable deprecation cycle. This
document describes general guidelines for making Thrift schema changes to the existing fields in
-[api.thrift](https://github.com/apache/aurora/blob/rel/0.15.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift).
+[api.thrift](https://github.com/apache/aurora/blob/rel/0.16.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift).
It is highly recommended to go through the
[Thrift: The Missing Guide](http://diwakergupta.github.io/thrift-missing-guide/) first to refresh on
@@ -33,7 +33,7 @@ communicate with scheduler/client from v
* Add a new field as an eventual replacement of the old one and implement a dual read/write
anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns
are marked as `NOT NULL`
-* Check [storage.thrift](https://github.com/apache/aurora/blob/rel/0.15.0/api/src/main/thrift/org/apache/aurora/gen/storage.thrift) to see if
+* Check [storage.thrift](https://github.com/apache/aurora/blob/rel/0.16.0/api/src/main/thrift/org/apache/aurora/gen/storage.thrift) to see if
the affected struct is stored in Aurora scheduler storage. If so, it's almost certainly also
necessary to perform a [DB migration](../db-migration/).
* Add a deprecation jira ticket into the vCurrent+1 release candidate
Modified: aurora/site/source/documentation/latest/features/containers.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/containers.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/features/containers.md (original)
+++ aurora/site/source/documentation/latest/features/containers.md Wed Sep 28 18:23:53 2016
@@ -1,16 +1,77 @@
Containers
==========
-Docker
-------
+Aurora supports several containerizers, notably the Mesos containerizer and the Docker
+containerizer. The Mesos containerizer uses native OS features directly to provide isolation between
+containers, while the Docker containerizer delegates container management to the Docker engine.
+
+The support for launching container images via both containerizers has to be
+[enabled by a cluster operator](../../operations/configuration/#containers).
+
+Mesos Containerizer
+-------------------
+
+The Mesos containerizer is the native Mesos containerizer solution. It allows tasks to be
+run with an array of [pluggable isolators](../resource-isolation/) and can launch tasks using
+[Docker](https://github.com/docker/docker/blob/master/image/spec/v1.md) images,
+[AppC](https://github.com/appc/spec/blob/master/SPEC.md) images, or directly on the agent host
+filesystem.
-Aurora has optional support for launching Docker containers, if correctly [configured by an Operator](../../operations/configuration/#docker-containers).
+The following example (available in our [Vagrant environment](../../getting-started/vagrant/))
+launches a hello world example within a `debian/jessie` Docker image:
-Example (available in the [Vagrant environment](../../getting-started/vagrant/)):
+ $ cat /vagrant/examples/jobs/hello_docker_image.aurora
+ hello_loop = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+ task = Task(
+ processes = [hello_loop],
+ resources = Resources(cpu=1, ram=1*MB, disk=8*MB)
+ )
+ jobs = [
+ Service(
+ cluster = 'devcluster',
+ environment = 'devel',
+ role = 'www-data',
+ name = 'hello_docker_image',
+ task = task,
+ container = Mesos(image=DockerImage(name='debian', tag='jessie'))
+ )
+ ]
+
+Docker and Appc images are designated using an appropriate `image` property of the `Mesos`
+configuration object. If either `container` or `image` is left unspecified, the host filesystem
+will be used. Further details of how to specify images can be found in the
+[Reference Documentation](../../reference/configuration/#mesos-object).
+
+By default, Aurora launches processes as the Linux user named like the used role (e.g. `www-data`
+in the example above). This user has to exist on the host filesystem. If it does not exist within
+the container image, it will be created automatically. Otherwise, this user and its primary group
+has to exist in the image with matching uid/gid.
+
+For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit
+[Mesos Containerizer](http://mesos.apache.org/documentation/latest/mesos-containerizer/) and
+[Mesos Container Images](http://mesos.apache.org/documentation/latest/container-image/).
+
+
+Docker Containerizer
+--------------------
+
+The Docker containerizer launches container images using the Docker engine. It may often provide
+more advanced features than the native Mesos containerizer, but has to be installed separately to
+Mesos on each agent host,
+
+Example (available in the [Vagrant environment](../../getting-started/vagrant/)):
- $ cat /vagrant/examples/jobs/docker/hello_docker.aurora
- hello_world_proc = Process(
+ $ cat /vagrant/examples/jobs/hello_docker_engine.aurora
+ hello_loop = Process(
name = 'hello',
cmdline = """
while true; do
@@ -19,42 +80,27 @@ Example (available in the [Vagrant envir
done
""")
- hello_world_docker = Task(
- name = 'hello docker',
- processes = [hello_world_proc],
- resources = Resources(cpu = 1, ram = 1*MB, disk=8*MB)
+ task = Task(
+ processes = [hello_loop],
+ resources = Resources(cpu=1, ram=1*MB, disk=8*MB)
)
jobs = [
Service(
cluster = 'devcluster',
environment = 'devel',
- role = 'docker-test',
+ role = 'www-data',
name = 'hello_docker',
- task = hello_world_docker,
- container = Container(docker = Docker(image = 'python:2.7'))
+ task = task,
+ container = Docker(image = 'python:2.7')
)
]
+Details of how to use Docker via the Docker engine can be found in the
+[Reference Documentation](../../reference/configuration/#docker-object). Please note that in order to
+correctly execute processes inside a job, the Docker container must have Python 2.7 and potentitally
+further Mesos dependencies installed. This limitation does not hold for Docker containers used via
+the Mesos containerizer.
-In order to correctly execute processes inside a job, the docker container must have Python 2.7
-installed. Further details of how to use Docker can be found in the
-[Reference Documentation](../../reference/configuration/#docker-object).
-
-Mesos
------
-
-*Note: In order to use filesystem images with Aurora, you must be running at least Mesos 0.28.x*
-
-Aurora supports specifying a task filesystem image to use with the [Mesos containerizer](http://mesos.apache.org/documentation/latest/container-image/).
-This is done by setting the ```container``` property of the Job to a ```Mesos``` container object
-that includes the image to use. Both [AppC](https://github.com/appc/spec/blob/master/SPEC.md) and
-[Docker](https://github.com/docker/docker/blob/master/image/spec/v1.md) images are supported.
-
-```
-job = Job(
- ...
- container = Mesos(image=DockerImage(name='my-image', tag='my-tag'))
- ...
-)
-```
+For more information on launching Docker containers through the Docker containerizer, visit
+[Docker Containerizer](http://mesos.apache.org/documentation/latest/docker-containerizer/)
Added: aurora/site/source/documentation/latest/features/custom-executors.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/custom-executors.md?rev=1762695&view=auto
==============================================================================
--- aurora/site/source/documentation/latest/features/custom-executors.md (added)
+++ aurora/site/source/documentation/latest/features/custom-executors.md Wed Sep 28 18:23:53 2016
@@ -0,0 +1,152 @@
+Custom Executors
+================
+
+If the need arises to use a Mesos executor other than the Thermos executor, the scheduler can be
+configured to utilize a custom executor by specifying the `-custom_executor_config` flag.
+The flag must be set to the path of a valid executor configuration file.
+
+The configuration file must be a valid **JSON array** and contain, at minimum,
+one executor configuration including the name, command and resources fields and
+must be pointed to by the `-custom_executor_config` flag when the scheduler is
+started.
+
+### Array Entry
+
+Property | Description
+----------------------- | ---------------------------------
+executor (required) | Description of executor.
+task_prefix (required) ) | Prefix given to tasks launched with this executor's configuration.
+volume_mounts (optional) | Volumes to be mounted in container running executor.
+
+#### executor
+
+Property | Description
+----------------------- | ---------------------------------
+name (required) | Name of the executor.
+command (required) | How to run the executor.
+resources (required) | Overhead to use for each executor instance.
+
+#### command
+
+Property | Description
+----------------------- | ---------------------------------
+value (required) | The command to execute.
+arguments (optional) | A list of arguments to pass to the command.
+uris (optional) | List of resources to download into the task sandbox.
+shell (optional) | Run executor via shell.
+
+A note on the command property (from [mesos.proto](https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto)):
+```
+1) If 'shell == true', the command will be launched via shell
+ (i.e., /bin/sh -c 'value'). The 'value' specified will be
+ treated as the shell command. The 'arguments' will be ignored.
+2) If 'shell == false', the command will be launched by passing
+ arguments to an executable. The 'value' specified will be
+ treated as the filename of the executable. The 'arguments'
+ will be treated as the arguments to the executable. This is
+ similar to how POSIX exec families launch processes (i.e.,
+ execlp(value, arguments(0), arguments(1), ...)).
+```
+
+##### uris (list)
+* Follows the [Mesos Fetcher schema](http://mesos.apache.org/documentation/latest/fetcher/)
+
+Property | Description
+----------------------- | ---------------------------------
+value (required) | Path to the resource needed in the sandbox.
+executable (optional) | Change resource to be executable via chmod.
+extract (optional) | Extract files from packed or compressed archives into the sandbox.
+cache (optional) | Use caching mechanism provided by Mesos for resources.
+
+#### resources (list)
+
+Property | Description
+------------------- | ---------------------------------
+name (required) | Name of the resource: cpus or mem.
+type (required) | Type of resource. Should always be SCALAR.
+scalar (required) | Value in float for cpus or int for mem (in MBs)
+
+### volume_mounts (list)
+
+Property | Description
+------------------------ | ---------------------------------
+host_path (required) | Host path to mount inside the container.
+container_path (required) | Path inside the container where `host_path` will be mounted.
+mode (required) | Mode in which to mount the volume, Read-Write (RW) or Read-Only (RO).
+
+A sample configuration is as follows:
+```
+[
+ {
+ "executor": {
+ "name": "myExecutor",
+ "command": {
+ "value": "myExecutor.a",
+ "shell": "false",
+ "arguments": [
+ "localhost:2181",
+ "-verbose",
+ "-config myConfiguration.config"
+ ],
+ "uris": [
+ {
+ "value": "/dist/myExecutor.a",
+ "executable": true,
+ "extract": false,
+ "cache": true
+ },
+ {
+ "value": "/home/user/myConfiguration.config",
+ "executable": false,
+ "extract": false,
+ "cache": false
+ }
+ ]
+ },
+ "resources": [
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": {
+ "value": 1.00
+ }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": {
+ "value": 512
+ }
+ }
+ ]
+ },
+ "volume_mounts": [
+ {
+ "mode": "RO",
+ "container_path": "/path/on/container",
+ "host_path": "/path/to/host/directory"
+ },
+ {
+ "mode": "RW",
+ "container_path": "/container",
+ "host_path": "/host"
+ }
+ ],
+ "task_prefix": "my-executor-"
+ }
+]
+
+```
+
+It should be noted that if you do not use Thermos or a Thermos based executor, links in the scheduler's
+Web UI for tasks will not work (at least for the time being).
+Some information about launched tasks can still be accessed via the Mesos Web UI or via the Aurora Client.
+
+### Using a custom executor
+
+At this time, it is not currently possible create a job that runs on a custom executor using the default
+Aurora client. To allow the scheduler to pick the correct executor, the `JobConfiguration.TaskConfig.ExecutorConfig.name`
+field must be set to match the name used in the custom executor configuration blob. (e.g. to run a job using myExecutor,
+`JobConfiguration.TaskConfig.ExecutorConfig.name` must be set to `myExecutor`). While support for modifying
+this field in Pystachio created, the easiest way to launch jobs with custom executors is to use
+an existing custom Client such as [gorealis](https://github.com/rdelval/gorealis).
Modified: aurora/site/source/documentation/latest/features/job-updates.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/job-updates.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/features/job-updates.md (original)
+++ aurora/site/source/documentation/latest/features/job-updates.md Wed Sep 28 18:23:53 2016
@@ -71,7 +71,7 @@ acknowledging ("heartbeating") job updat
service updates where explicit job health monitoring is vital during the entire job update
lifecycle. Such job updates would rely on an external service (or a custom client) periodically
pulsing an active coordinated job update via a
-[pulseJobUpdate RPC](https://github.com/apache/aurora/blob/rel/0.15.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift).
+[pulseJobUpdate RPC](https://github.com/apache/aurora/blob/rel/0.16.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift).
A coordinated update is defined by setting a positive
[pulse_interval_secs](../../reference/configuration/#updateconfig-objects) value in job configuration
Modified: aurora/site/source/documentation/latest/features/multitenancy.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/multitenancy.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/features/multitenancy.md (original)
+++ aurora/site/source/documentation/latest/features/multitenancy.md Wed Sep 28 18:23:53 2016
@@ -34,6 +34,25 @@ in the case of a typical Job, releases m
increasing level of stability: `devel`, `test`, `staging`, `production`.
+Configuration Tiers
+-------------------
+
+Tier is a predefined bundle of task configuration options. Aurora schedules tasks and assigns them
+resources based on their tier assignment. The default scheduler tier configuration allows for
+3 tiers:
+
+ - `revocable`: The `revocable` tier requires the task to run with [revocable](../resource-isolation/#oversubscription)
+ resources.
+ - `preemptible`: Setting the task’s tier to `preemptible` allows for the possibility of that task
+ being [preempted](#preemption) by other tasks when cluster is running low on resources.
+ - `preferred`: The `preferred` tier prevents the task from using [revocable](../resource-isolation/#oversubscription)
+ resources and from being [preempted](#preemption).
+
+Since it is possible that a cluster is configured with a custom tier configuration, users should
+consult their cluster administrator to be informed of the tiers supported by the cluster. Attempts
+to schedule jobs with an unsupported tier will be rejected by the scheduler.
+
+
Preemption
----------
@@ -46,12 +65,12 @@ prevents this. Active tasks can become t
- both candidate and victim are owned by the same role and the
[priority](../../reference/configuration/#job-objects) of a victim is lower than the
[priority](../../reference/configuration/#job-objects) of the candidate.
- - OR a victim is non-[production](../../reference/configuration/#job-objects) and the candidate is
- [production](../../reference/configuration/#job-objects).
+ - OR a victim is a `preemptible` or `revocable` [tier](#configuration-tiers) task and the candidate
+ is a `preferred` [tier](#configuration-tiers) task.
-In other words, tasks from [production](../../reference/configuration/#job-objects) jobs may preempt
-tasks from any non-production job. However, a production task may only be preempted by tasks from
-production jobs in the same role with higher [priority](../../reference/configuration/#job-objects).
+In other words, tasks from `preferred` [tier](../../reference/configuration/#job-objects) jobs may
+preempt tasks from any `preemptible` or `revocable` job. However, a `preferred` task may only be
+preempted by tasks from `preferred` jobs in the same role with higher [priority](../../reference/configuration/#job-objects).
Aurora requires resource quotas for [production non-dedicated jobs](../../reference/configuration/#job-objects).
Quota is enforced at the job role level and when set, defines a non-preemptible pool of compute resources within
Modified: aurora/site/source/documentation/latest/features/resource-isolation.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/resource-isolation.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/features/resource-isolation.md (original)
+++ aurora/site/source/documentation/latest/features/resource-isolation.md Wed Sep 28 18:23:53 2016
@@ -1,6 +1,9 @@
Resources Isolation and Sizing
==============================
+This document assumes Aurora and Mesos have been configured
+using our [recommended resource isolation settings](../../operations/configuration/#resource-isolation).
+
- [Isolation](#isolation)
- [Sizing](#sizing)
- [Oversubscription](#oversubscription)
@@ -11,11 +14,13 @@ Isolation
Aurora is a multi-tenant system; a single software instance runs on a
server, serving multiple clients/tenants. To share resources among
-tenants, it implements isolation of:
+tenants, it leverages Mesos for isolation of:
* CPU
+* GPU
* memory
* disk space
+* ports
CPU is a soft limit, and handled differently from memory and disk space.
Too low a CPU value results in throttling your application and
@@ -24,10 +29,10 @@ application goes over these values, it's
### CPU Isolation
-Mesos uses a quota based CPU scheduler (the *Completely Fair Scheduler*)
-to provide consistent and predictable performance. This is effectively
-a guarantee of resources -- you receive at least what you requested, but
-also no more than you've requested.
+Mesos can be configured to use a quota based CPU scheduler (the *Completely*
+*Fair Scheduler*) to provide consistent and predictable performance.
+This is effectively a guarantee of resources -- you receive at least what
+you requested, but also no more than you've requested.
The scheduler gives applications a CPU quota for every 100 ms interval.
When an application uses its quota for an interval, it is throttled for
@@ -103,11 +108,11 @@ will be killed shortly after. This is su
### GPU Isolation
-GPU isolation will be supported for Nvidia devices starting from Mesos 0.29.0.
+GPU isolation will be supported for Nvidia devices starting from Mesos 1.0.
Access to the allocated units will be exclusive with no sharing between tasks
-allowed (e.g. no fractional GPU allocation). Until official documentation is released,
-see [Mesos design document](https://docs.google.com/document/d/10GJ1A80x4nIEo8kfdeo9B11PIbS1xJrrB4Z373Ifkpo/edit#heading=h.w84lz7p4eexl)
-for more details.
+allowed (e.g. no fractional GPU allocation). For more details, see the
+[Mesos design document](https://docs.google.com/document/d/10GJ1A80x4nIEo8kfdeo9B11PIbS1xJrrB4Z373Ifkpo/edit#heading=h.w84lz7p4eexl)
+and the [Mesos agent configuration](http://mesos.apache.org/documentation/latest/configuration/).
### Other Resources
@@ -154,26 +159,23 @@ into the application's sandbox space.
GPU is highly dependent on your application requirements and is only limited
by the number of physical GPU units available on a target box.
+
Oversubscription
----------------
-**WARNING**: This feature is currently in alpha status. Do not use it in production clusters!
-
-Mesos [supports a concept of revocable tasks](http://mesos.apache.org/documentation/latest/oversubscription/)
-by oversubscribing machine resources by the amount deemed safe to not affect the existing
-non-revocable tasks. Aurora now supports revocable jobs via a `tier` setting set to `revocable`
-value.
-
-The Aurora scheduler must be configured to receive revocable offers from Mesos and accept revocable
-jobs. If not configured properly revocable tasks will never get assigned to hosts and will stay in
-`PENDING`. Set these scheduler flag to allow receiving revocable Mesos offers:
-
- -receive_revocable_resources=true
-
-Specify a tier configuration file path (unless you want to use the [default](https://github.com/apache/aurora/blob/rel/0.15.0/src/main/resources/org/apache/aurora/scheduler/tiers.json)):
-
- -tier_config=path/to/tiers/config.json
-
+Mesos supports [oversubscription of machine resources](http://mesos.apache.org/documentation/latest/oversubscription/)
+via the concept of revocable tasks. In contrast to non-revocable tasks, revocable tasks are best-effort.
+Mesos reserves the right to throttle or even kill them if they might affect existing high-priority
+user-facing services.
+
+As of today, the only revocable resource supported by Aurora are CPU and RAM resources. A job can
+opt-in to use those by specifying the `revocable` [Configuration Tier](../../features/multitenancy/#configuration-tiers).
+A revocable job will only be scheduled using revocable resources, even if there are plenty of
+non-revocable resources available.
+
+The Aurora scheduler must be [configured to receive revocable offers](../../operations/configuration/#resource-isolation)
+from Mesos and accept revocable jobs. If not configured properly revocable tasks will never get
+assigned to hosts and will stay in `PENDING`.
-See the [Configuration Reference](../../reference/configuration/) for details on how to mark a job
-as being revocable.
+For details on how to mark a job as being revocable, see the
+[Configuration Reference](../../reference/configuration/).
Modified: aurora/site/source/documentation/latest/features/sla-metrics.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/features/sla-metrics.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/features/sla-metrics.md (original)
+++ aurora/site/source/documentation/latest/features/sla-metrics.md Wed Sep 28 18:23:53 2016
@@ -6,6 +6,7 @@ Aurora SLA Measurement
- [Platform Uptime](#platform-uptime)
- [Job Uptime](#job-uptime)
- [Median Time To Assigned (MTTA)](#median-time-to-assigned-\(mtta\))
+ - [Median Time To Starting (MTTS)](#median-time-to-starting-\(mtts\))
- [Median Time To Running (MTTR)](#median-time-to-running-\(mttr\))
- [Limitations](#limitations)
@@ -62,7 +63,7 @@ relevant to uptime calculations. By appl
transition records, we can build a deterministic downtime trace for every given service instance.
A task going through a state transition carries one of three possible SLA meanings
-(see [SlaAlgorithm.java](https://github.com/apache/aurora/blob/rel/0.15.0/src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java) for
+(see [SlaAlgorithm.java](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java) for
sla-to-task-state mapping):
* Task is UP: starts a period where the task is considered to be up and running from the Aurora
@@ -109,7 +110,7 @@ metric that helps track the dependency o
* Per job - `sla_<job_key>_mtta_ms`
* Per cluster - `sla_cluster_mtta_ms`
* Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
-[ResourceAggregates.java](https://github.com/apache/aurora/blob/rel/0.15.0/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java)
+[ResourceBag.java](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java)
* By CPU:
* `sla_cpu_small_mtta_ms`
* `sla_cpu_medium_mtta_ms`
@@ -135,6 +136,42 @@ MTTA only considers instances that have
that are still PENDING. This ensures straggler instances (e.g. with unreasonable resource
constraints) do not affect metric curves.
+### Median Time To Starting (MTTS)
+
+*Median time a job waits for its tasks to reach STARTING state. This is a comprehensive metric
+reflecting on the overall time it takes for the Aurora/Mesos to start initializing the sandbox
+for a task.*
+
+**Collection scope:**
+
+* Per job - `sla_<job_key>_mtts_ms`
+* Per cluster - `sla_cluster_mtts_ms`
+* Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+[ResourceBag.java](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java)
+ * By CPU:
+ * `sla_cpu_small_mtts_ms`
+ * `sla_cpu_medium_mtts_ms`
+ * `sla_cpu_large_mtts_ms`
+ * `sla_cpu_xlarge_mtts_ms`
+ * `sla_cpu_xxlarge_mtts_ms`
+ * By RAM:
+ * `sla_ram_small_mtts_ms`
+ * `sla_ram_medium_mtts_ms`
+ * `sla_ram_large_mtts_ms`
+ * `sla_ram_xlarge_mtts_ms`
+ * `sla_ram_xxlarge_mtts_ms`
+ * By DISK:
+ * `sla_disk_small_mtts_ms`
+ * `sla_disk_medium_mtts_ms`
+ * `sla_disk_large_mtts_ms`
+ * `sla_disk_xlarge_mtts_ms`
+ * `sla_disk_xxlarge_mtts_ms`
+
+**Units:** milliseconds
+
+MTTS only considers instances in STARTING state. This ensures straggler instances (e.g. with
+unreasonable resource constraints) do not affect metric curves.
+
### Median Time To Running (MTTR)
*Median time a job waits for its tasks to reach RUNNING state. This is a comprehensive metric
@@ -145,7 +182,7 @@ reflecting on the overall time it takes
* Per job - `sla_<job_key>_mttr_ms`
* Per cluster - `sla_cluster_mttr_ms`
* Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
-[ResourceAggregates.java](https://github.com/apache/aurora/blob/rel/0.15.0/src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java)
+[ResourceBag.java](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java)
* By CPU:
* `sla_cpu_small_mttr_ms`
* `sla_cpu_medium_mttr_ms`
Modified: aurora/site/source/documentation/latest/index.html.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/index.html.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/index.html.md (original)
+++ aurora/site/source/documentation/latest/index.html.md Wed Sep 28 18:23:53 2016
@@ -20,6 +20,7 @@ Description of important Aurora features
* [Containers](features/containers/)
* [Cron Jobs](features/cron-jobs/)
+ * [Custom Executors](features/custom-executors/)
* [Job Updates](features/job-updates/)
* [Multitenancy](features/multitenancy/)
* [Resource Isolation](features/resource-isolation/)
Modified: aurora/site/source/documentation/latest/operations/configuration.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/operations/configuration.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/operations/configuration.md (original)
+++ aurora/site/source/documentation/latest/operations/configuration.md Wed Sep 28 18:23:53 2016
@@ -29,8 +29,9 @@ Like Mesos, Aurora uses command-line fla
# Environment variables controlling libmesos
export JAVA_HOME=...
export GLOG_v=1
- # Port used to communicate with the Mesos master and for the replicated log
+ # Port and public ip used to communicate with the Mesos master and for the replicated log
export LIBPROCESS_PORT=8083
+ export LIBPROCESS_IP=192.168.33.7
JAVA_OPTS="${JAVA_OPTS[*]}" exec "$AURORA_HOME/bin/aurora-scheduler" "${AURORA_FLAGS[@]}"
@@ -69,7 +70,7 @@ for Mesos replicated log files to ensure
### `-native_log_zk_group_path`
ZooKeeper path used for Mesos replicated log quorum discovery.
-See [code](https://github.com/apache/aurora/blob/rel/0.15.0/src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java) for
+See [code](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java) for
other available Mesos replicated log configuration options and default values.
### Changing the Quorum Size
@@ -90,196 +91,77 @@ or truncating of the replicated log used
Configuration options for the Aurora scheduler backup manager.
-### `-backup_interval`
-The interval on which the scheduler writes local storage backups. The default is every hour.
+* `-backup_interval`: The interval on which the scheduler writes local storage backups. The default is every hour.
+* `-backup_dir`: Directory to write backups to.
+* `-max_saved_backups`: Maximum number of backups to retain before deleting the oldest backup(s).
-### `-backup_dir`
-Directory to write backups to.
-### `-max_saved_backups`
-Maximum number of backups to retain before deleting the oldest backup(s).
+## Resource Isolation
+For proper CPU, memory, and disk isolation as mentioned in our [enduser documentation](../../features/resource-isolation/),
+we recommend to add the following isolators to the `--isolation` flag of the Mesos agent:
-## Process Logs
+* `cgroups/cpu`
+* `cgroups/mem`
+* `disk/du`
-### Log destination
-By default, Thermos will write process stdout/stderr to log files in the sandbox. Process object configuration
-allows specifying alternate log file destinations like streamed stdout/stderr or suppression of all log output.
-Default behavior can be configured for the entire cluster with the following flag (through the `-thermos_executor_flags`
-argument to the Aurora scheduler):
+In addition, we recommend to set the following [agent flags](http://mesos.apache.org/documentation/latest/configuration/):
- --runner-logger-destination=both
+* `--cgroups_limit_swap` to enable memory limits on both memory and swap instead of just memory.
+ Alternatively, you could disable swap on your agent hosts.
+* `--cgroups_enable_cfs` to enable hard limits on CPU resources via the CFS bandwidth limiting
+ feature.
+* `--enforce_container_disk_quota` to enable disk quota enforcement for containers.
-`both` configuration will send logs to files and stream to parent stdout/stderr outputs.
+To enable the optional GPU support in Mesos, please see the GPU related flags in the
+[Mesos configuration](http://mesos.apache.org/documentation/latest/configuration/).
+To enable the corresponding feature in Aurora, you have to start the scheduler with the
+flag
-See [Configuration Reference](../../reference/configuration/#logger) for all destination options.
+ -allow_gpu_resource=true
-### Log rotation
-By default, Thermos will not rotate the stdout/stderr logs from child processes and they will grow
-without bound. An individual user may change this behavior via configuration on the Process object,
-but it may also be desirable to change the default configuration for the entire cluster.
-In order to enable rotation by default, the following flags can be applied to Thermos (through the
--thermos_executor_flags argument to the Aurora scheduler):
+If you want to use revocable resources, first follow the
+[Mesos oversubscription documentation](http://mesos.apache.org/documentation/latest/oversubscription/)
+and then set set this Aurora scheduler flag to allow receiving revocable Mesos offers:
- --runner-logger-mode=rotate
- --runner-rotate-log-size-mb=100
- --runner-rotate-log-backups=10
+ -receive_revocable_resources=true
-In the above example, each instance of the Thermos runner will rotate stderr/stdout logs once they
-reach 100 MiB in size and keep a maximum of 10 backups. If a user has provided a custom setting for
-their process, it will override these default settings.
+Both CPUs and RAM are supported as revocable resources. The former is enabled by the default,
+the latter needs to be enabled via:
+ -enable_revocable_ram=true
+Unless you want to use the [default](https://github.com/apache/aurora/blob/rel/0.16.0/src/main/resources/org/apache/aurora/scheduler/tiers.json)
+tier configuration, you will also have to specify a file path:
-## Thermos Executor Wrapper
+ -tier_config=path/to/tiers/config.json
-If you need to do computation before starting the thermos executor (for example, setting a different
-`--announcer-hostname` parameter for every executor), then the thermos executor should be invoked
- inside a wrapper script. In such a case, the aurora scheduler should be started with
- `-thermos_executor_path` pointing to the wrapper script and `-thermos_executor_resources`
- set to a comma separated string of all the resources that should be copied into
- the sandbox (including the original thermos executor).
-For example, to wrap the executor inside a simple wrapper, the scheduler will be started like this
-`-thermos_executor_path=/path/to/wrapper.sh -thermos_executor_resources=/usr/share/aurora/bin/thermos_executor.pex`
+## Containers
-## Custom Executor
+Both the Mesos and Docker containerizers require configuration of the Mesos agent.
-If the need arises to use a Mesos executor other than the Thermos executor, the scheduler can be
-configured to utilize a custom executor by specifying the `-custom_executor_config` flag.
-The flag must be set to the path of a valid executor configuration file.
-
-The configuration file must be valid JSON and contain, at minimum, the name, command and resources fields.
-
-
-### executor
-
-Property | Description
------------------------ | ---------------------------------
-name (required) | Name of the executor.
-command (required) | How to run the executor.
-resources (required) | Overhead to use for each executor instance.
-
-#### command
-
-Property | Description
------------------------ | ---------------------------------
-value (required) | The command to execute.
-arguments (optional) | A list of arguments to pass to the command.
-uris (optional) | List of resources to download into the task sandbox.
-
-##### uris (list)
-* Follows the [Mesos Fetcher schema](http://mesos.apache.org/documentation/latest/fetcher/)
-
-Property | Description
------------------------ | ---------------------------------
-value (required) | Path to the resource needed in the sandbox.
-executable (optional) | Change resource to be executable via chmod.
-extract (optional) | Extract files from packed or compressed archives into the sandbox.
-cache (optional) | Use caching mechanism provided by Mesos for resources.
-
-#### resources (list)
-
-Property | Description
-------------------- | ---------------------------------
-name (required) | Name of the resource: cpus or mem.
-type (required) | Type of resource. Should always be SCALAR.
-scalar (required) | Value in float for cpus or int for mem (in MBs)
-
-### volume_mounts (list)
-
-Property | Description
------------------------- | ---------------------------------
-host_path (required) | Host path to mount inside the container.
-container_path (required) | Path inside the container where `host_path` will be mounted.
-mode (required) | Mode in which to mount the volume, Read-Write (RW) or Read-Only (RO).
-
-
-A sample configuration is as follows:
-```
- {
- "executor": {
- "name": "myExecutor",
- "command": {
- "value": "myExecutor.sh",
- "arguments": [
- "localhost:2181",
- "-verbose",
- "-config myConfiguration.config"
- ],
- "uris": [
- {
- "value": "/dist/myExecutor.sh",
- "executable": true,
- "extract": false,
- "cache": true
- },
- {
- "value": "/home/user/myConfiguration.config",
- "executable": false,
- "extract": false,
- "cache": false
- }
- ]
- },
- "resources": [
- {
- "name": "cpus",
- "type": "SCALAR",
- "scalar": {
- "value": 1.00
- }
- },
- {
- "name": "mem",
- "type": "SCALAR",
- "scalar": {
- "value": 512
- }
- }
- ]
- },
- "volume_mounts": [
- {
- "mode": "RO",
- "container_path": "/path/on/container",
- "host_path": "/path/to/host/directory"
- },
- {
- "mode": "RW",
- "container_path": "/container",
- "host_path": "/host"
- }
- ]
- }
-```
-
-It should be noted that if you do not use thermos or a thermos based executor, links in the scheduler's
-Web UI for tasks
will not work (at least for the time being).
-Some information about launched tasks can still be accessed via the Mesos Web UI or via the Aurora Client.
-Furthermore, this configuration replaces the default thermos executor.
-Work is in progress to allow support for multiple executors to co-exist within a single scheduler.
-
-### Docker containers
-In order for Aurora to launch jobs using docker containers, a few extra configuration options
-must be set. The [docker containerizer](http://mesos.apache.org/documentation/latest/docker-containerizer/)
-must be enabled on the Mesos agents by launching them with the `--containerizers=docker,mesos` option.
-
-By default, Aurora will configure Mesos to copy the file specified in `-thermos_executor_path`
-into the container's sandbox. If using a wrapper script to launch the thermos executor,
-specify the path to the wrapper in that argument. In addition, the path to the executor pex itself
-must be included in the `-thermos_executor_resources` option. Doing so will ensure that both the
-wrapper script and executor are correctly copied into the sandbox. Finally, ensure the wrapper
-script does not access resources outside of the sandbox, as when the script is run from within a
-docker container those resources will not exist.
+### Mesos Containerizer
-A scheduler flag, `-global_container_mounts` allows mounting paths from the host (i.e the agent machine)
-into all containers on that host. The format is a comma separated list of host_path:container_path[:mode]
-tuples. For example `-global_container_mounts=/opt/secret_keys_dir:/mnt/secret_keys_dir:ro` mounts
-`/opt/secret_keys_dir` from the agents into all launched containers. Valid modes are `ro` and `rw`.
+The minimal agent configuration requires to enable Docker and Appc image support for the Mesos
+containerizer:
+
+ --containerizers=mesos
+ --image_providers=appc,docker
+ --isolation=filesystem/linux,docker/runtime # as an addition to your other isolators
+
+Further details can be found in the corresponding [Mesos documentation](http://mesos.apache.org/documentation/latest/container-image/).
+
+### Docker Containerizer
+
+The [Docker containerizer](http://mesos.apache.org/documentation/latest/docker-containerizer/)
+requires the Docker engine is installed on each agent host. In addition, it must be enabled on the
+Mesos agents by launching them with the option:
+
+ --containerizers=mesos,docker
-If you would like to run a container with a read-only filesystem, it may also be necessary to
-pass to use the scheduler flag `-thermos_home_in_sandbox` in order to set HOME to the sandbox
+If you would like to run a container with a read-only filesystem, it may also be necessary to use
+the scheduler flag `-thermos_home_in_sandbox` in order to set HOME to the sandbox
before the executor runs. This will make sure that the executor/runner PEX extractions happens
inside of the sandbox instead of the container filesystem root.
@@ -300,3 +182,70 @@ should be used for every container that
level. The argument accepts a multimap format:
-default_docker_parameters="read-only=true,tmpfs=/tmp,tmpfs=/run"
+
+### Common Options
+
+The following Aurora options work for both containerizers.
+
+A scheduler flag, `-global_container_mounts` allows mounting paths from the host (i.e the agent machine)
+into all containers on that host. The format is a comma separated list of host_path:container_path[:mode]
+tuples. For example `-global_container_mounts=/opt/secret_keys_dir:/mnt/secret_keys_dir:ro` mounts
+`/opt/secret_keys_dir` from the agents into all launched containers. Valid modes are `ro` and `rw`.
+
+
+## Thermos Process Logs
+
+### Log destination
+By default, Thermos will write process stdout/stderr to log files in the sandbox. Process object
+configuration allows specifying alternate log file destinations like streamed stdout/stderr or
+suppression of all log output. Default behavior can be configured for the entire cluster with the
+following flag (through the `-thermos_executor_flags` argument to the Aurora scheduler):
+
+ --runner-logger-destination=both
+
+`both` configuration will send logs to files and stream to parent stdout/stderr outputs.
+
+See [Configuration Reference](../../reference/configuration/#logger) for all destination options.
+
+### Log rotation
+By default, Thermos will not rotate the stdout/stderr logs from child processes and they will grow
+without bound. An individual user may change this behavior via configuration on the Process object,
+but it may also be desirable to change the default configuration for the entire cluster.
+In order to enable rotation by default, the following flags can be applied to Thermos (through the
+`-thermos_executor_flags` argument to the Aurora scheduler):
+
+ --runner-logger-mode=rotate
+ --runner-rotate-log-size-mb=100
+ --runner-rotate-log-backups=10
+
+In the above example, each instance of the Thermos runner will rotate stderr/stdout logs once they
+reach 100 MiB in size and keep a maximum of 10 backups. If a user has provided a custom setting for
+their process, it will override these default settings.
+
+
+## Thermos Executor Wrapper
+
+If you need to do computation before starting the Thermos executor (for example, setting a different
+`--announcer-hostname` parameter for every executor), then the Thermos executor should be invoked
+inside a wrapper script. In such a case, the aurora scheduler should be started with
+`-thermos_executor_path` pointing to the wrapper script and `-thermos_executor_resources` set to a
+comma separated string of all the resources that should be copied into the sandbox (including the
+original Thermos executor). Ensure the wrapper script does not access resources outside of the
+sandbox, as when the script is run from within a Docker container those resources may not exist.
+
+For example, to wrap the executor inside a simple wrapper, the scheduler will be started like this
+`-thermos_executor_path=/path/to/wrapper.sh -thermos_executor_resources=/usr/share/aurora/bin/thermos_executor.pex`
+
+## Custom Executors
+
+The scheduler can be configured to utilize a custom executor by specifying the `-custom_executor_config` flag.
+The flag must be set to the path of a valid executor configuration file.
+
+For more information on this feature please see the custom executors [documentation](../../features/custom-executors/).
+
+## A note on increasing executor overhead
+
+Increasing executor overhead on an existing cluster, whether it be for custom executors or for Thermos,
+will result in degraded preemption performance until all task which began life with the previous
+executor configuration with less overhead are preempted/restarted.
+
Modified: aurora/site/source/documentation/latest/operations/installation.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/operations/installation.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/operations/installation.md (original)
+++ aurora/site/source/documentation/latest/operations/installation.md Wed Sep 28 18:23:53 2016
@@ -61,8 +61,8 @@ Any machines that users submit jobs from
sudo update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
- wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-scheduler_0.12.0_amd64.deb
- sudo dpkg -i aurora-scheduler_0.12.0_amd64.deb
+ wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-scheduler_0.15.0_amd64.deb
+ sudo dpkg -i aurora-scheduler_0.15.0_amd64.deb
### CentOS 7
@@ -83,8 +83,8 @@ Any machines that users submit jobs from
sudo yum install -y wget
- wget -c https://apache.bintray.com/aurora/centos-7/aurora-scheduler-0.12.0-1.el7.centos.aurora.x86_64.rpm
- sudo yum install -y aurora-scheduler-0.12.0-1.el7.centos.aurora.x86_64.rpm
+ wget -c https://apache.bintray.com/aurora/centos-7/aurora-scheduler-0.15.0-1.el7.centos.aurora.x86_64.rpm
+ sudo yum install -y aurora-scheduler-0.15.0-1.el7.centos.aurora.x86_64.rpm
### Finalizing
By default, the scheduler will start in an uninitialized mode. This is because external
@@ -123,8 +123,8 @@ CentOS: `sudo systemctl start aurora`
# for the python mesos native bindings.
sudo apt-get -y install libcurl4-nss-dev
- wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-executor_0.12.0_amd64.deb
- sudo dpkg -i aurora-executor_0.12.0_amd64.deb
+ wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-executor_0.15.0_amd64.deb
+ sudo dpkg -i aurora-executor_0.15.0_amd64.deb
### CentOS 7
@@ -137,8 +137,8 @@ CentOS: `sudo systemctl start aurora`
sudo yum install -y python2 wget
- wget -c https://apache.bintray.com/aurora/centos-7/aurora-executor-0.12.0-1.el7.centos.aurora.x86_64.rpm
- sudo yum install -y aurora-executor-0.12.0-1.el7.centos.aurora.x86_64.rpm
+ wget -c https://apache.bintray.com/aurora/centos-7/aurora-executor-0.15.0-1.el7.centos.aurora.x86_64.rpm
+ sudo yum install -y aurora-executor-0.15.0-1.el7.centos.aurora.x86_64.rpm
### Configuration
The executor typically does not require configuration. Command line arguments can
@@ -182,18 +182,6 @@ Aurora observer start scripts and restar
sudo sh -c 'echo "MESOS_ROOT=/tmp/mesos" >> /etc/default/thermos'
-NB: In Aurora releases up through 0.12.0, you'll also need to edit /etc/init/thermos.conf like so:
-
- diff -C 1 /etc/init/thermos.conf.orig /etc/init/thermos.conf
- *** /etc/init/thermos.conf.orig 2016-03-22 22:34:46.286199718 +0000
- --- /etc/init/thermos.conf 2016-03-22 17:09:49.357689038 +0000
- ***************
- *** 24,25 ****
- --- 24,26 ----
- --port=${OBSERVER_PORT:-1338} \
- + --mesos-root=${MESOS_ROOT:-/var/lib/mesos} \
- --log_to_disk=NONE \
-
#### CentOS 7
Make an edit to add the `--mesos-root` flag resulting in something like:
@@ -211,15 +199,15 @@ Make an edit to add the `--mesos-root` f
sudo apt-get install -y python2.7 wget
- wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-tools_0.12.0_amd64.deb
- sudo dpkg -i aurora-tools_0.12.0_amd64.deb
+ wget -c https://apache.bintray.com/aurora/ubuntu-trusty/aurora-tools_0.15.0_amd64.deb
+ sudo dpkg -i aurora-tools_0.15.0_amd64.deb
### CentOS 7
sudo yum install -y python2 wget
- wget -c https://apache.bintray.com/aurora/centos-7/aurora-tools-0.12.0-1.el7.centos.aurora.x86_64.rpm
- sudo yum install -y aurora-tools-0.12.0-1.el7.centos.aurora.x86_64.rpm
+ wget -c https://apache.bintray.com/aurora/centos-7/aurora-tools-0.15.0-1.el7.centos.aurora.x86_64.rpm
+ sudo yum install -y aurora-tools-0.15.0-1.el7.centos.aurora.x86_64.rpm
### Mac OS X
@@ -251,12 +239,12 @@ are identical for both.
sudo apt-get -y update
# Use `apt-cache showpkg mesos | grep [version]` to find the exact version.
- sudo apt-get -y install mesos=0.25.0-0.2.70.ubuntu1404
+ sudo apt-get -y install mesos=0.28.2-2.0.27.ubuntu1404_amd64
### Mesos on CentOS 7
sudo rpm -Uvh https://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
- sudo yum -y install mesos-0.25.0
+ sudo yum -y install mesos-0.28.2
Modified: aurora/site/source/documentation/latest/reference/configuration.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/reference/configuration.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/reference/configuration.md (original)
+++ aurora/site/source/documentation/latest/reference/configuration.md Wed Sep 28 18:23:53 2016
@@ -121,30 +121,27 @@ schedule.
#### logger
-The default behavior of Thermos is to store stderr/stdout logs in files which grow unbounded.
-In the event that you have large log volume, you may want to configure Thermos to automatically rotate logs
-after they grow to a certain size, which can prevent your job from using more than its allocated
-disk space.
-
-A Logger union consists of a destination enum, a mode enum and a rotation policy.
-It's to set where the process logs should be sent using `destination`. Default
-option is `file`. Its also possible to specify `console` to get logs output
-to stdout/stderr, `none` to suppress any logs output or `both` to send logs to files and
-console output. In case of using `none` or `console` rotation attributes are ignored.
-Rotation policies only apply to loggers whose mode is `rotate`. The acceptable values
-for the LoggerMode enum are `standard` and `rotate`. The rotation policy applies to both
-stderr and stdout.
+The default behavior of Thermos is to store stderr/stdout logs in files which grow unbounded.
+In the event that you have large log volume, you may want to configure Thermos to automatically
+rotate logs after they grow to a certain size, which can prevent your job from using more than its
+allocated disk space.
+
+Logger objects specify a `destination` for Process logs which is, by default, `file` - a pair of
+`stdout` and `stderr` files. Its also possible to specify `console` to get logs output to
+the Process stdout and stderr streams, `none` to suppress any logs output or `both` to send logs to
+files and console streams.
-By default, all processes use the `standard` LoggerMode.
+The default Logger `mode` is `standard` which lets the stdout and stderr streams grow without bound.
**Attribute Name** | **Type** | **Description**
------------------- | :---------------: | ---------------------------------
**destination** | LoggerDestination | Destination of logs. (Default: `file`)
**mode** | LoggerMode | Mode of the logger. (Default: `standard`)
- **rotate** | RotatePolicy | An optional rotation policy.
+ **rotate** | RotatePolicy | An optional rotation policy. (Default: `Empty`)
-A RotatePolicy describes log rotation behavior for when `mode` is set to `rotate`. It is ignored
-otherwise.
+A RotatePolicy describes log rotation behavior for when `mode` is set to `rotate` and it is ignored
+otherwise. If `rotate` is `Empty` or `RotatePolicy()` when the `mode` is set to `rotate` the
+defaults below are used.
**Attribute Name** | **Type** | **Description**
------------------- | :----------: | ---------------------------------
@@ -333,6 +330,9 @@ Job Schema
deprecated in favor of setting its value directly to the appropriate ```Docker``` or ```Mesos```
container type*
+*Note: Specifying preemption behavior of tasks through `production` flag is deprecated in favor of
+ electing appropriate task tier via `tier` attribute.*
+
name | type | description
------ | :-------: | -------
```task``` | Task | The Task object to bind to this job. Required.
@@ -349,11 +349,11 @@ Job Schema
```service``` | Boolean | If True, restart tasks regardless of success or failure. (Default: False)
```max_task_failures``` | Integer | Maximum number of failures after which the task is considered to have failed (Default: 1) Set to -1 to allow for infinite failures
```priority``` | Integer | Preemption priority to give the task (Default 0). Tasks with higher priorities may preempt tasks at lower priorities.
- ```production``` | Boolean | Whether or not this is a production task that may [preempt](../../features/multitenancy/#preemption) other tasks (Default: False). Production job role must have the appropriate [quota](../../features/multitenancy/#preemption).
+ ```production``` | Boolean | (Deprecated) Whether or not this is a production task that may [preempt](../../features/multitenancy/#preemption) other tasks (Default: False). Production job role must have the appropriate [quota](../../features/multitenancy/#preemption).
```health_check_config``` | ```HealthCheckConfig``` object | Parameters for controlling a task's health checks. HTTP health check is only used if a health port was assigned with a command line wildcard.
```container``` | Choice of ```Container```, ```Docker``` or ```Mesos``` object | An optional container to run all processes inside of.
```lifecycle``` | ```LifecycleConfig``` object | An optional task lifecycle configuration that dictates commands to be executed on startup/teardown. HTTP lifecycle is enabled by default if the "health" port is requested. See [LifecycleConfig Objects](#lifecycleconfig-objects) for more information.
- ```tier``` | String | Task tier type. The default scheduler tier configuration allows for 3 tiers: `revocable`, `preemptible`, and `preferred`. The `revocable` tier requires the task to run with Mesos revocable resources. Setting the task's tier to `preemptible` allows for the possibility of that task being preempted by other tasks when cluster is running low on resources. The `preferred` tier prevents the task from using revocable resources and from being preempted. Since it is possible that a cluster is configured with a custom tier configuration, users should consult their cluster administrator to be informed of the tiers supported by the cluster. Attempts to schedule jobs with an unsupported tier will be rejected by the scheduler.
+ ```tier``` | String | Task tier type. The default scheduler tier configuration allows for 3 tiers: `revocable`, `preemptible`, and `preferred`. If a tier is not elected, Aurora assigns the task to a tier based on its choice of `production` (that is `preferred` for production and `preemptible` for non-production jobs). See the section on [Configuration Tiers](../../features/multitenancy/#configuration-tiers) for more information.
```announce``` | ```Announcer``` object | Optionally enable Zookeeper ServerSet announcements. See [Announcer Objects] for more information.
```enable_hooks``` | Boolean | Whether to enable [Client Hooks](../client-hooks/) for this job. (Default: False)
@@ -451,37 +451,16 @@ tasks with the same static port allocati
External constraints such as agent attributes should be used to enforce such
guarantees should they be needed.
-### Container Objects
-
-*Note: Both Docker and Mesos unified-container support are currently EXPERIMENTAL.*
-*Note: In order to correctly execute processes inside a job, the Docker container must have python 2.7 installed.*
-*Note: For private docker registry, mesos mandates the docker credential file to be named as `.dockercfg`, even though docker may create a credential file with a different name on various platforms. Also, the `.dockercfg` file needs to be copied into the sandbox using the `-thermos_executor_resources` flag, specified while starting Aurora.*
+### Container Objects
Describes the container the job's processes will run inside. If not using Docker or the Mesos
unified-container, the container can be omitted from your job config.
param | type | description
----- | :----: | -----------
- ```docker``` | Docker | A docker container to use.
- ```mesos``` | Mesos | A mesos container to use.
-
-### Docker Object
-
- param | type | description
- ----- | :----: | -----------
- ```image``` | String | The name of the docker image to execute. If the image does not exist locally it will be pulled with ```docker pull```.
- ```parameters``` | List(Parameter) | Additional parameters to pass to the docker containerizer.
-
-### Docker Parameter Object
-
-Docker CLI parameters. This needs to be enabled by the scheduler `allow_docker_parameters` option.
-See [Docker Command Line Reference](https://docs.docker.com/reference/commandline/run/) for valid parameters.
-
- param | type | description
- ----- | :----: | -----------
- ```name``` | String | The name of the docker parameter. E.g. volume
- ```value``` | String | The value of the parameter. E.g. /usr/local/bin:/usr/bin:rw
+ ```mesos``` | Mesos | A native Mesos container to use.
+ ```docker``` | Docker | A Docker container to use (via Docker engine)
### Mesos Object
@@ -491,8 +470,6 @@ See [Docker Command Line Reference](http
### AppcImage
-*Note: In order to correctly execute processes inside a job, the filesystem image must include python 2.7.*
-
Describes an AppC filesystem image.
param | type | description
@@ -502,8 +479,6 @@ Describes an AppC filesystem image.
### DockerImage
-*Note: In order to correctly execute processes inside a job, the filesystem image must include python 2.7.*
-
Describes a Docker filesystem image.
param | type | description
@@ -511,6 +486,28 @@ Describes a Docker filesystem image.
```name``` | String | The name of the docker image.
```tag``` | String | The tag that identifies the docker image.
+
+### Docker Object
+
+*Note: In order to correctly execute processes inside a job, the Docker container must have Python 2.7 installed.*
+*Note: For private docker registry, mesos mandates the docker credential file to be named as `.dockercfg`, even though docker may create a credential file with a different name on various platforms. Also, the `.dockercfg` file needs to be copied into the sandbox using the `-thermos_executor_resources` flag, specified while starting Aurora.*
+
+ param | type | description
+ ----- | :----: | -----------
+ ```image``` | String | The name of the docker image to execute. If the image does not exist locally it will be pulled with ```docker pull```.
+ ```parameters``` | List(Parameter) | Additional parameters to pass to the Docker engine.
+
+### Docker Parameter Object
+
+Docker CLI parameters. This needs to be enabled by the scheduler `-allow_docker_parameters` option.
+See [Docker Command Line Reference](https://docs.docker.com/reference/commandline/run/) for valid parameters.
+
+ param | type | description
+ ----- | :----: | -----------
+ ```name``` | String | The name of the docker parameter. E.g. volume
+ ```value``` | String | The value of the parameter. E.g. /usr/local/bin:/usr/bin:rw
+
+
### LifecycleConfig Objects
*Note: The only lifecycle configuration supported is the HTTP lifecycle via the HttpLifecycleConfig.*
Modified: aurora/site/source/documentation/latest/reference/scheduler-configuration.md
URL: http://svn.apache.org/viewvc/aurora/site/source/documentation/latest/reference/scheduler-configuration.md?rev=1762695&r1=1762694&r2=1762695&view=diff
==============================================================================
--- aurora/site/source/documentation/latest/reference/scheduler-configuration.md (original)
+++ aurora/site/source/documentation/latest/reference/scheduler-configuration.md Wed Sep 28 18:23:53 2016
@@ -22,6 +22,8 @@ Required flags:
Max number of idle connections to the database via MyBatis
-framework_authentication_file
Properties file which contains framework credentials to authenticate with Mesosmaster. Must contain the properties 'aurora_authentication_principal' and 'aurora_authentication_secret'.
+-ip
+ The ip address to listen. If not set, the scheduler will listen on all interfaces.
-mesos_master_address [not null]
Address for the mesos master, can be a socket address or zookeeper path.
-mesos_role
@@ -34,12 +36,16 @@ Required flags:
Path to the thermos executor entry point.
-tier_config [file must be readable]
Configuration file defining supported task tiers, task traits and behaviors.
+-webhook_config [file must exist, file must be readable]
+ Path to webhook configuration file.
-zk_endpoints [must have at least 1 item]
Endpoint specification for the ZooKeeper servers.
Optional flags:
-allow_docker_parameters (default false)
Allow to pass docker container parameters in the job.
+-allow_gpu_resource (default false)
+ Allow jobs to request Mesos GPU resource.
-allowed_container_types (default [MESOS])
Container types that are allowed to be used by jobs.
-async_slot_stat_update_interval (default (1, mins))
@@ -76,10 +82,16 @@ Optional flags:
List of domains for which CORS support should be enabled.
-enable_h2_console (default false)
Enable H2 DB management console.
+-enable_mesos_fetcher (default false)
+ Allow jobs to pass URIs to the Mesos Fetcher. Note that enabling this feature could pose a privilege escalation threat.
-enable_preemptor (default true)
Enable the preemptor and preemption
+-enable_revocable_cpus (default true)
+ Treat CPUs as a revocable resource.
+-enable_revocable_ram (default false)
+ Treat RAM as a revocable resource.
-executor_user (default root)
- User to start the executor. Defaults to "root". Set this to an unprivileged user if the mesos master was started with "--no-root_submissions". If set to anything other than "root", the executor will ignore the "role" setting for jobs since it can't use setuid() anymore. This means that all your jobs will run under the specified user and the user has to exist on the mesos slaves.
+ User to start the executor. Defaults to "root". Set this to an unprivileged user if the mesos master was started with "--no-root_submissions". If set to anything other than "root", the executor will ignore the "role" setting for jobs since it can't use setuid() anymore. This means that all your jobs will run under the specified user and the user has to exist on the Mesos agents.
-first_schedule_delay (default (1, ms))
Initial amount of time to wait before first attempting to schedule a PENDING task.
-flapping_task_threshold (default (5, mins))
@@ -163,7 +175,7 @@ Optional flags:
-offer_hold_jitter_window (default (1, mins))
Maximum amount of random jitter to add to the offer hold time window.
-offer_reservation_duration (default (3, mins))
- Time to reserve a slave's offers while trying to satisfy a task preempting another.
+ Time to reserve a agent's offers while trying to satisfy a task preempting another.
-populate_discovery_info (default false)
If true, Aurora populates DiscoveryInfo field of Mesos TaskInfo.
-preemption_delay (default (3, mins))
@@ -174,6 +186,10 @@ Optional flags:
Time interval between pending task preemption slot searches.
-receive_revocable_resources (default false)
Allows receiving revocable resource offers from Mesos.
+-reconciliation_explicit_batch_interval (default (5, secs))
+ Interval between explicit batch reconciliation requests.
+-reconciliation_explicit_batch_size (default 1000) [must be > 0]
+ Number of tasks in a single batch request sent to Mesos for explicit reconciliation.
-reconciliation_explicit_interval (default (60, mins))
Interval on which scheduler will ask Mesos for status updates of all non-terminal tasks known to scheduler.
-reconciliation_implicit_interval (default (60, mins))
@@ -186,7 +202,7 @@ Optional flags:
If false, Docker tasks may run without an executor (EXPERIMENTAL)
-shiro_ini_path
Path to shiro.ini for authentication and authorization configuration.
--shiro_realm_modules (default [org.apache.aurora.scheduler.app.MoreModules$1@13c9d689])
+-shiro_realm_modules (default [org.apache.aurora.scheduler.app.MoreModules$1@158a8276])
Guice modules for configuring Shiro Realms.
-sla_non_prod_metrics (default [])
Metric categories collected for non production tasks.
@@ -218,8 +234,6 @@ Optional flags:
Whether to use the experimental database-backed task store.
-viz_job_url_prefix (default )
URL prefix for job container stats.
--webhook_config [file must be readable]
- File to configure a HTTP webhook to receive task state change events.
-zk_chroot_path
chroot path to use for the ZooKeeper connections
-zk_digest_credentials
@@ -228,7 +242,7 @@ Optional flags:
Launches an embedded zookeeper server for local testing causing -zk_endpoints to be ignored if specified.
-zk_session_timeout (default (4, secs))
The ZooKeeper session timeout.
--zk_use_curator (default false)
- Uses Apache Curator as the zookeeper client; otherwise a copy of Twitter commons/zookeeper (the legacy library) is used.
+-zk_use_curator (default true)
+ DEPRECATED: Uses Apache Curator as the zookeeper client; otherwise a copy of Twitter commons/zookeeper (the legacy library) is used.
-------------------------------------------------------------------------
```