You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by ke...@apache.org on 2014/10/17 02:45:35 UTC
git commit: Cron documentation.
Repository: incubator-aurora
Updated Branches:
refs/heads/master b9211e8db -> e339f18ca
Cron documentation.
Testing Done:
Pushed to Github.
Bugs closed: AURORA-440
Reviewed at https://reviews.apache.org/r/26836/
Project: http://git-wip-us.apache.org/repos/asf/incubator-aurora/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-aurora/commit/e339f18c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-aurora/tree/e339f18c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-aurora/diff/e339f18c
Branch: refs/heads/master
Commit: e339f18ca520562ae4de2bbb49f35e04c1a6f6c6
Parents: b9211e8
Author: Kevin Sweeney <ke...@apache.org>
Authored: Thu Oct 16 17:44:44 2014 -0700
Committer: Kevin Sweeney <ke...@apache.org>
Committed: Thu Oct 16 17:44:44 2014 -0700
----------------------------------------------------------------------
docs/configuration-reference.md | 4 +-
docs/cron-jobs.md | 130 +++++++++++++++++++++++++++++
examples/jobs/cron_hello_world.aurora | 14 ++++
3 files changed, 146 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/e339f18c/docs/configuration-reference.md
----------------------------------------------------------------------
diff --git a/docs/configuration-reference.md b/docs/configuration-reference.md
index 83d0d10..f3cb257 100644
--- a/docs/configuration-reference.md
+++ b/docs/configuration-reference.md
@@ -313,8 +313,8 @@ Job Schema
```environment``` | String | Job environment, default ```devel```. Must be one of ```prod```, ```devel```, ```test``` or ```staging<number>```.
```contact``` | String | Best email address to reach the owner of the job. For production jobs, this is usually a team mailing list.
```instances```| Integer | Number of instances (sometimes referred to as replicas or shards) of the task to create. (Default: 1)
- ```cron_schedule``` **(Present, but not supported and a no-op)** | String | UTC Cron schedule in cron format. May only be used with non-service jobs. Default: None (not a cron job.)
- ```cron_collision_policy``` **(Present, but not supported and a no-op)** | String | Policy to use when a cron job is triggered while a previous run is still active. KILL_EXISTING Kill the previous run, and schedule the new run CANCEL_NEW Let the previous run continue, and cancel the new run. RUN_OVERLAP Let the previous run continue, and schedule the new run. (Default: KILL_EXISTING)
+ ```cron_schedule``` | String | Cron schedule in cron format. May only be used with non-service jobs. See [Cron Jobs](cron-jobs.md) for more information. Default: None (not a cron job.)
+ ```cron_collision_policy``` | String | Policy to use when a cron job is triggered while a previous run is still active. KILL_EXISTING Kill the previous run, and schedule the new run CANCEL_NEW Let the previous run continue, and cancel the new run. (Default: KILL_EXISTING)
```update_config``` | ```update_config``` object | Parameters for controlling the rate and policy of rolling updates.
```constraints``` | dict | Scheduling constraints for the tasks. See the section on the [constraint specification language](#Specifying-Scheduling-Constraints)
```service``` | Boolean | If True, restart tasks regardless of success or failure. (Default: False)
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/e339f18c/docs/cron-jobs.md
----------------------------------------------------------------------
diff --git a/docs/cron-jobs.md b/docs/cron-jobs.md
new file mode 100644
index 0000000..7d04bcd
--- /dev/null
+++ b/docs/cron-jobs.md
@@ -0,0 +1,130 @@
+# Cron Jobs
+
+Aurora supports execution of scheduled jobs on a Mesos cluster using cron-style syntax.
+
+- [Overview](#overview)
+- [Collision Policies](#collision-policies)
+ - [KILL_EXISTING](#kill_existing)
+ - [CANCEL_NEW](#cancel_new)
+- [Failure recovery](#failure-recovery)
+- [Interacting with cron jobs via the Aurora CLI](#interacting-with-cron-jobs-via-the-aurora-cli)
+ - [cron schedule](#cron-schedule)
+ - [cron deschedule](#cron-deschedule)
+ - [cron start](#cron-start)
+ - [job killall, job restart, job kill](#job-killall-job-restart-job-kill)
+- [Technical Note About Syntax](#technical-note-about-syntax)
+- [Caveats](#caveats)
+ - [Failovers](#failovers)
+ - [Collision policy is best-effort](#collision-policy-is-best-effort)
+ - [Timezone Configuration](#timezone-configuration)
+
+## Overview
+
+A job is identified as a cron job by the presence of a
+`cron_schedule` attribute containing a cron-style schedule in the
+[`Job`](configuration-reference.md#job-objects) object. Examples of cron schedules
+include "every 5 minutes" (`*/5 * * * *`), "Fridays at 17:00" (`* 17 * * FRI`), and
+"the 1st and 15th day of the month at 03:00" (`0 3 1,15 *`).
+
+Example (available in the [Vagrant environment](vagrant.md)):
+
+ $ cat /vagrant/examples/job/cron_hello_world.aurora
+ # cron_hello_world.aurora
+ # A cron job that runs every 5 minutes.
+ jobs = [
+ Job(
+ cluster = 'devcluster',
+ role = 'www-data',
+ environment = 'test',
+ name = 'cron_hello_world',
+ cron_schedule = '*/5 * * * *',
+ task = SimpleTask(
+ 'cron_hello_world',
+ 'echo "Hello world from cron, the time is now $(date --rfc-822)"'),
+ ),
+ ]
+
+## Collision Policies
+
+The `cron_collision_policy` field specifies the scheduler's behavior when a new cron job is
+triggered while an older run hasn't finished. The scheduler has two policies available,
+[KILL_EXISTING](#kill_existing) and [CANCEL_NEW](#cancel_new).
+
+### KILL_EXISTING
+
+The default policy - on a collision the old instances are killed and a instances with the current
+configuration are started.
+
+### CANCEL_NEW
+
+On a collision the new run is cancelled.
+
+Note that the use of this flag is likely a code smell - interrupted cron jobs should be able
+to recover their progress on a subsequent invocation, otherwise they risk having their work queue
+grow faster than they can process it.
+
+## Failure recovery
+
+Unlike with services, which aurora will always re-execute regardless of exit status, instances of
+cron jobs retry according to the `max_task_failures` attribute of the
+[Task](configuration-reference.md#task-objects) object. To get "run-until-failure" semantics,
+set `max_task_failures` to `-1`.
+
+## Interacting with cron jobs via the Aurora CLI
+
+Most interaction with cron jobs takes place using the `cron` subcommand. See `aurora2 help cron`
+for up-to-date usage instructions.
+
+### cron schedule
+Schedules a new cron job on the Aurora cluster for later runs, or updates an existing job.
+
+ $ aurora2 cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora
+
+### cron deschedule
+Deschedules a cron job, preventing future runs but allowing current runs to complete.
+
+ $ aurora2 cron deschedule devcluster/www-data/test/cron_hello_world
+
+### cron start
+Start a cron job immediately, outside of its normal cron schedule.
+
+ $ aurora2 cron start devcluster/www-data/test/cron_hello_world
+
+### job killall, job restart, job kill
+Cron jobs create instances running on the cluster that you can interact with like normal Aurora
+tasks with `job kill` and `job restart`.
+
+## Technical Note About Syntax
+
+`cron_schedule` uses a restricted subset of BSD crontab syntax. While the
+execution engine currently uses Quartz, the schedule parsing is custom, a subset of FreeBSD
+[crontab(5)](http://www.freebsd.org/cgi/man.cgi?crontab(5)) syntax. See
+[the source](https://github.com/apache/incubator-aurora/blob/master/src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java#L106-L124)
+for details.
+
+## Caveats
+
+### Failovers
+No failover recovery. Aurora does not record the latest minute it fired
+triggers for across failovers. Therefore it's possible to miss triggers
+on failover. Note that this behavior may change in the future.
+
+It's necessary to sync time between schedulers with something like `ntpd`.
+Clock skew could cause double or missed triggers in the case of a failover.
+
+### Collision policy is best-effort
+Aurora aims to always have *at least one copy* of a given instance running at a time - it's
+an AP system, meaning it chooses Availability and Partition Tolerance at the expense of
+Consistency.
+
+If your collision policy was `CANCEL_NEW` and a task has terminated but
+Aurora has not noticed this Aurora will go ahead and create your new
+task.
+
+If your collision policy was `KILL_EXISTING` and a task was marked `LOST`
+but not yet GCed Aurora will go ahead and create your new task without
+attempting to kill the old one (outside the GC interval).
+
+### Timezone Configuration
+Cron timezone is configured indepdendently of JVM timezone with the `-cron_timezone` flag and
+defaults to UTC.
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/e339f18c/examples/jobs/cron_hello_world.aurora
----------------------------------------------------------------------
diff --git a/examples/jobs/cron_hello_world.aurora b/examples/jobs/cron_hello_world.aurora
new file mode 100644
index 0000000..f562df2
--- /dev/null
+++ b/examples/jobs/cron_hello_world.aurora
@@ -0,0 +1,14 @@
+# cron_hello_world.aurora
+# A cron job that runs every 5 minutes.
+jobs = [
+ Job(
+ cluster = 'devcluster',
+ role = 'www-data',
+ environment = 'test',
+ name = 'cron_hello_world',
+ cron_schedule = '*/5 * * * *',
+ task = SimpleTask(
+ 'cron_hello_world',
+ 'echo "Hello world from cron, the time is now $(date --rfc-822)"'),
+ ),
+]