You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ar...@apache.org on 2016/10/17 16:47:08 UTC
[1/2] incubator-airflow git commit: [AIRFLOW-575] Clarify tutorial
and FAQ about `schedule_interval` always inheriting from DAG object
Repository: incubator-airflow
Updated Branches:
refs/heads/master 0235d59d0 -> 916f1eb2f
[AIRFLOW-575] Clarify tutorial and FAQ about `schedule_interval` always inheriting from DAG object
- Update the tutorial with a comment helping to explain the use of default_args and
include all the possible parameters in line
- Clarify in the FAQ the possibility of an unexpected default `schedule_interval`in case
airflow users mistakenly try to overwrite the default `schedule_interval` in a DAG's
`default_args` parameter
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/80d3c8d4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/80d3c8d4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/80d3c8d4
Branch: refs/heads/master
Commit: 80d3c8d461f1c95d173aa72a055737d8ad379ae1
Parents: 11ad53a
Author: lauralorenz <la...@gmail.com>
Authored: Tue Apr 19 17:03:46 2016 -0400
Committer: lauralorenz <la...@gmail.com>
Committed: Mon Oct 17 12:36:38 2016 -0400
----------------------------------------------------------------------
airflow/example_dags/tutorial.py | 14 ++++++++++++--
docs/faq.rst | 5 +++++
2 files changed, 17 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/80d3c8d4/airflow/example_dags/tutorial.py
----------------------------------------------------------------------
diff --git a/airflow/example_dags/tutorial.py b/airflow/example_dags/tutorial.py
index 9462463..e929389 100644
--- a/airflow/example_dags/tutorial.py
+++ b/airflow/example_dags/tutorial.py
@@ -10,6 +10,8 @@ from datetime import datetime, timedelta
seven_days_ago = datetime.combine(datetime.today() - timedelta(7),
datetime.min.time())
+# these args will get passed on to each operator
+# you can override them on a per-task basis during operator initialization
default_args = {
'owner': 'airflow',
'depends_on_past': False,
@@ -22,11 +24,19 @@ default_args = {
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
- # 'schedule_interval': timedelta(1),
# 'end_date': datetime(2016, 1, 1),
+ # 'wait_for_downstream': False,
+ # 'dag': dag,
+ # 'adhoc':False,
+ # 'sla': timedelta(hours=2),
+ # 'execution_timeout': timedelta(seconds=300),
+ # 'on_failure_callback': some_function,
+ # 'on_success_callback': some_other_function,
+ # 'on_retry_callback': another_function,
+ # 'trigger_rule': u'all_success'
}
-dag = DAG('tutorial', default_args=default_args)
+dag = DAG('tutorial', default_args=default_args, schedule_interval=timedelta(days=1))
# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/80d3c8d4/docs/faq.rst
----------------------------------------------------------------------
diff --git a/docs/faq.rst b/docs/faq.rst
index 6418dcb..b5b28af 100644
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -17,6 +17,11 @@ Here are some of the common causes:
- Is your ``start_date`` set properly? The Airflow scheduler triggers the
task soon after the ``start_date + scheduler_interval`` is passed.
+- Is your ``schedule_interval`` set properly? The default ``schedule_interval``
+ is one day (``datetime.timedelta(1)``). You must specify a different ``schedule_interval``
+ directly to the DAG object you instantiate, not as a ``default_param``, as task instances
+ do not override their parent DAG's ``schedule_interval``.
+
- Is your ``start_date`` beyond where you can see it in the UI? If you
set your it to some time say 3 months ago, you won't be able to see
it in the main view in the UI, but you should be able to see it in the
[2/2] incubator-airflow git commit: Merge pull request #1402 from
lauralorenz/schedule_interval_default_args_docs
Posted by ar...@apache.org.
Merge pull request #1402 from lauralorenz/schedule_interval_default_args_docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/916f1eb2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/916f1eb2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/916f1eb2
Branch: refs/heads/master
Commit: 916f1eb2feedae4f4d827466cfe91821ef30f885
Parents: 0235d59 80d3c8d
Author: Arthur Wiedmer <ar...@gmail.com>
Authored: Mon Oct 17 09:46:57 2016 -0700
Committer: Arthur Wiedmer <ar...@gmail.com>
Committed: Mon Oct 17 09:46:57 2016 -0700
----------------------------------------------------------------------
airflow/example_dags/tutorial.py | 14 ++++++++++++--
docs/faq.rst | 5 +++++
2 files changed, 17 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/916f1eb2/airflow/example_dags/tutorial.py
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/916f1eb2/docs/faq.rst
----------------------------------------------------------------------