incubator-airflow git commit: [AIRFLOW-2415] Make airflow DAG templating render numbers

[fo...@apache.org]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master 357e46d27 -> ce9c7bbdf


[AIRFLOW-2415] Make airflow DAG templating render numbers

Currently, if you have an operator with a template
fields argument, that is a dictionary, e.g.:
template_fields = ([dict_args])

And you populate that dictionary with a field that
an integer in a DAG, e.g.:
...
dict_args = {'ds': '{{ ds }}', num_times: 5}
...

Then ariflow will give you the following error:
{base_task_runner.py:95} INFO - Subtask:
airflow.exceptions.AirflowException: Type '<type
'int'>' used for parameter 'dict_args[num_times]'
is not supported for templating

This fix aims to resolves that issue by
immediately resolving numbers without attempting
to template them

Closes #3410 from
ArgentFalcon/support_numeric_template_fields


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/ce9c7bbd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/ce9c7bbd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/ce9c7bbd

Branch: refs/heads/master
Commit: ce9c7bbdfcbc931029077f5564e60746dba05cfa
Parents: 357e46d
Author: Max Payton <mp...@lyft.com>
Authored: Thu May 24 10:03:31 2018 +0200
Committer: Fokko Driesprong <fo...@godatadriven.com>
Committed: Thu May 24 10:03:31 2018 +0200

----------------------------------------------------------------------
 airflow/models.py |  3 +++
 tests/models.py   | 16 +++++++++++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/ce9c7bbd/airflow/models.py
----------------------------------------------------------------------
diff --git a/airflow/models.py b/airflow/models.py
index bcd12fb..28138fb 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -41,6 +41,7 @@ import zipfile
 import jinja2
 import json
 import logging
+import numbers
 import os
 import pickle
 import re
@@ -2664,6 +2665,8 @@ class BaseOperator(LoggingMixin):
             result = jinja_env.from_string(content).render(**context)
         elif isinstance(content, (list, tuple)):
             result = [rt(attr, e, context) for e in content]
+        elif isinstance(content, numbers.Number):
+            result = content
         elif isinstance(content, dict):
             result = {
                 k: rt("{}[{}]".format(attr, k), v, context)

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/ce9c7bbd/tests/models.py
----------------------------------------------------------------------
diff --git a/tests/models.py b/tests/models.py
index 2bf61fe..6d70e8b 100644
--- a/tests/models.py
+++ b/tests/models.py
@@ -374,9 +374,23 @@ class DagTest(unittest.TestCase):
         result = task.render_template('', '{{ foo }}', dict())
         self.assertEqual(result, 'bar')
 
+    def test_render_template_numeric_field(self):
+        """ Tests if render_template from a field works,
+            if a custom filter was defined"""
+
+        dag = DAG('test-dag',
+                  start_date=DEFAULT_DATE,
+                  user_defined_macros=dict(foo='bar'))
+
+        with dag:
+            task = DummyOperator(task_id='op1')
+
+        result = task.render_template('', 1, dict())
+        self.assertEqual(result, 1)
+
     def test_user_defined_filters(self):
         def jinja_udf(name):
-            return 'Hello %s' %name
+            return 'Hello %s' % name
 
         dag = models.DAG('test-dag',
                          start_date=DEFAULT_DATE,