incubator-airflow git commit: [AIRFLOW-342] Do not use amqp, rpc as result backend

[fo...@apache.org]

Repository: incubator-airflow
Updated Branches:
  refs/heads/master aa737a582 -> bdafb12f8


[AIRFLOW-342] Do not use amqp, rpc as result backend

amqp and rpc (and redis most likely) cannot store
results for tasks
long enough.

Closes #2830 from bolkedebruin/AIRFLOW-342


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/bdafb12f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/bdafb12f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/bdafb12f

Branch: refs/heads/master
Commit: bdafb12f8dd9e6fcf508b0df869f9cc45bb9593d
Parents: aa737a5
Author: Bolke de Bruin <bo...@xs4all.nl>
Authored: Tue Dec 5 10:14:50 2017 +0100
Committer: Fokko Driesprong <fo...@godatadriven.com>
Committed: Tue Dec 5 10:14:50 2017 +0100

----------------------------------------------------------------------
 airflow/config_templates/default_celery.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/bdafb12f/airflow/config_templates/default_celery.py
----------------------------------------------------------------------
diff --git a/airflow/config_templates/default_celery.py b/airflow/config_templates/default_celery.py
index 390e3ef..4f8c92d 100644
--- a/airflow/config_templates/default_celery.py
+++ b/airflow/config_templates/default_celery.py
@@ -18,6 +18,7 @@ from airflow.exceptions import AirflowConfigException, AirflowException
 from airflow import configuration
 from airflow.utils.log.logging_mixin import LoggingMixin
 
+log = LoggingMixin().log
 
 broker_transport_options = configuration.getsection('celery_broker_transport_options')
 if broker_transport_options is None:
@@ -41,7 +42,6 @@ celery_ssl_active = False
 try:
     celery_ssl_active = configuration.getboolean('celery', 'CELERY_SSL_ACTIVE')
 except AirflowConfigException as e:
-    log = LoggingMixin().log
     log.warning("Celery Executor will run without SSL")
 
 try:
@@ -60,3 +60,7 @@ except Exception as e:
                            'Please ensure you want to use '
                            'SSL and/or have all necessary certs and key ({}).'.format(e))
 
+result_backend = DEFAULT_CELERY_CONFIG['result_backend']
+if 'amqp' in result_backend or 'redis' in result_backend or 'rpc' in result_backend:
+    log.warning("You have configured a result_backend of %s, it is highly recommended "
+                "to use an alternative result_backend (i.e. a database).", result_backend)