You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Bert Desmet (JIRA)" <ji...@apache.org> on 2017/03/23 16:21:41 UTC
[jira] [Created] (AIRFLOW-1033) TypeError: can't compare
datetime.datetime to NoneType in prev_dagrun_dep.py
Bert Desmet created AIRFLOW-1033:
------------------------------------
Summary: TypeError: can't compare datetime.datetime to NoneType in prev_dagrun_dep.py
Key: AIRFLOW-1033
URL: https://issues.apache.org/jira/browse/AIRFLOW-1033
Project: Apache Airflow
Issue Type: Bug
Components: DagRun
Affects Versions: Airflow 1.8
Environment: Centos 7;
db: PostgreSQL 9.5
python version: 2.7
Installation via pip
Reporter: Bert Desmet
Priority: Critical
Attachments: test_dag.py, test_dag.py.log
Dear,
When starting a specific new dag we get the following error:
[2017-03-23 16:51:16,354] {jobs.py:354} DagFileProcessor908 ERROR - Got an exception! Propagating...
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/airflow/jobs.py", line 346, in helper
pickle_dags)
File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
result = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/airflow/jobs.py", line 1581, in process_file
self._process_dags(dagbag, dags, ti_keys_to_schedule)
File "/usr/lib/python2.7/site-packages/airflow/jobs.py", line 1174, in _process_dags
self._process_task_instances(dag, tis_out)
File "/usr/lib/python2.7/site-packages/airflow/jobs.py", line 905, in _process_task_instances
session=session):
File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
result = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1116, in are_dependencies_met
session=session):
File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1140, in get_failed_dep_statuses
dep_context):
File "/usr/lib/python2.7/site-packages/airflow/ti_deps/deps/base_ti_dep.py", line 94, in get_dep_statuses
for dep_status in self._get_dep_statuses(ti, session, dep_context):
File "/usr/lib/python2.7/site-packages/airflow/ti_deps/deps/prev_dagrun_dep.py", line 47, in _get_dep_statuses
if dag.previous_schedule(ti.execution_date) < ti.task.start_date:
TypeError: can't compare datetime.datetime to NoneType
I have added some debug code to the file 'prev_dagrun_dep.py:
dag = ti.task.dag
print 'Start dates:'
print 'previous_exection_date: %s'%(dag.previous_schedule(ti.execution_date))
print 'current start date: %s'%(ti.task.start_date)
if dag.catchup:
if dag.previous_schedule(ti.execution_date) < ti.task.start_date:
And this is the output I get:
Start dates:
previous_exection_date: None
current start date: 2017-03-19 00:00:00
I think it is normall that the previous_exection_date is null, since it is the first time this dag is being run. But why is the start_date of the dag important, and not the start date of the run?
I have the feeling the cause is the 'schedule_interval', which is set to None.
Please find an example and it's log file as an attachment to this mail.
Bert
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)