You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/03/01 12:10:00 UTC
[jira] [Commented] (AIRFLOW-3962) Scheduler throwing an exception
while creating dag_run for a dag which doesn't contain any task
[ https://issues.apache.org/jira/browse/AIRFLOW-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781610#comment-16781610 ]
ASF GitHub Bot commented on AIRFLOW-3962:
-----------------------------------------
ashb commented on pull request #4781: [AIRFLOW-3962] Added graceful handling for creation of dag_run of a dag which doesn't have any task
URL: https://github.com/apache/airflow/pull/4781
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Scheduler throwing an exception while creating dag_run for a dag which doesn't contain any task
> -----------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3962
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3962
> Project: Apache Airflow
> Issue Type: Bug
> Affects Versions: 1.10.2
> Reporter: Vardan Gupta
> Assignee: Vardan Gupta
> Priority: Major
>
> Steps to Reproduce
> ---- * Copy a python dag file which doesn't contain in tasks(Sample python code available at the bottom)
> * Make sure dag is not paused
> * During create_dag_run method call of schedule the time of file processing, task_start_dates list will come out to be empty [here|https://github.com/apache/airflow/blob/f3dfc9495be41221bd08ea753555df0975deb961/airflow/jobs.py#L840], due to this *_next_run_date_* will remain None
> * Above will cause *_next_run_date > timezone.utcnow()_* comparison [here|https://github.com/apache/airflow/blob/f3dfc9495be41221bd08ea753555df0975deb961/airflow/jobs.py#L869] to fail and resulting in below exception
> {code:java}
> Traceback (most recent call last):
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 398, in helper
> pickle_dags)
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 69, in wrapper
> result = func(*args, **kwargs)
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1749, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1438, in _process_dags
> dag_run = self.create_dag_run(dag)
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 69, in wrapper
> result = func(*args, **kwargs)
> File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 869, in create_dag_run
> if next_run_date > timezone.utcnow():
> TypeError: can't compare datetime.datetime to NoneType{code}
>
> Sample Python file without any task
>
> {code:java}
> import airflow
> from airflow import DAG
> from airflow.utils.trigger_rule import TriggerRule
> from datetime import datetime, timedelta
> import dateutil.parser
> # Add task level imports
>
> default_global_args = {
> 'owner': 'owner1',
> 'email': ['no-reply@domain.com'],
> 'email_on_failure': False,
> 'email_on_retry': False,
> 'start_date': dateutil.parser.parse('2019-02-21T00:00:00.000+05:30', ignoretz=True),
> 'end_date': dateutil.parser.parse('2019-02-22T00:00:00.000+05:30', ignoretz=True)
> }
> dag = DAG(
> dag_id = 'edce5a655a5e73ff0a494113_mlexperiment03_34sik4',
> default_args = default_global_args,
> schedule_interval = '@once',
> )
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)