[jira] [Updated] (AIRFLOW-5863) Airflow dag with past end_date doesn't trigger schedule

["Alex Anto Navis Lawrence (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alex Anto Navis Lawrence updated AIRFLOW-5863:
----------------------------------------------
    Description: 
Hi all,

I have a airflow dag with the following configuration:
{code:java}
{
 'start_date': datetime(2017, 1, 1, 0, 0, 0)
 'end_date': datetime(2019, 6, 1, 0, 0, 0),
  schedule_interval='@yearly',
  catchup=True
}

{code}
For the above, 2 schedules are created and run.
{code:java}
=> 2017-01-01, 2018-01-01.{code}
It doesn't create for 2019-01-01, since I understand the scheduler schedules only after the schedule interval is completed. In this case 2019 is still not over, so it is not scheduled.

Since I have given the end_date which is in the past, shouldn't the scheduler understand the end_date is over and trigger that.

I know there is a solution where by adjusting the start_date to one year back and adjusting the date ranges(execution_date + 1 year) within the job can solve this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 schedules (2016, 2017, 2018) to get all the windows.

Is there anyone way to handle this or am I missing something ?. Any pointers would be really helpful.

 

  was:
Hi all,

I have a airflow dag with the following configuration:

{{}}
{code:java}
{
 'start_date': datetime(2017, 1, 1, 0, 0, 0)
 'end_date': datetime(2019, 6, 1, 0, 0, 0),
  schedule_interval='@yearly',
  catchup=True
}

{code}
{{}}

For the above, 2 schedules are created and run.

{{}}
{code:java}
=> 2017-01-01, 2018-01-01.{code}
It doesn't create for 2019-01-01, since I understand the scheduler schedules only after the schedule interval is completed. In this case 2019 is still not over, so it is not scheduled.

Since I have given the end_date which is in the past, shouldn't the scheduler understand the end_date is over and trigger that.

I know there is a solution where by adjusting the start_date to one year back and adjusting the date ranges(execution_date + 1 year) within the job can solve this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 schedules (2016, 2017, 2018) to get all the windows.

Is there anyone way to handle this or am I missing something ?. Any pointers would be really helpful.

{{}}

{{}}


> Airflow dag with past end_date doesn't trigger schedule
> -------------------------------------------------------
>
>                 Key: AIRFLOW-5863
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5863
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.4
>            Reporter: Alex Anto Navis Lawrence
>            Priority: Minor
>              Labels: beginner
>
> Hi all,
> I have a airflow dag with the following configuration:
> {code:java}
> {
>  'start_date': datetime(2017, 1, 1, 0, 0, 0)
>  'end_date': datetime(2019, 6, 1, 0, 0, 0),
>   schedule_interval='@yearly',
>   catchup=True
> }
> {code}
> For the above, 2 schedules are created and run.
> {code:java}
> => 2017-01-01, 2018-01-01.{code}
> It doesn't create for 2019-01-01, since I understand the scheduler schedules only after the schedule interval is completed. In this case 2019 is still not over, so it is not scheduled.
> Since I have given the end_date which is in the past, shouldn't the scheduler understand the end_date is over and trigger that.
> I know there is a solution where by adjusting the start_date to one year back and adjusting the date ranges(execution_date + 1 year) within the job can solve this issue. (e.g. schedule it from start_date: 2016-01-01). So we will have 3 schedules (2016, 2017, 2018) to get all the windows.
> Is there anyone way to handle this or am I missing something ?. Any pointers would be really helpful.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)