You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Víctor Portals (JIRA)" <ji...@apache.org> on 2019/04/22 08:56:00 UTC
[jira] [Updated] (AIRFLOW-4384) Support exclude from scheduling
time windows
[ https://issues.apache.org/jira/browse/AIRFLOW-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Víctor Portals updated AIRFLOW-4384:
------------------------------------
Description:
I think it would be useful to include as DAG attribute a list of calendar events to be excluded from scheduling. I've came across with the need of excluding specific dates from scheduling, for example holidays, and I think it could be useful for maintenance windows too.
I've been doing some tests with a custom subclass of DAG. It would accept an attribute like
exclude_calendar = [\{ start_date: '2019-04-22T10:00:00.000+00:00 ', end_date: ' 2019-04-22T11:00:00.000+00:00' }, \\{ start_date: '2019-04-22T17:00:00.000+00:00 ', end_date: ' 2019-04-22T18:00:00.000+00:00' }]
Then, when scheduler calls following_schedule function from DAG it would check if the following scheduled date belongs to any of the time windows included on exclude_calendar, and if it does, it will jump to the end of the time window, and check again, and so on until it finds a valid date.
Would be great if it could accept include_calendar too with the inverse behavior, only scheduling if it belongs to one of the time windows, but I think excluding will be enough for most use cases.
was:
I think it would be useful to include as DAG attribute a list of calendar events to be excluded from scheduling. I've came across with the need of excluding specific dates from scheduling, for example holidays, and I think it could be useful for maintenance windows too.
I've been doing some tests with a custom subclass of DAG. It would accept an attribute like
```exclude_calendar = [\{ start_date: '2019-04-22T10:00:00.000+00:00 ', end_date: ' 2019-04-22T11:00:00.000+00:00' }, \{ start_date: '2019-04-22T17:00:00.000+00:00 ', end_date: ' 2019-04-22T18:00:00.000+00:00' }] ```
Then, when scheduler calls following_schedule function from DAG it would check if the following scheduled date belongs to any of the time windows included on exclude_calendar, and if it does, it will jump to the end of the time window, and check again, and so on until it finds a valid date.
Would be great if it could accept include_calendar too with the inverse behavior, only scheduling if it belongs to one of the time windows, but I think excluding will be enough for most use cases.
> Support exclude from scheduling time windows
> --------------------------------------------
>
> Key: AIRFLOW-4384
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4384
> Project: Apache Airflow
> Issue Type: Improvement
> Components: DAG, scheduler
> Reporter: Víctor Portals
> Priority: Minor
> Labels: calendar, features, scheduler
>
> I think it would be useful to include as DAG attribute a list of calendar events to be excluded from scheduling. I've came across with the need of excluding specific dates from scheduling, for example holidays, and I think it could be useful for maintenance windows too.
> I've been doing some tests with a custom subclass of DAG. It would accept an attribute like
> exclude_calendar = [\{ start_date: '2019-04-22T10:00:00.000+00:00 ', end_date: ' 2019-04-22T11:00:00.000+00:00' }, \\{ start_date: '2019-04-22T17:00:00.000+00:00 ', end_date: ' 2019-04-22T18:00:00.000+00:00' }]
> Then, when scheduler calls following_schedule function from DAG it would check if the following scheduled date belongs to any of the time windows included on exclude_calendar, and if it does, it will jump to the end of the time window, and check again, and so on until it finds a valid date.
> Would be great if it could accept include_calendar too with the inverse behavior, only scheduling if it belongs to one of the time windows, but I think excluding will be enough for most use cases.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)