You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "uranusjr (via GitHub)" <gi...@apache.org> on 2024/08/14 01:30:30 UTC
[PR] Remove schedule_interval and timetable DAG args [airflow]
uranusjr opened a new pull request, #41453:
URL: https://github.com/apache/airflow/pull/41453
The 'timetable' attribute still exists on DAG, and is now the only value that reflects the DAG's schedule. The 'schedule_interval' attribute is removed from DAG.
The 'schedule_interval' on DagModel used to store a string representation of DAG's attribute of the same name, is now replaced by timetable_summary, which should (mostly?) work the same as before. We can fix minor UI differences as we go. Some use cases that rely on that field are also changed to use other fields instead (dataset_expression, for example, can be used to check whether a DAG is dataset-triggered).
The API field 'schedule_interval' has also been removed since that field no longer exists in the database. This has some side effects. The field previously contains some type information for delta values, but now becomes a simple string. It is unclear if anyone really needs the information, but we can always bring it back if needed.
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1719811192
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
Why `timetable_summary` and not `schedule`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "mertozb (via GitHub)" <gi...@apache.org>.
mertozb commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1724141565
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
As a daily user, most of jobs that i run did not require 'dataset' features at all and when i got to know that feature it was because of my interest and at that time years was passed about when i start using Apache Airflow. I hear similar convos around, so i discourage to just flying away / replacing 'schedule_interval' that almost all DAGs are using in order to provide some meaning integrity that about a feature not yet adopted for now. @kaxil
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr merged PR #41453:
URL: https://github.com/apache/airflow/pull/41453
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1724278749
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
I don’t understand what the comment is trying to say, to be honest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1724278749
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
I don’t understand what the comment is trying to say, to be honest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1726185195
##########
newsfragments/24842.significant.rst:
##########
@@ -0,0 +1,6 @@
+Default DAG schedule changed to *None*
Review Comment:
ah got what you mean
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1726182578
##########
newsfragments/24842.significant.rst:
##########
@@ -0,0 +1,6 @@
+Default DAG schedule changed to *None*
Review Comment:
They are logically different changes (one is a default value change; the other is removal of deprecated features) and should be kept separate in the release notes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "mertozb (via GitHub)" <gi...@apache.org>.
mertozb commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1724141565
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
As a daily user, most of jobs that i run did not require 'dataset' features at all and when i got to know that feature it was because of my interest and at that time years was passed about when i start using Apache Airflow. I hear similar convos around, so i discourage to just flying away / replacing 'schedule_interval' that almost all DAGs are using in order to provide some meaning integrity that about a feature not yet adopted for now. @kaxil
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on PR #41453:
URL: https://github.com/apache/airflow/pull/41453#issuecomment-2303606435
> schedule_interval -> timetable_summary
These two are sort of interchangable for display purposes, but not if you want to use the value for something else. For example:
```python
with DAG(schedule=timedelta(days=2)) as d1:
...
# This can not be changed to timetable_summary.
# d1.timetable would work, but fail for other use cases.
with DAG(schedule=d1.schedule_interval) as d2:
...
```
If we’re going the linter (ruff-like_ route, this would be a case for showing a linting error without an automated fix available.
> schedule=NOTSET, schedule=None -> schedule=timedelta(days=1),
`schedule=NOTSET` should be changed to `timedelta(days=1)`, but `schedule=None` should be kept as-is (its behaviour is not changed in 3.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1724773889
##########
newsfragments/24842.significant.rst:
##########
@@ -0,0 +1,6 @@
+Default DAG schedule changed to *None*
Review Comment:
Should we merge this into `41453.significant.rst`? or is it needed as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #41453:
URL: https://github.com/apache/airflow/pull/41453#issuecomment-2301840275
if we're to add migration rule for this PR to https://github.com/apache/airflow/issues/41641, we probably could do the following 2?
* `schedule_interval` -> `timetable_summary`
* `schedule=NOTSET,` `schedule=None` -> `schedule=timedelta(days=1),`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Unify DAG schedule args and change default to None [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #41453:
URL: https://github.com/apache/airflow/pull/41453#discussion_r1720763890
##########
newsfragments/41453.significant.rst:
##########
@@ -0,0 +1,11 @@
+Removed legacy scheduling arguments on DAG
+
+The ``schedule_interval`` and ``timetable`` arguments are removed from ``DAG``.
+
+The ``schedule_interval`` _attribute_ has also been removed. In the API, a new
Review Comment:
This matches the column name on `DagModel`. We can’t just call this `schedule` since there’s also a field `timetable_summary`. I guess if we really want to we can call this `schedule_summary` (and also rename the description field), but I prefer *timetable* here since it does not represent datasets.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org