You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/22 21:15:43 UTC
[GitHub] [airflow] HerrB92 opened a new pull request, #25883: Update templates-ref.rst
HerrB92 opened a new pull request, #25883:
URL: https://github.com/apache/airflow/pull/25883
Added some text to explain in more detail, which dates are to be expected in ds/ds_nodash etc.
It would be nice, if someone else could add a code example on how to use the macros to avoid e.g. yesterday_ds (listed as deprecated). This is out of my abilities ;-)
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25883:
URL: https://github.com/apache/airflow/pull/25883#discussion_r954858023
##########
docs/apache-airflow/templates-ref.rst:
##########
@@ -26,12 +26,29 @@ The following come for free out of the box with Airflow.
Additional custom macros can be added globally through :doc:`/plugins`, or at a DAG level through the
``DAG.user_defined_macros`` argument.
+.. _templates:basics:
+
+Note on Scheduling Basics
+-------------------------
+
+Airflow schedules tasks at the **end** of the interval (see :doc:`/scheduler`).
+
+Meaning that when you do:
+ | start_date: datetime(2018, 1, 1, 8, 0,0)
+ | schedule_interval: '0 8 * * *'
+
+The first run will kick in at ``2018-01-02 at 08:00+-`` (depends on resources) - but ds and ds_nodash will
+contain ``2018-01-01``/``20180101`` as **logical** start date. That is **yesterday** from the point of view
+of the actual execution day.
+
+(Elad Kalif/Peter Mortensen: https://stackoverflow.com/a/65196624)
Review Comment:
Would it be a good idea if we *move* some of the content here? The main thing I’m wondering is it is difficult to keep multiple documentation sections in sync (for example if we need to change the semantics of those variables a bit—which we actually just did not too long ago), so minimise duplication would be helpful. If the content reads well here for you, would you say some of the content might not flow as well in their existing locations? (This is purely an open question; I have no answers and your opinions would be most helpful.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #25883: Update templates-ref.rst
URL: https://github.com/apache/airflow/pull/25883
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25883:
URL: https://github.com/apache/airflow/pull/25883#discussion_r954858023
##########
docs/apache-airflow/templates-ref.rst:
##########
@@ -26,12 +26,29 @@ The following come for free out of the box with Airflow.
Additional custom macros can be added globally through :doc:`/plugins`, or at a DAG level through the
``DAG.user_defined_macros`` argument.
+.. _templates:basics:
+
+Note on Scheduling Basics
+-------------------------
+
+Airflow schedules tasks at the **end** of the interval (see :doc:`/scheduler`).
+
+Meaning that when you do:
+ | start_date: datetime(2018, 1, 1, 8, 0,0)
+ | schedule_interval: '0 8 * * *'
+
+The first run will kick in at ``2018-01-02 at 08:00+-`` (depends on resources) - but ds and ds_nodash will
+contain ``2018-01-01``/``20180101`` as **logical** start date. That is **yesterday** from the point of view
+of the actual execution day.
+
+(Elad Kalif/Peter Mortensen: https://stackoverflow.com/a/65196624)
Review Comment:
Would it be a good idea if we *move* some of the content here? The main thing I’m wondering is it is difficult to keep multiple documentation sections in sync (for example if we need to change the semantics of those variables a bit—which we actually just did not too long ago), so minimise duplication would be helpful. If the content reads well here for you, would you say some of the content might not flow as well in their existing locations? (This is purely an open question; I have no answers.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #25883:
URL: https://github.com/apache/airflow/pull/25883#issuecomment-1223065703
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #25883:
URL: https://github.com/apache/airflow/pull/25883#issuecomment-1272662874
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25883:
URL: https://github.com/apache/airflow/pull/25883#discussion_r952093487
##########
docs/apache-airflow/templates-ref.rst:
##########
@@ -26,12 +26,29 @@ The following come for free out of the box with Airflow.
Additional custom macros can be added globally through :doc:`/plugins`, or at a DAG level through the
``DAG.user_defined_macros`` argument.
+.. _templates:basics:
+
+Note on Scheduling Basics
+-------------------------
+
+Airflow schedules tasks at the **end** of the interval (see :doc:`/scheduler`).
+
+Meaning that when you do:
+ | start_date: datetime(2018, 1, 1, 8, 0,0)
+ | schedule_interval: '0 8 * * *'
+
+The first run will kick in at ``2018-01-02 at 08:00+-`` (depends on resources) - but ds and ds_nodash will
+contain ``2018-01-01``/``20180101`` as **logical** start date. That is **yesterday** from the point of view
+of the actual execution day.
+
+(Elad Kalif/Peter Mortensen: https://stackoverflow.com/a/65196624)
Review Comment:
If I recall correct there are already at least two sections explaining this, before this one was added. You might want to look them up and reference those instead of adding an entire new section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] HerrB92 commented on a diff in pull request #25883: Update templates-ref.rst
Posted by GitBox <gi...@apache.org>.
HerrB92 commented on code in PR #25883:
URL: https://github.com/apache/airflow/pull/25883#discussion_r954740113
##########
docs/apache-airflow/templates-ref.rst:
##########
@@ -26,12 +26,29 @@ The following come for free out of the box with Airflow.
Additional custom macros can be added globally through :doc:`/plugins`, or at a DAG level through the
``DAG.user_defined_macros`` argument.
+.. _templates:basics:
+
+Note on Scheduling Basics
+-------------------------
+
+Airflow schedules tasks at the **end** of the interval (see :doc:`/scheduler`).
+
+Meaning that when you do:
+ | start_date: datetime(2018, 1, 1, 8, 0,0)
+ | schedule_interval: '0 8 * * *'
+
+The first run will kick in at ``2018-01-02 at 08:00+-`` (depends on resources) - but ds and ds_nodash will
+contain ``2018-01-01``/``20180101`` as **logical** start date. That is **yesterday** from the point of view
+of the actual execution day.
+
+(Elad Kalif/Peter Mortensen: https://stackoverflow.com/a/65196624)
Review Comment:
Yes, there are. That is why I added the reference as first line (see :doc:`/scheduler`). Unfortunately, the explanation there is not that easy to understand, that as a result "ds" contains "yesterday".
As in the scheduler explanation the reference to ds etc. does not belong (IMHO), I would like to see it here - as by using Google & searching for ds etc. you will like land here, first.
But I agree with you that I can reduce the explanation to a short sentence.
Beside that I have also now an example for the use of macros and I will add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org