You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/04/19 20:52:47 UTC

[GitHub] [airflow] collinmcnulty opened a new pull request, #23099: Document built in Timetables

collinmcnulty opened a new pull request, #23099:
URL: https://github.com/apache/airflow/pull/23099

   Adding descriptions of the 3 built in, user-facing timetables to the Timetables doc page.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853550928


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.

Review Comment:
   We can use ``:class:`datetime.timedelta` `` since we already have intersphinx set up for CPython. We should probably do the same for dateutil as well (maybe in a separate PR).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853552341


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting
+events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable.
+
+The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally,
+the ``restrict_to_events`` flag can be used to force manual runs of the DAG to use the time of the most recent (or very
+first) event for the data interval, otherwise manual runs will run with a ``data_interval_start`` and
+``data_interval_end`` equal to the time at which the manual run was begun. You can also name the set of events using the
+``description`` parameter, which will be displayed in the Airflow UI.
+
+.. code-block:: python
+
+    @dag(
+        timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"),
+                                               pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"),
+                                               pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago")],
+                                  description="My Team's Baseball Games",
+                                  restrict_to_events=False)
+        ...
+    )

Review Comment:
   Let’s reformat this according to Black rules.
   
   ```suggestion
       @dag(
           timetable=EventsTimetable(
               event_dates=[
                   pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"),
                   pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"),
                   pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"),
               ],
               description="My Team's Baseball Games",
               restrict_to_events=False,
           ),
           ...,
       )
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853481174


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------

Review Comment:
   ```suggestion
   ^^^^^^^^^^^^^^^^^^^^^^^^^
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------

Review Comment:
   ```suggestion
   -------------------
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid

Review Comment:
   ```suggestion
   Set schedule based on a cron expression. Can be selected by providing a string that is a valid
   ```
   I think this sounds better?



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+--------------------------

Review Comment:
   ```suggestion
   ^^^^^^^^^^^^^^^^^^^^^^^^^^
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+--------------------------
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG.
+
+.. code-block:: python
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)  # At 01:00 on Wednesday.
+        ...
+    )
+
+EventsTimetable
+---------------
+Simply pass a list of datetimes for the DAG to run after. Useful for timing based on sporting

Review Comment:
   ```suggestion
   Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.

Review Comment:
   ```suggestion
     cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+--------------------------
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG.
+
+.. code-block:: python
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)  # At 01:00 on Wednesday.
+        ...
+    )
+
+EventsTimetable
+---------------

Review Comment:
   ```suggestion
   ^^^^^^^^^^^^^^^
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+--------------------------
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG.
+
+.. code-block:: python
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)  # At 01:00 on Wednesday.

Review Comment:
   ```suggestion
           schedule_interval=datetime.timedelta(minutes=30)
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,51 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+-------------------------
+Set timing based on a cron schedule. Can be selected by providing a string that is a valid
+  cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+--------------------------
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG.

Review Comment:
   ```suggestion
     ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] collinmcnulty commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1103155106

   @jedcunningham adding this documentation as per your suggestion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] eladkal commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1169732015

   @collinmcnulty can you please check the test failures?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ashb commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
ashb commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1170247144

   @collinmcnulty Something went very arry here with this PR, shout if you want a hand rebasing it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853546197


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.

Review Comment:
   ```suggestion
   may be available in plugins.
   ```
   
   They come from plugins, not providers, right?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1108352068

   Not sure what’s going on but code blocks are failing. Maybe the imports must be added?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r910326034


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...

Review Comment:
   ```suggestion
   ```
   
   These are what's blowing up the static checks.



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting
+events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable.
+
+The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally,
+the ``restrict_to_events`` flag can be used to force manual runs of the DAG to use the time of the most recent (or very
+first) event for the data interval, otherwise manual runs will run with a ``data_interval_start`` and
+``data_interval_end`` equal to the time at which the manual run was begun. You can also name the set of events using the
+``description`` parameter, which will be displayed in the Airflow UI.
+
+.. code-block:: python
+
+    from airflow.timetables.events import EventsTimetable
+
+    @dag(
+        timetable=EventsTimetable(
+            event_dates=[
+                pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"),
+                pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"),
+                pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"),
+            ],
+            description="My Team's Baseball Games",
+            restrict_to_events=False,
+        ),
+        ...,

Review Comment:
   ```suggestion
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...

Review Comment:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r910327652


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting

Review Comment:
   ```suggestion
   Simply pass a list of ``datetime``\s for the DAG to run after. Useful for timing based on sporting
   ```
   
   And this should fix the docs error?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] collinmcnulty commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1170262135

   Oh geez. Yeah, I thought I did the rebasing but obviously things fell apart.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853529408


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,55 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python

Review Comment:
   ```suggestion
   .. code-block:: python
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,55 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting
+events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable.
+
+The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally,
+the ``restrict_to_events`` flag can be used to force manual runs of the DAG to use the time of the most recent (or very
+first) event for the data interval, otherwise manual runs will run with a ``data_interval_start`` and
+``data_interval_end`` equal to the time at which the manual run was begun. You can also name the set of events using the
+``description`` parameter, which will be displayed in the Airflow UI.
+
+.. code-block:: python

Review Comment:
   ```suggestion
   .. code-block:: python
   
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,55 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python

Review Comment:
   ```suggestion
   .. code-block:: python
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] collinmcnulty commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853548198


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.

Review Comment:
   Yes, that's true. I will correct it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] collinmcnulty commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1170529719

   I'm at a loss. Do the example blocks need to be fully functional standalone python code? Is there some way to tell the CI that they're not?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r854190204


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,62 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting
+events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable.
+
+The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally,
+the ``restrict_to_events`` flag can be used to force manual runs of the DAG to use the time of the most recent (or very
+first) event for the data interval, otherwise manual runs will run with a ``data_interval_start`` and
+``data_interval_end`` equal to the time at which the manual run was begun. You can also name the set of events using the
+``description`` parameter, which will be displayed in the Airflow UI.
+
+.. code-block:: python
+
+    @dag(

Review Comment:
   We might want to show the import for `EventsTimetable` as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853552638


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval=datetime.timedelta(minutes=30)
+        ...
+    )
+
+EventsTimetable
+^^^^^^^^^^^^^^^
+
+Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting

Review Comment:
   ```suggestion
   Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1103245633

   The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1170754145

   They don’t need to be strictly functional, but must be valid Python syntax. So the easiest way to get around the checks is to just add
   
   ```python
   def dag():
       ...
   ```
   
   under those decorators.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham merged pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham merged PR #23099:
URL: https://github.com/apache/airflow/pull/23099


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853559287


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,62 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  `:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.

Review Comment:
   ```suggestion
     :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on PR #23099:
URL: https://github.com/apache/airflow/pull/23099#issuecomment-1171567426

   🎉 Thanks Collin!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r910256076


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.

Review Comment:
   ```suggestion
   :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.
   ```



##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,64 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in plugins.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.

Review Comment:
   ```suggestion
   cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23099: Document built in Timetables

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23099:
URL: https://github.com/apache/airflow/pull/23099#discussion_r853550928


##########
docs/apache-airflow/concepts/timetable.rst:
##########
@@ -48,3 +48,58 @@ the schedule. Some examples are:
 As such, Airflow allows for custom timetables to be written in plugins and used by
 DAGs. An example demonstrating a custom timetable can be found in the
 :doc:`/howto/timetable` how-to guide.
+
+Built In Timetables
+-------------------
+
+Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables
+may be available in provider packages.
+
+CronDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Set schedule based on a cron expression. Can be selected by providing a string that is a valid
+  cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation.
+
+.. code-block:: python
+
+    @dag(
+        schedule_interval="0 1 * * 3"  # At 01:00 on Wednesday.
+        ...
+    )
+
+DeltaDataIntervalTimetable
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Schedules data intervals with a time delta. Can be selected by providing a
+  ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG.

Review Comment:
   We can use `` :class:`datetime.timedelta` `` since we already intersphinx set up for CPython. We should probably do the same for dateutil as well (maybe in a separate PR).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org