You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "eladkal (via GitHub)" <gi...@apache.org> on 2023/03/03 12:30:35 UTC
[GitHub] [airflow] eladkal opened a new issue, #29900: Add continues scheduling option
eladkal opened a new issue, #29900:
URL: https://github.com/apache/airflow/issues/29900
### Body
There are some use cases where users want to trigger new DAG run as soon as one finished. This is a request I've seen several times with some variations (for example like this [Stackoverflow question](https://stackoverflow.com/q/75623153/14624409)) but the basic request is the same.
The workaround users do to get such functionality is place `TriggerDagRunOperator` as last task of their DAG invoking the same DAG:
```
from datetime import datetime
from airflow import DAG
from airflow.operators.empty import EmptyOperator
from airflow.operators.trigger_dagrun import TriggerDagRunOperator
with DAG(
dag_id="example",
start_date=datetime(2023, 1, 1,),
catchup=False,
schedule=None,
) as dag:
task = EmptyOperator(task_id="first")
trigger = TriggerDagRunOperator(
task_id="trigger",
trigger_dag_id="example",
)
task >> trigger
```
As you can see this works nicely:
My suggestion is to add first class support for this use case, so the above example will be changed to:
```
from datetime import datetime
from airflow import DAG
from airflow.operators.empty import EmptyOperator
from airflow.operators.trigger_dagrun import TriggerDagRunOperator
with DAG(
dag_id="example",
start_date=datetime(2023, 1, 1,),
catchup=False,
schedule="@continues",
) as dag:
task = EmptyOperator(task_id="first")
```
I guess it won't exactly be "@continues" but more likely new [ScheduleArg](https://github.com/apache/airflow/blob/8b8552f5c4111fe0732067d7af06aa5285498a79/airflow/models/dag.py#L127) type but I show it like that just for simplification of the idea.
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #29900: Add continues scheduling option
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #29900: Add continues scheduling option
URL: https://github.com/apache/airflow/issues/29900
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SamWheating commented on issue #29900: Add continues scheduling option
Posted by "SamWheating (via GitHub)" <gi...@apache.org>.
SamWheating commented on issue #29900:
URL: https://github.com/apache/airflow/issues/29900#issuecomment-1454034149
I'm interested in working on this, if you aren't already - I'll try to throw together a test implementation today and share my findings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #29900: Add continues scheduling option
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #29900:
URL: https://github.com/apache/airflow/issues/29900#issuecomment-1454084667
sure all yours
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org