You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/03 14:27:03 UTC
[GitHub] [airflow] snehlata08 opened a new issue #14580: Request for sub process
snehlata08 opened a new issue #14580:
URL: https://github.com/apache/airflow/issues/14580
I need a sub process kind of thing for multiple dags.
I know we can call a child dag from parent dag through TriggerDagRunOperator but in that process how can we notify the parent if the child dag has run successfully or not.
Basically what i am looking at is something like the [BPMN tool](https://flowable.com/open-source/docs/bpmn/ch07b-BPMN-Constructs/#sub-processes-and-call-activities) . I also understand that basically this is a data workflow but something like this is required so that we know at what stage is the dag running.
This is my parent dag.
```
import airflow
from airflow.models import DAG
from airflow.operators.trigger_dagrun import TriggerDagRunOperator
dag = DAG(
dag_id='parent',
default_args={'start_date': airflow.utils.dates.days_ago(2), 'owner': 'airflow'},
schedule_interval='@once',
)
def trigger(context, dag_run_obj):
dag_run_obj.payload = {'message': context['params']['message']}
return dag_run_obj
test_trigger_dagrun = TriggerDagRunOperator(
dag=dag,
task_id='test_trigger_dagrun',
trigger_dag_id="child",
conf={'message': 'Hello World'}
)
```
This is child dag
```
import airflow
from airflow.models import DAG
from airflow.operators.python import PythonOperator
dag = DAG(
dag_id='child',
default_args={'start_date': airflow.utils.dates.days_ago(2), 'owner': 'airflow'},
schedule_interval=None,
)
def run_this_func(*args, **kwargs):
print("Remotely received a message: {}".
format(kwargs['dag_run'].conf.get('message')))
run_this = PythonOperator(
task_id='run_this',
python_callable=run_this_func,
provide_context=True,
dag=dag,
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka commented on issue #14580: Request for sub process
Posted by GitBox <gi...@apache.org>.
vikramkoka commented on issue #14580:
URL: https://github.com/apache/airflow/issues/14580#issuecomment-812236746
@snehlata08 Based on the conversation, this does not seem like a bug, therefore closing this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang edited a comment on issue #14580: Request for sub process
Posted by GitBox <gi...@apache.org>.
xinbinhuang edited a comment on issue #14580:
URL: https://github.com/apache/airflow/issues/14580#issuecomment-789898758
You can use the `wait_for_completion` parameter for the `TriggerDagRunOperator`
```python
test_trigger_dagrun = TriggerDagRunOperator(
dag=dag,
task_id='test_trigger_dagrun',
trigger_dag_id="child",
wait_for_completion=True, # `False` by default
conf={'message': 'Hello World'}
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka closed issue #14580: Request for sub process
Posted by GitBox <gi...@apache.org>.
vikramkoka closed issue #14580:
URL: https://github.com/apache/airflow/issues/14580
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang commented on issue #14580: Request for sub process
Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on issue #14580:
URL: https://github.com/apache/airflow/issues/14580#issuecomment-789898758
You can use the `wait_for_completion` parameter for the `TriggerDagRunOperator` (by default, it's `False`)
```python
test_trigger_dagrun = TriggerDagRunOperator(
dag=dag,
task_id='test_trigger_dagrun',
trigger_dag_id="child",
wait_for_completion=True,
conf={'message': 'Hello World'}
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14580: Request for sub process
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14580:
URL: https://github.com/apache/airflow/issues/14580#issuecomment-789752182
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org