DAG triggered by another DAG

[Pedro Machado <pe...@205datalab.com>]

I have a DAG with 3 parallel branches. Each branch processes a file
(multiple steps). Each branch is triggered by the arrival of one of
three files which can happen at any time. After a file is processed, we
have to trigger a database update (we use dbt).

I had a dbt operator at the end of each branch but this may result in two
tasks running too close to each other. We need to wait at least 1h between
updates.

I'd like change the DAG so that each branch starts another DAG that will
update the DB but I want this DAG to run with concurrency = 1 and wait at
least 1 hour between successive runs.

This way if two files arrive around the same time, the first one will
trigger a database update and the second will trigger the second update but
not immediately after the first one.

The reason we need to wait 1 hour is that we have a process feeding data
back every 30 minutes and need to wait for this data to be fed back before
we run another update.

Do you have a different idea to implement this? How do you suggest we
implement the second DAG so that it won't run more than one DagRun at the
same time and so that it will wait at least one hour after its prior
execution before it runs again.

Thanks!

Pedro