[GitHub] [airflow] dimonchik-suvorov opened a new issue #16087: Bckfill shouldn't interfere with scheduled run

[GitBox <gi...@apache.org>]

dimonchik-suvorov opened a new issue #16087:
URL: https://github.com/apache/airflow/issues/16087


   Example:
   DAG structure:
   ```
   default_args = {
       'owner': 'dimon',
       'depends_on_past': False,
       'start_date': datetime(2021, 1, 10)
   }
   
   dag = DAG(
       'dummy-dag',
       schedule_interval='21 2 * * *',
       catchup=False,
       default_args=default_args
   )
   
   DagContext.push_context_managed_dag(dag)
   task1 = BashOperator(task_id='task1', bash_command='echo 1')
   task2 = BashOperator(task_id='task2', bash_command='echo 2')
   task2 << task1
   task3 = BashOperator(task_id='task3', bash_command='echo 3)
   ```
   
   1. Scheduler creates a DagRun and starts Task Instances processing
   2. Scheduler starts “task1” and “task3”
   3. At the same time a backfill started for single task - "task3"
   
   Backfill updates scheduler’s DagRun DB table row and scheduler forgets about “task2” and it never gets triggered. Also the DugRun will stuck in the "running" state because backfill will not finish it correctly.
   So we have to make a check - if we are starting a backfill with some date, but there is a DagRun in the "running" state for that date already we should not perform a backfill.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org