You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/16 14:06:44 UTC

[GitHub] [airflow] eejbyfeldt opened a new issue #14260: Clearing using ExternalTaskMarker will not activate external DagRuns

eejbyfeldt opened a new issue #14260:
URL: https://github.com/apache/airflow/issues/14260

**Apache Airflow version**:

2.0.1

**What happened**:

When clearing task across dags using `ExternalTaskMarker` the dag state of the external `DagRun` is not set to active. So cleared tasks in the external dag will not automatically start if the `DagRun` is a Failed or Succeeded state.

**What you expected to happen**:

The external `DagRun` run should also be set to Running state.

**How to reproduce it**:

Clear tasks in an external dag using an ExternalTaskMarker.

**Anything else we need to know**:

Looking at the code is has:

https://github.com/apache/airflow/blob/b23fc137812f5eabf7834e07e032915e2a504c17/airflow/models/dag.py#L1323-L1335

It seems like it intentionally calls the dag member method `set_dag_run_state` instead of letting the helper function `clear_task_insntances` set the `DagRun` state. But the member method will only change the state of `DagRun`s of dag where the original task is, while I believe `clear_task_instances` would correctly change the state of all involved `DagRun`s.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] yuqian90 closed issue #14260: Clearing using ExternalTaskMarker will not activate external DagRuns

Posted by GitBox <gi...@apache.org>.

yuqian90 closed issue #14260:
URL: https://github.com/apache/airflow/issues/14260


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] yuqian90 commented on issue #14260: Clearing using ExternalTaskMarker will not activate external DagRuns

Posted by GitBox <gi...@apache.org>.

yuqian90 commented on issue #14260:
URL: https://github.com/apache/airflow/issues/14260#issuecomment-817780114


   Does anyone know why this code  needs to call `self.set_dag_runs_state` separately? If we simply remove the `self.set_dag_runs_state` call and leave `activate_dag_runs` as the default `True`, things will just work as expected.
   
   ```python
    if do_it: 
        clear_task_instances( 
            tis, 
            session, 
            dag=self, 
            activate_dag_runs=False,  # We will set DagRun state later. 
        ) 
        self.set_dag_runs_state( 
            session=session, 
            start_date=start_date, 
            end_date=end_date, 
            state=dag_run_state, 
        ) 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] zachliu commented on issue #14260: Clearing using ExternalTaskMarker will not activate external DagRuns

Posted by GitBox <gi...@apache.org>.

zachliu commented on issue #14260:
URL: https://github.com/apache/airflow/issues/14260#issuecomment-817854265


   @yuqian90 the fastest way to get an answer is to submit a PR to it
   this issue hasn't been assigned to anyone nor to a milestone, seems to be low priority
   submitting a PR could definitely raise some awareness and speed things up :grin: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] zachliu commented on issue #14260: Clearing using ExternalTaskMarker will not activate external DagRuns

Posted by GitBox <gi...@apache.org>.

zachliu commented on issue #14260:
URL: https://github.com/apache/airflow/issues/14260#issuecomment-816954310


   oh no, very sad to see this :bug:
   once this is fixed :crossed_fingers: i'll be able to use the `ExternalTaskSensor` and `ExternalTaskMarker` in the same DAG to achieve something like a "task reference" which allows me to control multiple parts in different DAGs using another DAG. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org