You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Kjir (via GitHub)" <gi...@apache.org> on 2026/07/07 15:47:21 UTC
[D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
GitHub user Kjir created a discussion: How to clear an older TaskInstance and run it with the latest dag version?
We recently updated our Airlflow from 2.x to the latest 3.x. I now need to re-run some older instances that were created with the 2.x version of Airflow, but the task is never enqueued with this log message:
```
2026-07-07T15:39:54.730911Z [warning [] TaskInstance <TaskInstance: [...]> does not have a dag_version_id set, cannot be enqueued. This would get unstuck and dag_version_id updated. [airflow.jobs.scheduler_job_runner.SchedulerJobRunner] loc=scheduler_job_runner.py:1033
```
How can I re-run this task instance and use the latest version of the dag for it? I do not see any option for that in the clear modal.
GitHub link: https://github.com/apache/airflow/discussions/69561
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "Kjir (via GitHub)" <gi...@apache.org>.
GitHub user Kjir added a comment to the discussion: How to clear an older TaskInstance and run it with the latest dag version?
Defining `rerun_with_latest_version=True` at the DAG level seems to work, even though the checkbox doesn't always consistently show up in the UI.
At the very least this obstacle seems to have been surpassed.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17585938
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "Kjir (via GitHub)" <gi...@apache.org>.
GitHub user Kjir edited a comment on the discussion: How to clear an older TaskInstance and run it with the latest dag version?
Defining `rerun_with_latest_version=True` at the DAG level after the upgrade to 3.3.0 seems to work, even though the checkbox doesn't always consistently show up in the UI.
At the very least this obstacle seems to have been surpassed.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17585938
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "apoorva-01 (via GitHub)" <gi...@apache.org>.
GitHub user apoorva-01 added a comment to the discussion: How to clear an older TaskInstance and run it with the latest dag version?
Short version: those task instances have no `dag_version_id` because 2.x had no DAG versioning and the upgrade doesn't backfill it. The 3.x scheduler refuses to enqueue any TI that's missing one, which is exactly the warning you're seeing at `scheduler_job_runner.py`.
It's meant to self-heal: when the run goes back through `verify_integrity`, the scheduler stamps the *latest* DagVersion onto the run's task instances and they enqueue. For that to actually fire, two things have to hold:
1. The DAG has to be parsed and active in 3.x right now, so there's a current row in the `dag_version` table to point at. If the DAG got dropped from your bundle or isn't parsing, `get_latest_version` returns nothing and there's nothing to backfill from. That's the usual reason one of these stays wedged forever instead of recovering on its own.
2. Clear the whole **DAG run**, not just the single task. Clearing one TI doesn't re-verify the run, so the null `dag_version_id` never gets filled in. Clearing the run puts it back through the scheduler, which re-verifies and backfills the version onto the tasks.
One version note: the "skip and backfill instead of crashing the scheduler" handling landed in 3.1.8 ([#61813](https://github.com/apache/airflow/pull/61813)). If you're below that, upgrade first, otherwise these null-version TIs can wedge the scheduler.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17565722
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "Kjir (via GitHub)" <gi...@apache.org>.
GitHub user Kjir added a comment to the discussion: How to clear an older TaskInstance and run it with the latest dag version?
I spoke too soon: some DAGs I seem to be able to do that, but other dags do not present the checkbox to run with the latest version. It's unclear why.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17570868
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "Kjir (via GitHub)" <gi...@apache.org>.
GitHub user Kjir added a comment to the discussion: How to clear an older TaskInstance and run it with the latest dag version?
Clearing the DAG run not only does not work, unfortunately, but also doesn't propose an option to clear future DAG runs. Basically there is not easy way to clear hundreds of DAG runs.
My Airflow version is currently 3.2.2, but I see 3.3.0 was just released and should have an improvement in this regard. I'll try to update and see whether that works.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17570508
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] How to clear an older TaskInstance and run it with the latest dag version? [airflow]
Posted by "Kjir (via GitHub)" <gi...@apache.org>.
GitHub user Kjir added a comment to the discussion: How to clear an older TaskInstance and run it with the latest dag version?
While I wasn't able to make this work with Airflow 3.2.2, Airflow 3.3.0 does provide the option of using the latest version on clear, and that fixes the issue.
GitHub link: https://github.com/apache/airflow/discussions/69561#discussioncomment-17570686
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org