You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "RNHTTR (via GitHub)" <gi...@apache.org> on 2025/05/01 22:18:08 UTC

[I] Can't select the desired dag version for a backfill dag run [airflow]

RNHTTR opened a new issue, #50102:
URL: https://github.com/apache/airflow/issues/50102

### Apache Airflow version

3.0.0

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

I am making a bunch of edits to a dag, and I want to re-run a backfill for a specified date range. When I do, Airflow performs the backfill with the version of the dag that originally ran, and there's seemingly no way in the UI to specify that I want to re-run using the latest dag version.

### What you think should happen instead?

The latest version should maybe be the default version for backfills, and users should be able to select the version that runs.

### How to reproduce

1. Run a backfill over a certain date.
2. Update the dag code
3. Run a backfill over the same date from (1)

### Operating System

n/a

### Versions of Apache Airflow Providers

n/a

### Deployment

Astronomer

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.

vatsrahul1001 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-2846414165

   @RNHTTR are you using localbundle or gitbundle?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "Cir02 (via GitHub)" <gi...@apache.org>.

Cir02 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-3606434594

   Hi, is this issue solve? i can take care of this one as [good first issue](https://github.com/apache/airflow/issues?q=state%3Aopen%20label%3A%22good%20first%20issue%22)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "aritra24 (via GitHub)" <gi...@apache.org>.

aritra24 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-2848441124

   @RNHTTR I think there was some decision made here #49097 for how the version of dag is decided. @dstandish do you have any opinions on this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "aritra24 (via GitHub)" <gi...@apache.org>.

aritra24 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-2847968060

   From the code for the api 
   ```
   def _create_backfill(
   ...
   ):
      ...
       with create_session() as session:
           serdag = session.scalar(SerializedDagModel.latest_item_select_object(dag_id))
   ```
   I think it doesn't allow selecting a dag version right now and just uses the latest available version?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-3986456199

   @Cir02 We are unassigning you from this issue as part of our updated [assignment policy](https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#contribute-code-changes).
   
   This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.
   
   We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.

RNHTTR commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4730581003

   @VivinAT04 that appears at least directionally accurate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "ogulcanaydogan (via GitHub)" <gi...@apache.org>.

ogulcanaydogan commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4088285602

Hi! I noticed that `run_on_latest_version` was added in #52177 and its default was fixed to `True` in #59304, which partially addresses this — users can now choose between the latest version and the original version when running backfill.

However, the remaining scope seems to be **specific version selection**: letting users pick an arbitrary `dag_version_id` rather than just toggling between latest and original. Currently, `_create_backfill()` in `airflow-core/src/airflow/models/backfill.py` only accepts a boolean `run_on_latest_version`, and the `Backfill` model has no `dag_version_id` column.

A potential approach:
1. Add an optional `dag_version_id` parameter to `_create_backfill()` (when provided, use that specific version instead of latest/original)
2. Add a `dag_version_id` field to the API's `BackfillPostBody`
3. Add `--dag-version-id` CLI argument
4. Add a version selector in the UI's `RunBackfillForm` (using the existing `DagVersionService`)

Is this the right remaining scope, or has this been fully addressed by the `run_on_latest_version` work? Happy to work on a PR if specific version selection is still wanted. cc @aritra24 @dstandish

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "VivinAT04 (via GitHub)" <gi...@apache.org>.

VivinAT04 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4731697297

   Thanks for confirming.
   
   I've started investigating an implementation based on the remaining scope discussed above. My current approach is:
   
   * Add an optional `dag_version_number` field to backfill creation requests.
   * Resolve the requested `DagVersion` during backfill creation and use its associated `SerializedDagModel`.
   * Preserve existing behavior when no version is specified.
   * Add test coverage for creating a backfill against a specific DAG version.
   
   Before expanding this further into CLI/UI support, I'd appreciate confirmation from a maintainer that selecting an explicit DAG version is still the intended direction for this issue.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "safaeelm1-rgb (via GitHub)" <gi...@apache.org>.

safaeelm1-rgb commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-6312507864

   Hi @RNHTTR ! I'd like to work on this issue as my first contribution to Apache Airflow.
   
   I understand that the goal is to allow users to select a specific DAG version when triggering a backfill, rather than always using the latest version.
   
   I noticed that DagVersion.get_version() already supports retrieving a DAG version by its version number. Before I start working on this, could you confirm whether this issue is still open for contributions and whether selecting a specific version number is the intended approach?Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "dzpan0 (via GitHub)" <gi...@apache.org>.

dzpan0 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4070366021

   Hi, is this still available? I'd like to work on this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "VivinAT04 (via GitHub)" <gi...@apache.org>.

VivinAT04 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4747968582

   Just following up on this.
   
   I've prototyped the backend portion locally by adding optional DAG version selection during backfill creation and adding initial test coverage. Before continuing with CLI/UI work, I wanted to confirm whether selecting an explicit DAG version is still the intended direction for this issue.
   
   Happy to adjust the approach if maintainers prefer using `dag_version_id` directly rather than a user-facing `dag_version_number`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "VivinAT04 (via GitHub)" <gi...@apache.org>.

VivinAT04 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-4725409739

   Hi, I'd like to work on this issue.
   
   From the discussion, my understanding is that the original issue was partially addressed by `run_on_latest_version`, but the remaining scope is allowing users to select a specific `dag_version_id` rather than only choosing between the original and latest versions.
   
   Before I start investigating the implementation, could a maintainer confirm whether this is the intended remaining scope for this issue?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "aritra24 (via GitHub)" <gi...@apache.org>.

aritra24 commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-2848172173

   Huh, I'll check out the code once more. I'm just not sure on the expectation of if we do want to allow providing a version field. 
   
   If we decide on yes, I can go about making the change. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Can't select the desired dag version for a backfill dag run [airflow]

Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.

RNHTTR commented on issue #50102:
URL: https://github.com/apache/airflow/issues/50102#issuecomment-2848169069

   ![Image](https://github.com/user-attachments/assets/54a08ddf-c0d0-4744-9c88-67d2e3345f5a)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org