You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2026/03/19 07:04:21 UTC
[PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
amoghrajesh opened a new pull request, #63916:
URL: https://github.com/apache/airflow/pull/63916
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
closes: https://github.com/apache/airflow/issues/63828
When running `airflow tasks render`, the command would fail with `DetachedInstanceError` when accessing `consumed_asset_events` on a detached DagRun instance.
To fix this I did a couple things:
1. Added eager loading of `consumed_asset_events` before template rendering in the CLI
2. Updated `DagRun.safe_extract_from_orm()` to safely extract only loaded attributes and provide defaults for unloaded relationships like `note` was handled earlier.
Working example:
```shell
[Breeze:3.10.19] root@44d338a2a42a:/opt/airflow$ airflow tasks render builtin_macros_most_recent showdoc_datetime manual__2026-03-19T06:44:52.286606+00:00
2026-03-19T06:46:01.163154Z [info ] setup plugin alembic.autogenerate.schemas [alembic.runtime.plugins] loc=plugins.py:37
2026-03-19T06:46:01.163280Z [info ] setup plugin alembic.autogenerate.tables [alembic.runtime.plugins] loc=plugins.py:37
2026-03-19T06:46:01.163370Z [info ] setup plugin alembic.autogenerate.types [alembic.runtime.plugins] loc=plugins.py:37
2026-03-19T06:46:01.163476Z [info ] setup plugin alembic.autogenerate.constraints [alembic.runtime.plugins] loc=plugins.py:37
2026-03-19T06:46:01.163580Z [info ] setup plugin alembic.autogenerate.defaults [alembic.runtime.plugins] loc=plugins.py:37
2026-03-19T06:46:01.163671Z [info ] setup plugin alembic.autogenerate.comments [alembic.runtime.plugins] loc=plugins.py:37
# ----------------------------------------------------------
# property: templates_dict
# ----------------------------------------------------------
None
# ----------------------------------------------------------
# property: op_args
# ----------------------------------------------------------
[<class 'datetime.datetime'>]
# ----------------------------------------------------------
# property: op_kwargs
# ----------------------------------------------------------
{}
```
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958426106
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
`values = values | {"note": insp.dict.get("note") }` can be done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "Nataneljpwd (via GitHub)" <gi...@apache.org>.
Nataneljpwd commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2964195774
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
Ok, understood, thank you
About the code suggestion, what do you mean different code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958425255
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
"note" was wrong actually. And `insp.dict` wouldn't have had it. The real relationship is called "dag_run_note"
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
`values = values | {"note": insp.dict.get("note") }` can be done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #63916:
URL: https://github.com/apache/airflow/pull/63916#issuecomment-4088265181
Umm it seems the new hook for execution API version changes is a little more restrictive that I like, proposing a fix for that so it ignores internal changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958438491
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
if "note" not in values:
values["note"] = insp.dict.get("note")
# or
values = values | {"note": insp.dict.get("note") }
isn't valid cos we have different code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2964129629
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
K, let me explain.
In the `DagRun` model, note is an association proxy and not a mapped column. The real attribute in the ORM state is `dag_run_note`.
`insp.dict` reflects the mapper attributes -- so `dag_run_note`. If `dag_run_note` is in `insp.dict`, the relationship would have loaded and we can safely use `data.note` but if not, accessing `data.note` would trigger a lazy load and cause DetachedInstanceError on a detached instance which we want to prevent.
So we check `dag_run_note` to know whether it’s safe to read note.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "Nataneljpwd (via GitHub)" <gi...@apache.org>.
Nataneljpwd commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2961395777
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
I am not sure I understood
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #63916:
URL: https://github.com/apache/airflow/pull/63916#issuecomment-4103119790
Failed checks are unrelated, merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh merged PR #63916:
URL: https://github.com/apache/airflow/pull/63916
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "Nataneljpwd (via GitHub)" <gi...@apache.org>.
Nataneljpwd commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958296430
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
It looks like previously it searched for a "note" key, yet now you search for "dag_run_note", what is the reason?
It also looks like this can be done shorter
```python
if "note" not in values:
values["note"] = insp.dict.get("note")
# or
values = values | {"note": insp.dict.get("note") }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958438491
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -325,19 +330,24 @@ def extract_dag_run_note(cls, data: Any) -> Any:
# Not a SQLAlchemy object, return as-is for Pydantic to handle
return data
+ values = {}
+
+ for field_name in cls.model_fields:
+ if field_name in insp.dict:
+ values[field_name] = insp.dict[field_name]
+ elif field_name == "state" and "_state" in insp.dict:
+ values["state"] = insp.dict["_state"]
+
+ if "consumed_asset_events" not in values:
+ values["consumed_asset_events"] = []
+
# Check if dag_run_note is already loaded (avoid lazy load on detached instance)
- if "note" in insp.dict:
- note_value: str | None = insp.dict["note"]
- else:
- note_value = None
-
- # Convert to dict to avoid further lazy loading issues
- values = {
- field_name: getattr(data, field_name, None)
- for field_name in cls.model_fields
- if field_name != "note"
- }
- values["note"] = note_value
+ if "note" not in values:
+ if "dag_run_note" in insp.dict:
+ values["note"] = data.note
+ else:
+ values["note"] = None
Review Comment:
```python
if "note" not in values:
values["note"] = insp.dict.get("note")
# or
values = values | {"note": insp.dict.get("note") }
```
isn't valid cos we have different code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Rectify DetachedInstanceError for airflow tasks render command [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #63916:
URL: https://github.com/apache/airflow/pull/63916#discussion_r2958238034
##########
airflow-core/src/airflow/api_fastapi/execution_api/datamodels/taskinstance.py:
##########
@@ -309,8 +309,13 @@ class DagRun(StrictBaseModel):
@model_validator(mode="before")
@classmethod
- def extract_dag_run_note(cls, data: Any) -> Any:
- """Extract the `note` (`str | None` from `association_proxy("dag_run_note", "content")`) relationship from `DagRun` to prevent `DetachedInstanceError` when constructing `DagRunContext` or `TIRunContext` models."""
Review Comment:
Renamed to a more appropriate method name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org