[PR] Fix DetachedInstanceError crash in mapped task expansion [airflow]

["anmolxlight (via GitHub)" <gi...@apache.org>]

anmolxlight opened a new pull request, #73451:
URL: https://github.com/apache/airflow/pull/73451

   Closes: #73311
   
   **Root cause:** in `TaskMap.expand_mapped_task`, the Dag run is obtained via a lazy relationship (`unmapped_ti.dag_run`) or a scalar fetch, but `context_carrier` is only read much later after flushes, queries, and mutation hooks. By then session commits elsewhere in the scheduling chain can have expired and detached the Dag run, so the late attribute access raises `DetachedInstanceError`. The scheduling error handler then touches detached `run.run_id`/`run.dag_id`, raising a second `DetachedInstanceError` that escapes and kills the scheduler.
   
   **Fix:**
   - Read the trace carrier into a local at both points where the Dag run is obtained, while it is still bound to the session, and pass the local to `new_task_run_carrier`. A plain dict local is immune to later expire/detach, with zero extra queries.
   - Capture `run_id`/`dag_id` into locals before the try block in `_schedule_all_dag_runs` so the error handler can never throw.
   
   **Verification:** new logic proven with a standalone SQLAlchemy repro (old access pattern raises, captured local survives commit plus detach); `test_taskmap.py` green via breeze (3 passed); mypy, ruff, ruff-format green.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix DetachedInstanceError crash in mapped task expansion [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on PR #73451:
URL: https://github.com/apache/airflow/pull/73451#issuecomment-5839324711

   Hello @anmolxlight - thank you for your contributions to Apache Airflow!
   
   The Airflow community has introduced a limit of **5 open pull requests at a time** for contributors without write access to the repository. You currently have 10 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:
   
   * #70227 - Fix Execution API JWT reissue TOCTOU race
   * #71959 - Document REMOTE_TASK_LOG/DEFAULT_REMOTE_CONN_ID for custom logging configs
   * #72767 - Ignore stale defer-exit success once the resumed attempt is running
   * #73451 - Fix DetachedInstanceError crash in mapped task expansion
   
   These pull requests stay open because maintainers are already engaged in them - they count towards your limit:
   
   * #68239 - Fix spurious 'remote_log_handler_unavailable' warning in triggerer
   * #68557 - Use Configuration.socket_options instead of monkey-patching urllib3 in cncf-kubernetes
   * #70029 - Fix OAuth redirect URI scheme behind reverse proxy (#70023)
   * #71074 - Fix duplicate pending Dag runs for the same asset partition key
   * #71939 - Fix Spark standalone driver status polling to use the REST API port
   * #72495 - Add machine-readable RBAC reference to BaseAuthManager
   
   This is **not** a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.
   
   What we ask you to do is to make your **first prioritization decision**: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or `gh pr reopen <PR_NUMBER> --repo apache/airflow`. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.
   
   While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.
   
   Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.
   
   ---
   Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix DetachedInstanceError crash in mapped task expansion [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk closed pull request #73451: Fix DetachedInstanceError crash in mapped task expansion
URL: https://github.com/apache/airflow/pull/73451


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org