[PR] Add durable task instance launch records (RFC) [airflow]

["wolvery (via GitHub)" <gi...@apache.org>]

wolvery opened a new pull request, #70931:
URL: https://github.com/apache/airflow/pull/70931

   ## Summary
   
   This is a draft/RFC vertical slice that prevents **expected** stale executor launches from surfacing as opaque `/run` `404` failures. It introduces durable executor launch records and a typed `409 stale_executor_launch` response, so that a launch attempt from an executor whose task instance row has already been superseded (reset, rescheduled, cleared, or re-adopted) can be distinguished from a genuinely unknown task.
   
   Refs #69760.
   
   ## Motivation
   
   When a task instance transitions out of its original launch (scheduler failover, adoption, stuck-queued reschedule, orphan reset, or a manual clear), the executor that was originally assigned may still attempt to `/run` the task. Because the launch identity lived only on the mutable `TaskInstance` row, that late launch produced an opaque `404 not_found`, which is indistinguishable from a truly unknown task and consumes a retry / blocks clean requeueing.
   
   ## Approach
   
   - **Durable launch records.** New `TaskInstanceLaunch` ORM persists an immutable executor launch token independently of the mutable/deleted `TaskInstance` row, with an `active -> consumed | superseded` lifecycle, guarded state transitions, lookup helpers, indexes, and a state check constraint.
   - **Atomic creation.** A launch record is written during the `SCHEDULED -> QUEUED` transition in the scheduler.
   - **Supersession.** Reschedule of stuck-queued tasks, failed adoption, orphan reset, and TaskInstance clear/next-try all supersede any active launch so an old token cannot be reused if the old executor comes back. Successful adoption preserves the token.
   - **Typed conflict.** `/run` validates the launch token:
     - Known **terminal** token (consumed/superseded) -> `409 stale_executor_launch`.
     - Unknown token -> remains `404 not_found` (legacy tokens predating this change still return `404`).
     - Successful `/run` guarded-consumes the launch; duplicate-running remains idempotent.
   - **Worker behavior.** New `TaskInstanceSupersededError` in the Task SDK; a superseded worker logs once and exits successfully instead of failing.
   - **Executor-agnostic.** Works across Kubernetes, Celery, and Edge executors. No authorization changes and no fallback by logical task key.
   
   ## Changes
   
   - New model `airflow/models/task_instance_launch.py` + migration `3c5f8e9a1d2b` (down-revision `7a98f1b7dbd3`), registered with DB cleanup.
   - Scheduler: launch creation on queueing; supersession on reschedule / failed adoption / orphan reset.
   - `TaskInstance` clear/next-try: supersede active launch.
   - Execution API `/run`: durable-token validation + typed `409`; Cadwyn version adds optional `external_executor_id`.
   - Task SDK: propagate token on start, map `409 stale_executor_launch` to `TaskInstanceSupersededError`; supervisor exits `0` on supersession.
   - Tests: model, migration, scheduler, execution API, SDK client, and supervisor.
   
   ## Rollout / compatibility
   
   - Additive migration; no backfill required.
   - Unknown legacy tokens (launched before this change) still return `404` — behavior only changes for tokens that have a durable record.
   
   ## Status
   
   Draft / RFC for review of the approach before hardening. Feedback welcome on the launch-record lifecycle and the `409` contract.
   
   > Note: two pre-existing Task SDK supervisor tests (`test_run_simple_dag`, `test_supervise_handles_deferred_task`) fail on a clean checkout in this environment and are unrelated to this change.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add durable task instance launch records (RFC) [airflow]

["kaxil (via GitHub)" <gi...@apache.org>]

kaxil commented on PR #70931:
URL: https://github.com/apache/airflow/pull/70931#issuecomment-5942542404

   Does the stale-launch case need a new table? This adds `TaskInstanceLaunch`, a migration, an Execution API field and regenerated SDK models to return 409 instead of 404 on a stale `/run`. Main already returns 410 Gone for retired attempts by looking them up in `TaskInstanceHistory`, which is keyed by the attempt id (`_raise_ti_not_in_live_table(..., archived_in_history=...)`), and the `/run` route could use the same lookup. #69760 is also closed, and the branch edits the released `v2026_06_30` version and uses a migration revision main has moved past. Is there a case the history lookup can't cover? Converting to draft until that's settled.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org