[I] Lang-SDK runtimes: one transport for native Dag parsing and the task-handler probe [airflow]

["jason810496 (via GitHub)" <gi...@apache.org>]

jason810496 opened a new issue, #74058:
URL: https://github.com/apache/airflow/issues/74058

   Native Dag parsing (#74035, `LangSDKDagFileProcessorProcess` in `dag_processing/lang_sdk_processor.py`) and the task-handler probe (#73974, `LangSDKTaskHandlerProcessorProcess` in `dag_processing/task_handler_processor.py`) each carry their own copy of the runtime transport: the listeners, connection checks, frame reading, request handling, timeouts and the runtime kill, plus their own `StartLangSDKRuntime` and `LangSDKRuntimeSchemaVersion` models. Once both are on `main`:
   
   - Extract one shared base and move both processes onto it, with a single `tests/unit/dag_processing/fake_lang_sdk.py`.
   - Keep that commit free of behaviour changes. Then, as separate changes, give native parsing what the probe already has: a reported start failure (`LangSDKRuntimeStartFailed`), non-blocking frame reads, killing what a runtime leaves behind, and cleanup when a start fails after the fork. The parent-death signal stays opt-in, because a native parse may be started from a short-lived thread.
   - Results from a Lang-SDK runtime must not carry `task_handler_bindings` or `probed_artifacts`; the manager ignores those fields from a runtime.
   
   ---
   
   Drafted with Claude Code (Opus 5.5).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Lang-SDK runtimes: one transport for native Dag parsing and the task-handler probe [airflow]

["garvit-arora (via GitHub)" <gi...@apache.org>]

garvit-arora commented on issue #74058:
URL: https://github.com/apache/airflow/issues/74058#issuecomment-6067969811

   I'd like to investigate and take the first implementation slice here: extract the shared Lang-SDK runtime transport only after confirming the prerequisite parser/task-handler changes on main, with no behavior changes in the refactor. I will use a separate fork branch and add focused transport tests before opening a PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org