You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jobs@airflow.apache.org by GitBox <gi...@apache.org> on 2026/06/18 11:10:59 UTC
[GH] (airflow/openlineage-execute-in-thread): Workflow run "Tests (AMD)" failed!
The GitHub Actions job "Tests (AMD)" on airflow.git/openlineage-execute-in-thread has failed.
Run started by GitHub user mobuchowski (triggered by mobuchowski).
Head commit for run:
81b87fde1b8a3d8cc8a788bd7cba5b8313bb1033 / Maciej Obuchowski <ma...@datadoghq.com>
OpenLineage: add execute_in_thread to emit task events without forking
By default the OpenLineage listener emits each task-level event from a
forked child process (os.fork() with no exec). That child inherits the
task runner's connection to the Airflow supervisor; if the child's event
emission blocks (e.g. a slow or unreachable lineage backend), the
inherited connection can be left in a state that prevents the task from
being marked complete, leaving it stuck in the `running` state.
Add an opt-in `[openlineage] execute_in_thread` option (default False).
When enabled, task-level emission runs in a time-bounded daemon thread
instead of forking: nothing is inherited, so a blocked emission can never
strand the task, and the task runner waits at most
`[openlineage] execution_timeout` for emission before proceeding.
Metadata extraction still runs in-process with full access to the task
runtime, so Operators whose extractors resolve Connections, Variables or
XComs keep working.
The default (fork) path is unchanged.
Signed-off-by: Maciej Obuchowski <ma...@datadoghq.com>
Report URL: https://github.com/apache/airflow/actions/runs/27751736227
With regards,
GitHub Actions via GitBox
---------------------------------------------------------------------
To unsubscribe, e-mail: jobs-unsubscribe@airflow.apache.org
For additional commands, e-mail: jobs-help@airflow.apache.org