Re: [PR] Fix custom stats metrics not being sent from plugins/listeners [airflow]

["Aaryan123456679 (via GitHub)" <gi...@apache.org>]

Aaryan123456679 commented on code in PR #69270:
URL: https://github.com/apache/airflow/pull/69270#discussion_r3941358680


##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -105,6 +106,37 @@ def initialize(
     _factory = factory
     _backend = None
     _export_legacy_names = export_legacy_names
+    _propagate_to_sibling_modules(factory=factory, export_legacy_names=export_legacy_names)
+
+
+def _propagate_to_sibling_modules(
+    *,
+    factory: Callable[[], StatsLogger | NoStatsLogger],
+    export_legacy_names: bool,
+) -> None:
+    """
+    Apply the same configuration to other loaded copies of this module.
+
+    This source file is symlinked into multiple distributions (e.g. ``airflow-core`` and
+    ``task-sdk``), each importing it under a different module name (``airflow._shared...`` vs
+    ``airflow.sdk._shared...``). Python treats each as a distinct module object with its own
+    module-level globals, so a process that has both loaded (e.g. the scheduler, which also runs

Review Comment:
   Removed `_propagate_to_sibling_modules` entirely. Each copy of `stats.py` now lazily resolves its own factory the first time it's actually used (in `_get_backend()`), instead of one copy reaching into another's globals when `initialize()` runs: it derives its own distribution root from its own `__name__` (`airflow._shared...` → `airflow`, `airflow.sdk._shared...` → `airflow.sdk`) and dynamically imports `<root>.observability.metrics.stats_utils.get_stats_factory()` / `<root>.configuration.conf`.
   
   This directly fixes the ordering concern — since resolution happens lazily on first use rather than reactively when some other copy's `initialize()` runs, it no longer matters whether the sibling module is loaded before or after, or via a local/deferred import.
   
   `initialize()` itself is unchanged for the caller (still only sets `_factory`/`_backend`/`_export_legacy_names` on the module it's called on) — kept as the eager path so the 5 existing call sites (scheduler, executor, Dag processor, API server, triggerer) still catch a misconfiguration and log a warning at startup, rather than have it surface on the first metric emission.
   
   See 7d99318cb5.
   
   ---
   Drafted-by: Claude Code (Sonnet 5); reviewed by @Aaryan123456679 before posting



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org