You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "xBis7 (via GitHub)" <gi...@apache.org> on 2026/03/19 14:02:21 UTC
[PR] Remove the DualStatsManager and the Stats interfaces [airflow]
xBis7 opened a new pull request, #63932:
URL: https://github.com/apache/airflow/pull/63932
This is an initial approach at removing the `DualStatsManager` and the `Stats` interfaces.
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [X] Yes (please specify the tool below)
Claude Sonnet 4.6 Extended
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2961008392
##########
scripts/ci/prek/check_metrics_synced_with_the_registry.py:
##########
@@ -51,7 +51,7 @@
"timer": "timer",
}
-STATS_OBJECTS = {"Stats", "stats", "DualStatsManager"}
+STATS_OBJECTS = {"Stats", "stats"}
Review Comment:
because in certain places we had
```
self.stats.incr(...
```
but now, if the `Stats` shim is also removed from the core and sdk, then it can be just `stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154599466
##########
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py:
##########
@@ -225,10 +225,6 @@ def _update_orphaned_jobs(self, session: Session) -> bool:
"queue": job.queue,
"state": str(TaskInstanceState.FAILED),
}
- Stats.incr(
- f"edge_worker.ti.finish.{job.queue}.{TaskInstanceState.FAILED}.{job.dag_id}.{job.task_id}",
- tags=tags,
- )
Stats.incr("edge_worker.ti.finish", tags=tags)
Review Comment:
Yes, it's generated from the registry. We don't need it anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3153594766
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +78,246 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
-
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
-
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
-
- type.__setattr__(cls, "instance", instance)
-
- return getattr(instance, name)
-
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
-
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _export_legacy_names = export_legacy_names
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, creating it on first access."""
+ global _backend
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
Review Comment:
I made the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "kacpermuda (via GitHub)" <gi...@apache.org>.
kacpermuda commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4371244385
No problem, we caught that early, we should have had tests for that and now we'll add them, sounds like a win to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3128980574
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
This wasn't intentional. It was a bad outcome from the latest rebase. I'll fix it. Good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011001768
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
I think ift should be changed to:
`from airflow._shared.observability.metrics.stats import Stats
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011691485
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
aah, I see. It was meant for any kind of external packages like 3rd-party providers that you mentioned. That's why the warning message was suggesting to use the sdk path. And I thought it was some kind of oversight.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4287217850
@ashb Thanks for the review! The rebase broke the CI. I'm going to fix it and then I'll address all comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2960976375
##########
scripts/ci/prek/check_metrics_synced_with_the_registry.py:
##########
@@ -51,7 +51,7 @@
"timer": "timer",
}
-STATS_OBJECTS = {"Stats", "stats", "DualStatsManager"}
+STATS_OBJECTS = {"Stats", "stats"}
Review Comment:
why are there two?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3092169249
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
This creates another issue. Let's assume that this PR gets merged with checks such as
```
if AIRFLOW_V_3_3_PLUS:
```
and then we cut a new minor release that includes the changes. The target version in the minor release will be changed from `3.3.0` to `3.2.1`. In that case, the providers will break because they will always execute the `else` block but `airflow-core` will need the code from the 1st block.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3137885938
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +72,239 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
+
+
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
+
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
+ extra_tags = {k: v for k, v in (tags or {}).items() if k not in name_var_set}
+ return formatted_name, extra_tags
+
+
+def _defined(**kwargs: Any) -> dict[str, Any]:
+ """Return only kwargs with a meaningful (non-None, non-empty) value."""
+ result = {}
+ for k, v in kwargs.items():
+ if v is None:
+ continue
+ try:
+ if len(v) == 0:
+ continue
+ except TypeError:
+ pass # Numerics and bools don't have len, always include
+ result[k] = v
+ return result
+
+
+def incr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Increment a counter metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().incr(legacy_name, **_defined(count=count, rate=rate, tags=extra_tags))
+ _get_backend().incr(stat, **_defined(count=count, rate=rate, tags=tags))
+
+
+def decr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Decrement a counter metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().decr(legacy_name, **_defined(count=count, rate=rate, tags=extra_tags))
+ _get_backend().decr(stat, **_defined(count=count, rate=rate, tags=tags))
+
+
+def gauge(
+ stat: str,
+ value: float,
+ rate: int | float | None = None,
+ delta: bool | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Set a gauge metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().gauge(legacy_name, value, **_defined(rate=rate, delta=delta, tags=extra_tags))
+ _get_backend().gauge(stat, value, **_defined(rate=rate, delta=delta, tags=tags))
+
+
+def timing(
+ stat: str,
+ dt: DeltaType,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Record a timing metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().timing(legacy_name, dt, **_defined(tags=extra_tags))
+ _get_backend().timing(stat, dt, **_defined(tags=tags))
+
+
+def timer(
+ stat: str | None = None,
+ tags: dict[str, Any] | None = None,
+ **kwargs,
+) -> Timer:
+ """
+ Context manager that times a block and emits a timer metric.
+
+ When ``stat`` is None, returns the backend timer directly for stopwatch
+ use (duration is available but no metric is emitted).
+
+ When legacy name export is enabled and the metric has a legacy name in the
+ registry, both the legacy metric name and the regular name are timed
+ simultaneously via an ExitStack. Otherwise, the backend timer is returned
+ directly, preserving the ability to access ``timer.duration`` on the returned object.
+ """
+ if stat is None:
+ return _get_backend().timer()
+
+ regular_kw: dict[str, Any] = {**kwargs}
+ if tags:
+ regular_kw["tags"] = tags
+
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ legacy_kw: dict[str, Any] = {**kwargs}
+ if extra_tags:
+ legacy_kw["tags"] = extra_tags
+
+ stack = ExitStack()
+ stack.enter_context(
+ cast(
+ "AbstractContextManager[Any]",
+ _get_backend().timer(legacy_name, **legacy_kw),
)
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
+ )
+ real_timer = stack.enter_context(_get_backend().timer(stat, **regular_kw))
+ return _ExitStackTimer(stack, real_timer)
Review Comment:
`_ExitStackTimer` does not support the stopwatch protocol
`timer()` declares `-> Timer` as its return type, and `Timer`/`TimerProtocol` defines a stopwatch-style API: `.start()`, `.stop(send=True|False)`, `.duration`. The scheduler uses this pattern today:
```python
timer = Stats.timer("scheduler.critical_section_duration")
timer.start()
try:
# ...
timer.stop(send=True)
except OperationalError:
timer.stop(send=False)
```
When `timer()` returns an `_ExitStackTimer` (i.e. when the metric has a legacy name in the registry — the `if legacy_name is not None:` branch here), this stopwatch API is broken:
- `.stop(send=True)` — records `duration` and stops the *regular* backend timer, but the `ExitStack` is never exited, so the legacy timer is never stopped/sent. Resource leak + missing legacy metric.
- `.stop(send=False)` — records `duration` only. Neither timer is properly cleaned up.
This is not a live bug today because the metrics that use stopwatch style (`scheduler.critical_section_duration`, `scheduler.critical_section_query_duration`) both have `legacy_name: "-"` in the YAML, so `timer()` returns a plain backend timer, not `_ExitStackTimer`. But if anyone later adds a legacy name to a metric that uses stopwatch style, it will silently break.
Additionally, there is a double-start issue when `_ExitStackTimer` is used as a context manager. `timer()` calls `stack.enter_context()` on both backend timers, which calls `__enter__()` → `start()` on each — both are now running. Then when the caller does `with _exit_stack_timer as t:`, `__enter__()` (inherited from `Timer`) calls `self.start()` → `self.real_timer.start()`, which starts the regular backend timer a **second** time (overwriting its `_start_time` and re-starting the underlying statsd timer). The legacy timer on the stack is *not* re-started, so the two timers measure from different start points (legacy from `timer()` call time, regular from `with` entry time). The delta is microseconds in practice, but it is an asymmetry that could confuse someone reading the code.
Suggested fix: don't eagerly start the timers in `timer()`. Instead, have `_ExitStackTimer.__enter__` create and start both timers, and store them for `__exit__` to clean up. This would fix both the stopwatch protocol and the double-start. Alternatively, if stopwatch mode genuinely is not needed for dual-emit metrics, document that clearly and narrow the return type.
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +72,239 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
+
+
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
+
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
+
+ return _backend
Review Comment:
The PID check on every call to `_get_backend()` could be replaced by `os.register_at_fork(after_in_child=...)` that resets `_backend = None`. Then `_get_backend()` simplifies to just `if _backend is None` — no PID tracking needed.
This also removes the need to call `_get_backend()` multiple times per emission (currently happens in `incr`/`decr`/`gauge`/`timing`/`timer` when a legacy name exists).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154721703
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -230,7 +233,8 @@ def on_running():
if not doc:
doc, doc_type = get_dag_documentation(dag)
- with Stats.timer(f"ol.extract.{event_type}.{operator_name}"):
+ ctx = Stats.timer("ol.extract", tags={"event_type": event_type, "operator_name": operator_name})
+ with ctx:
Review Comment:
I think it was used in a previous commit but now it's just pointless. I'll remove it along with the other occurrences.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3120195163
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,71 +71,235 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
- type.__setattr__(cls, "instance", instance)
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
- return getattr(instance, name)
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
+ return _backend
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def _get_legacy_stat(stat: str, variables: dict[str, Any]) -> str | None:
+ """
+ Look up and format the legacy name for a metric from the registry.
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+ Returns the formatted legacy name, or None if the metric has no legacy name.
+ Raises ValueError if the metric is not in the registry or required variables are missing.
+ """
+ stat_from_registry = _registry.get(name=stat)
+
+ if not stat_from_registry:
+ raise ValueError(
+ f"Metric '{stat}' not found in the registry. Add the metric to the YAML file before using it."
+ )
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ if legacy_name == "-":
+ return None
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ missing_vars = set(required_vars) - set(variables.keys())
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided variables: {sorted(variables.keys())}. "
+ f"Provide all required variables."
+ )
+
+ legacy_vars = {k: variables[k] for k in required_vars if k in variables}
+ return legacy_name.format(**legacy_vars)
+
+
+def _defined(**kwargs: Any) -> dict[str, Any]:
+ """Return only kwargs with a meaningful (non-None, non-empty) value."""
+ result = {}
+ for k, v in kwargs.items():
+ if v is None:
+ continue
+ try:
+ if len(v) == 0:
+ continue
+ except TypeError:
+ pass # Numerics and bools don't have len, always include
+ result[k] = v
+ return result
+
+
+def _merged(
+ tags: dict[str, Any] | None,
+ legacy_name_tags: dict[str, Any] | None,
+) -> dict[str, Any] | None:
+ """Merge tags and legacy_name_tags into a single dict, returning None if both are empty."""
+ if not tags and not legacy_name_tags:
+ return None
+ merged: dict[str, Any] = {}
+ if tags:
+ merged.update(tags)
+ if legacy_name_tags:
+ merged.update(legacy_name_tags)
+ return merged or None
+
+
+def _emit_legacy(stat: str, legacy_name_tags: dict[str, Any]) -> str:
+ """Return the formatted legacy stat name, raising ValueError if it has none."""
+ legacy_stat = _get_legacy_stat(stat, legacy_name_tags)
+ if legacy_stat is None:
+ raise ValueError(f"Stat '{stat}' doesn't have a legacy name registered in the YAML file.")
+ return legacy_stat
+
+
+def incr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Increment a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().incr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().incr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def decr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Decrement a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().decr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().decr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def gauge(
+ stat: str,
+ value: float,
+ rate: int | float | None = None,
+ delta: bool | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Set a gauge metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().gauge(
+ _emit_legacy(stat, legacy_name_tags), value, **_defined(rate=rate, delta=delta, tags=tags)
+ )
+ _get_backend().gauge(
+ stat, value, **_defined(rate=rate, delta=delta, tags=_merged(tags, legacy_name_tags))
+ )
+
+
+def timing(
+ stat: str,
+ dt: DeltaType,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Record a timing metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().timing(_emit_legacy(stat, legacy_name_tags), dt, **_defined(tags=tags))
+ _get_backend().timing(stat, dt, **_defined(tags=_merged(tags, legacy_name_tags)))
+
+
+def timer(
+ stat: str | None = None,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+ **kwargs,
+) -> Timer:
+ """
+ Context manager that times a block and emits a timer metric.
+
+ When ``stat`` is None, returns the backend timer directly for stopwatch
+ use (duration is available but no metric is emitted).
+
+ When ``legacy_name_tags`` is provided and legacy name export is enabled, both the
+ legacy metric name and the regular name are timed simultaneously via an
+ ExitStack. Otherwise, the backend timer is returned directly, preserving the
+ ability to access ``timer.duration`` on the returned object.
+ """
+ if stat is None:
+ return _get_backend().timer()
+
+ regular_kw: dict[str, Any] = {**kwargs}
+ merged = _merged(tags, legacy_name_tags)
+ if merged:
+ regular_kw["tags"] = merged
+
+ if _export_legacy_names and legacy_name_tags is not None:
+ legacy_kw: dict[str, Any] = {**kwargs}
+ if tags is not None:
+ legacy_kw["tags"] = tags
+
+ stack = ExitStack()
+ stack.enter_context(
+ cast(
+ "AbstractContextManager[Any]",
+ _get_backend().timer(_emit_legacy(stat, legacy_name_tags), **legacy_kw),
+ )
+ )
+ stack.enter_context(_get_backend().timer(stat, **regular_kw))
+ return cast("Timer", stack)
Review Comment:
Fixed. I went with the wrapper approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3120318868
##########
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py:
##########
@@ -225,11 +226,14 @@ def _update_orphaned_jobs(self, session: Session) -> bool:
"queue": job.queue,
"state": str(TaskInstanceState.FAILED),
}
- Stats.incr(
- f"edge_worker.ti.finish.{job.queue}.{TaskInstanceState.FAILED}.{job.dag_id}.{job.task_id}",
- tags=tags,
- )
- Stats.incr("edge_worker.ti.finish", tags=tags)
+ if AIRFLOW_V_3_2_1_PLUS:
Review Comment:
as in the other un-resolved comment, needs to be updated
```suggestion
if AIRFLOW_V_3_2_2_PLUS:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3128980574
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
This is a bad outcome from the latest rebase. I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3125226108
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
I thought that the above code was released but this is the code from the last release of the `edge3` provider and it's using the `try-except` pattern which means that we can safely remove the `DualStatsManager` and not maintain a shim
https://github.com/apache/airflow/blob/providers-edge3/3.4.0/providers/edge3/src/airflow/providers/edge3/models/edge_worker.py#L32-L35
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011755892
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
@potiuk I updated it and added comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3081392850
##########
airflow-core/src/airflow/__init__.py:
##########
@@ -89,7 +89,6 @@
# Deprecated lazy imports
"AirflowException": (".exceptions", "AirflowException", True),
"Dataset": (".sdk", "Asset", True),
Review Comment:
This comment is invalid. This is just for `airflow-core` which has been migrated and doesn't need a deprecation process. The `Stats` shim is meant to be used by 3rd party libraries such as providers and not `airflow-core`.
Even if we were to add it here, the lazy import wouldn't make sense anymore since the metaclass has been removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154932211
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -86,7 +86,10 @@ def _executor_initializer():
try:
from airflow.observability.metrics import stats_utils
- Stats.initialize(factory=stats_utils.get_stats_factory(Stats))
+ Stats.initialize(
+ factory=stats_utils.get_stats_factory(),
+ export_legacy_names=conf.getboolean("metrics", "legacy_names_on"),
Review Comment:
Added the fallback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "kacpermuda (via GitHub)" <gi...@apache.org>.
kacpermuda commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4370832546
Just FYI, the OL provider needed to be adjusted (#66351 ) after this change, we were using `get_stats_factory`, and the removal of arg that this function expected was a breaking change. I think it was the only place changed in this PR, where `get_stats_factory` is used in a provider, so hopefully no other places will be impacted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3128980574
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
This wasn't intentional. It was a bad outcome from the latest rebase. I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3010524605
##########
airflow-core/src/airflow/dag_processing/manager.py:
##########
@@ -1179,7 +1182,7 @@ def prepare_file_queue(self, known_files: dict[str, set[DagFileInfo]]):
if self._parsing_start_time is not None:
emit_metrics(
parse_time=time.perf_counter() - self._parsing_start_time,
- stats=list(self._file_stats.values()),
+ dag_file_stats=list(self._file_stats.values()),
Review Comment:
The old `stats` param was shadowing the new `stats` module import and we were ending up with a mypy error
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
I'm not so sure about the changes in this file. I could add an import to the `Stats` shim.
Also, importing from `airflow.sdk.observability...` instead of `airflow.observability...` seemed like an oversight during a copy-paste. I don't know if it was intentional.
##########
shared/observability/src/airflow_shared/observability/metrics/datadog_logger.py:
##########
@@ -177,7 +176,7 @@ def get_dogstatsd_logger(
from datadog import DogStatsd
dogstatsd_kwargs: dict[str, Any] = {
- "constant_tags": cls.get_constant_tags(tags_in_string=tags_in_string),
Review Comment:
This used to be
```python
@classmethod
def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
"""Get constant DataDog tags to add to all stats."""
if not tags_in_string:
return []
return tags_in_string.split(",")
```
It's much simpler to include the code here directly, especially since this is the only usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011693121
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
I'll update it accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4090522501
cc @dstandish @ashb @potiuk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2960997001
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -179,70 +174,36 @@ def set_metrics(
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- if DualStatsManager is not None:
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
- else:
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
+ Stats.gauge(
+ "edge_worker.connected",
+ int(connected),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.maintenance",
+ int(maintenance),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.jobs_active",
+ jobs_active,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.concurrency",
+ concurrency,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
Review Comment:
ooh I see.
Now it's
```
if version >= 3.2:
DualStatsManager.incr(...
else:
Stats.incr(...
```
but it should be
```
if version >= 3.2:
stats.incr(...
else:
Stats.incr(...
```
and the shims in core and sdk should be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011546957
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
Hmm - but they will get "Stats not exist" error I think - not the warning ?
I do not think indeed we want to encourage using stats here, it's more of a potential compatibility issue if somene "still" uses "from airflow.stats import Stats" - in 3rd-party providers for example.
It might be fine of course, but I think in this case we should rather raise explicit error (even if we want to tell people "deprecation period ended". but even in this case we might simply remove the whole file because `Stats` was the only thing that someone wanted to use from there.
Or am I missing something ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3126994059
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
The change is now dropping the "new feature" added three days ago that additional custom metrics can be published.
Specific reason it is removed now? Because also documented in https://github.com/boschglobal/airflow/blob/main/providers/edge3/docs/deployment.rst#worker-monitoring with an example in https://github.com/boschglobal/airflow/blob/main/providers/edge3/src/airflow/providers/edge3/cli/example_extended_sysinfo.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124940640
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
I simplified the approach and we don't need any back-compat checks anymore. Everything is handled internally by stats and the public interface stays the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3116206245
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1863,8 +1863,10 @@ def main():
global SUPERVISOR_COMMS
SUPERVISOR_COMMS = CommsDecoder[ToTask, ToSupervisor](log=log)
- stats_factory = stats_utils.get_stats_factory(Stats)
- Stats.initialize(factory=stats_factory)
+ stats.initialize(
+ factory=stats_utils.get_stats_factory(),
Review Comment:
Do we ever pass anything but `factory=stats_utils.get_stats_factory()` for this arg? Any reason we shouldn't put this inside `stats.initialize` directly?
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/adapter.py:
##########
@@ -162,18 +158,18 @@ def emit(self, event: RunEvent):
transport_type = f"{self._client.transport.kind}".lower()
try:
- with ExitStack() as stack:
- if DualStatsManager is not None:
- stack.enter_context(
- DualStatsManager.timer(
- "ol.emit.attempts",
- extra_tags={"event_type": event_type, "transport_type": transport_type},
- )
- )
- else:
- stack.enter_context(Stats.timer(f"ol.emit.attempts.{event_type}.{transport_type}"))
- stack.enter_context(Stats.timer("ol.emit.attempts"))
-
+ ctx: AbstractContextManager[object]
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.emit.attempts",
+ legacy_name_tags={"event_type": event_type, "transport_type": transport_type},
Review Comment:
I'm still confused by this parameter name. Isn't aren't these tags the _non_ legacy version?
##########
task-sdk/src/airflow/sdk/observability/metrics/stats_utils.py:
##########
Review Comment:
Ditto -- duplicating shared code?
##########
airflow-core/src/airflow/cli/commands/daemon_utils.py:
##########
@@ -75,9 +75,7 @@ def run_command_with_daemon_option(
with ctx:
# in daemon context stats client needs to be reinitialized.
- from airflow._shared.observability.metrics.stats import Stats
-
- Stats.instance = None
+ # The stats module auto-detects fork via PID and re-initializes lazily.
Review Comment:
Comment only makes sense in context of the diff, after that we shouldn't keep the comment
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -204,7 +205,13 @@ def on_running():
if not doc:
doc, doc_type = get_dag_documentation(dag)
- with Stats.timer(f"ol.extract.{event_type}.{operator_name}"):
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.extract", legacy_name_tags={"event_type": event_type, "operator_name": operator_name}
+ )
+ else:
+ ctx = Stats.timer(f"ol.extract.{event_type}.{operator_name}")
+ with ctx:
Review Comment:
Could/should this difference be handled once somewhere in `airflow.providers.common.compat.sdk` perhaps?
##########
task-sdk/src/airflow/sdk/observability/metrics/datadog_logger.py:
##########
Review Comment:
I wonder why this file even exists -- isn't this a copy of what is in there shared lib? (Can be addressed in a different PR)
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,71 +71,235 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
- type.__setattr__(cls, "instance", instance)
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
- return getattr(instance, name)
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
+ return _backend
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def _get_legacy_stat(stat: str, variables: dict[str, Any]) -> str | None:
+ """
+ Look up and format the legacy name for a metric from the registry.
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+ Returns the formatted legacy name, or None if the metric has no legacy name.
+ Raises ValueError if the metric is not in the registry or required variables are missing.
+ """
+ stat_from_registry = _registry.get(name=stat)
+
+ if not stat_from_registry:
+ raise ValueError(
+ f"Metric '{stat}' not found in the registry. Add the metric to the YAML file before using it."
+ )
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ if legacy_name == "-":
+ return None
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ missing_vars = set(required_vars) - set(variables.keys())
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided variables: {sorted(variables.keys())}. "
+ f"Provide all required variables."
+ )
+
+ legacy_vars = {k: variables[k] for k in required_vars if k in variables}
+ return legacy_name.format(**legacy_vars)
+
+
+def _defined(**kwargs: Any) -> dict[str, Any]:
+ """Return only kwargs with a meaningful (non-None, non-empty) value."""
+ result = {}
+ for k, v in kwargs.items():
+ if v is None:
+ continue
+ try:
+ if len(v) == 0:
+ continue
+ except TypeError:
+ pass # Numerics and bools don't have len, always include
+ result[k] = v
+ return result
+
+
+def _merged(
+ tags: dict[str, Any] | None,
+ legacy_name_tags: dict[str, Any] | None,
+) -> dict[str, Any] | None:
+ """Merge tags and legacy_name_tags into a single dict, returning None if both are empty."""
+ if not tags and not legacy_name_tags:
+ return None
+ merged: dict[str, Any] = {}
+ if tags:
+ merged.update(tags)
+ if legacy_name_tags:
+ merged.update(legacy_name_tags)
+ return merged or None
+
+
+def _emit_legacy(stat: str, legacy_name_tags: dict[str, Any]) -> str:
+ """Return the formatted legacy stat name, raising ValueError if it has none."""
+ legacy_stat = _get_legacy_stat(stat, legacy_name_tags)
+ if legacy_stat is None:
+ raise ValueError(f"Stat '{stat}' doesn't have a legacy name registered in the YAML file.")
+ return legacy_stat
+
+
+def incr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Increment a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().incr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().incr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def decr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Decrement a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().decr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().decr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def gauge(
+ stat: str,
+ value: float,
+ rate: int | float | None = None,
+ delta: bool | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Set a gauge metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().gauge(
+ _emit_legacy(stat, legacy_name_tags), value, **_defined(rate=rate, delta=delta, tags=tags)
+ )
+ _get_backend().gauge(
+ stat, value, **_defined(rate=rate, delta=delta, tags=_merged(tags, legacy_name_tags))
+ )
+
+
+def timing(
+ stat: str,
+ dt: DeltaType,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Record a timing metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().timing(_emit_legacy(stat, legacy_name_tags), dt, **_defined(tags=tags))
+ _get_backend().timing(stat, dt, **_defined(tags=_merged(tags, legacy_name_tags)))
+
+
+def timer(
+ stat: str | None = None,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+ **kwargs,
+) -> Timer:
+ """
+ Context manager that times a block and emits a timer metric.
+
+ When ``stat`` is None, returns the backend timer directly for stopwatch
+ use (duration is available but no metric is emitted).
+
+ When ``legacy_name_tags`` is provided and legacy name export is enabled, both the
+ legacy metric name and the regular name are timed simultaneously via an
+ ExitStack. Otherwise, the backend timer is returned directly, preserving the
+ ability to access ``timer.duration`` on the returned object.
+ """
+ if stat is None:
+ return _get_backend().timer()
+
+ regular_kw: dict[str, Any] = {**kwargs}
+ merged = _merged(tags, legacy_name_tags)
+ if merged:
+ regular_kw["tags"] = merged
+
+ if _export_legacy_names and legacy_name_tags is not None:
+ legacy_kw: dict[str, Any] = {**kwargs}
+ if tags is not None:
+ legacy_kw["tags"] = tags
+
+ stack = ExitStack()
+ stack.enter_context(
+ cast(
+ "AbstractContextManager[Any]",
+ _get_backend().timer(_emit_legacy(stat, legacy_name_tags), **legacy_kw),
+ )
+ )
+ stack.enter_context(_get_backend().timer(stat, **regular_kw))
+ return cast("Timer", stack)
Review Comment:
Also this -- exit stack is not a timer, and the return should be a Timer.
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -22,18 +21,28 @@
import re
import socket
from collections.abc import Callable
-from typing import TYPE_CHECKING
+from contextlib import AbstractContextManager, ExitStack
+from typing import TYPE_CHECKING, Any, cast
from .base_stats_logger import NoStatsLogger
+from .metrics_registry import MetricsRegistry
if TYPE_CHECKING:
from .base_stats_logger import StatsLogger
+ from .protocols import DeltaType, Timer
log = logging.getLogger(__name__)
_VALID_STAT_NAME_CHARS_RE = re.compile(r"^[a-zA-Z0-9_.-]+$")
_INVALID_STAT_NAME_CHARS_RE = re.compile(r"[^a-zA-Z0-9_.-]")
+# Module-level singleton state.
+_factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
+_backend: StatsLogger | NoStatsLogger | None = None
+_instance_pid: int | None = None
+_export_legacy_names: bool = True
+_registry: MetricsRegistry = MetricsRegistry()
+
Review Comment:
@xBis7 I think this one is worth addressing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3120344678
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
Mhm, I am not sure, even if we rated `DualStatsManager` being an internal and nto public API it was used at least for one release of the providers to be be compatible with Airflow 3.2.0 and 3.2.1.
If we move this to use common-compat now this is fine... in future versions. But if the (last) Edge provider release is used with Airflow 3.2.2 then the interface is missing and breaking the provider?
How did we treat this in other cases where forward core versions against "older" providers break compatibility? Do we need to have at least some compat shim even if we decided not having this public API?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4352543472
🤞🏻
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154928708
##########
airflow-core/tests/unit/jobs/test_scheduler_job.py:
##########
@@ -436,8 +436,10 @@ def test_only_idle_with_dag_exits_after_n_idle_runs(self, caplog, dag_maker, ses
assert total_runs_val > idle_runs_val, "Some runs should not be idle"
@mock.patch("airflow.jobs.scheduler_job_runner.TaskCallbackRequest")
- @mock.patch("airflow.jobs.scheduler_job_runner.Stats.incr")
- def test_process_executor_events(self, mock_stats_incr, mock_task_callback, dag_maker):
+ @mock.patch("airflow._shared.observability.metrics.stats._get_backend")
+ def test_process_executor_events(self, mock_get_backend, mock_task_callback, dag_maker):
+ mock_stats = mock.MagicMock()
Review Comment:
Fixed. Also updated occurrences in other files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3156985864
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +78,246 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
-
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
-
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
-
- type.__setattr__(cls, "instance", instance)
-
- return getattr(instance, name)
-
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
-
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _export_legacy_names = export_legacy_names
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, creating it on first access."""
+ global _backend
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
Review Comment:
Cool, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3123141604
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/adapter.py:
##########
@@ -162,18 +158,18 @@ def emit(self, event: RunEvent):
transport_type = f"{self._client.transport.kind}".lower()
try:
- with ExitStack() as stack:
- if DualStatsManager is not None:
- stack.enter_context(
- DualStatsManager.timer(
- "ol.emit.attempts",
- extra_tags={"event_type": event_type, "transport_type": transport_type},
- )
- )
- else:
- stack.enter_context(Stats.timer(f"ol.emit.attempts.{event_type}.{transport_type}"))
- stack.enter_context(Stats.timer("ol.emit.attempts"))
-
+ ctx: AbstractContextManager[object]
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.emit.attempts",
+ legacy_name_tags={"event_type": event_type, "transport_type": transport_type},
Review Comment:
In the places where it's provided, it is then merged with the `tags` param and the values then end up as tags in the modern name.
On second thought, I think a have a simpler approach than using this param, that will help us get rid of the back compat checks for the params.
We will always use just `tags` and get the legacy name variables from the YAML. Internally, `stats` will cross-reference the provided tags with what is listed on the YAML file.
There won't be any merging of the dictionaries and the YAML will be the single source of truth which is the actual intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124956768
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -204,7 +205,13 @@ def on_running():
if not doc:
doc, doc_type = get_dag_documentation(dag)
- with Stats.timer(f"ol.extract.{event_type}.{operator_name}"):
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.extract", legacy_name_tags={"event_type": event_type, "operator_name": operator_name}
+ )
+ else:
+ ctx = Stats.timer(f"ol.extract.{event_type}.{operator_name}")
+ with ctx:
Review Comment:
Same here, I simplified the approach and we don't need any back-compat checks anymore. Everything is handled internally by stats and the public interface stays the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3120207188
##########
airflow-core/src/airflow/cli/commands/daemon_utils.py:
##########
@@ -75,9 +75,7 @@ def run_command_with_daemon_option(
with ctx:
# in daemon context stats client needs to be reinitialized.
- from airflow._shared.observability.metrics.stats import Stats
-
- Stats.instance = None
+ # The stats module auto-detects fork via PID and re-initializes lazily.
Review Comment:
It made sense to me based on the above comment
```
# in daemon context stats client needs to be reinitialized.
```
I'll remove both of them. Stats won't be used in this part of the code anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4355366492
@ashb @jscheffl Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3000354072
##########
airflow-core/src/airflow/observability/stats.py:
##########
@@ -15,8 +15,39 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Re-exports from airflow._shared.observability.metrics.stats for compatibility."""
+"""Stats module — re-exports from airflow._shared.observability.metrics.stats."""
from __future__ import annotations
-from airflow._shared.observability.metrics.stats import Stats as Stats
+from airflow._shared.observability.metrics.stats import (
+ decr,
+ gauge,
+ incr,
+ initialize,
+ normalize_name_for_stats,
+ timer,
+ timing,
+)
+
+
+class Stats:
Review Comment:
As discussed, there is a common shim under `_shared` to be used by the providers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2960971192
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -179,70 +174,36 @@ def set_metrics(
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- if DualStatsManager is not None:
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
- else:
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
+ Stats.gauge(
+ "edge_worker.connected",
+ int(connected),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.maintenance",
+ int(maintenance),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.jobs_active",
+ jobs_active,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.concurrency",
+ concurrency,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
Review Comment:
why do we use Stats in some places and stats in others?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3149791441
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +78,246 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
-
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
-
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
-
- type.__setattr__(cls, "instance", instance)
-
- return getattr(instance, name)
-
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
-
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _export_legacy_names = export_legacy_names
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, creating it on first access."""
+ global _backend
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
Review Comment:
Else I hope this is merged :-D soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3089125502
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
Are you sure this is getting 3_2_plus? Meanwhile this needs to be 3_3_PLUS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3119912154
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -22,18 +21,28 @@
import re
import socket
from collections.abc import Callable
-from typing import TYPE_CHECKING
+from contextlib import AbstractContextManager, ExitStack
+from typing import TYPE_CHECKING, Any, cast
from .base_stats_logger import NoStatsLogger
+from .metrics_registry import MetricsRegistry
if TYPE_CHECKING:
from .base_stats_logger import StatsLogger
+ from .protocols import DeltaType, Timer
log = logging.getLogger(__name__)
_VALID_STAT_NAME_CHARS_RE = re.compile(r"^[a-zA-Z0-9_.-]+$")
_INVALID_STAT_NAME_CHARS_RE = re.compile(r"[^a-zA-Z0-9_.-]")
+# Module-level singleton state.
+_factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
+_backend: StatsLogger | NoStatsLogger | None = None
+_instance_pid: int | None = None
+_export_legacy_names: bool = True
+_registry: MetricsRegistry = MetricsRegistry()
+
Review Comment:
Yes, you are right. This is a valid point and I made the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3125008033
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1863,8 +1863,10 @@ def main():
global SUPERVISOR_COMMS
SUPERVISOR_COMMS = CommsDecoder[ToTask, ToSupervisor](log=log)
- stats_factory = stats_utils.get_stats_factory(Stats)
- Stats.initialize(factory=stats_factory)
+ stats.initialize(
+ factory=stats_utils.get_stats_factory(),
Review Comment:
The issue is that any code under `shared` doesn't have access to the airflow conf. So, both `airflow-core` and `task-sdk` need to read the config, get the correct values and use them to initialize `stats` which live under `shared`.
`stats_utils.get_stats_factory()` reads conf, initializes the correct factory and then passes it to `stats`.
That's the reason why there is code duplication in these directories
https://github.com/xBis7/airflow/tree/simplify-stats/airflow-core/src/airflow/observability/metrics
https://github.com/xBis7/airflow/tree/simplify-stats/task-sdk/src/airflow/sdk/observability/metrics
For example,
https://github.com/xBis7/airflow/blob/simplify-stats/airflow-core/src/airflow/observability/metrics/otel_logger.py
and
https://github.com/xBis7/airflow/blob/simplify-stats/task-sdk/src/airflow/sdk/observability/metrics/otel_logger.py
All the files are the same and the only difference is in the imports.
##########
task-sdk/src/airflow/sdk/observability/metrics/datadog_logger.py:
##########
Review Comment:
Same explanation
https://github.com/apache/airflow/pull/63932#discussion_r3125008033
##########
task-sdk/src/airflow/sdk/observability/metrics/stats_utils.py:
##########
Review Comment:
Same explanation
https://github.com/apache/airflow/pull/63932#discussion_r3125008033
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3137885950
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +72,239 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
+
+
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
+
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
+
+ return _backend
Review Comment:
The PID check on every call to `_get_backend()` could be replaced by `os.register_at_fork(after_in_child=...)` that resets `_backend = None`. Then `_get_backend()` simplifies to just `if _backend is None` — no PID tracking needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3130586928
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
I added it back. Because it's custom and dynamic there is no way of adding it in the registry. I think there won't be an issue. It won't be found in the registry and therefore the legacy check will be skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124947423
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/adapter.py:
##########
@@ -162,18 +158,18 @@ def emit(self, event: RunEvent):
transport_type = f"{self._client.transport.kind}".lower()
try:
- with ExitStack() as stack:
- if DualStatsManager is not None:
- stack.enter_context(
- DualStatsManager.timer(
- "ol.emit.attempts",
- extra_tags={"event_type": event_type, "transport_type": transport_type},
- )
- )
- else:
- stack.enter_context(Stats.timer(f"ol.emit.attempts.{event_type}.{transport_type}"))
- stack.enter_context(Stats.timer("ol.emit.attempts"))
-
+ ctx: AbstractContextManager[object]
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.emit.attempts",
+ legacy_name_tags={"event_type": event_type, "transport_type": transport_type},
Review Comment:
@ashb I changed this and restored it to just use `tags`. Everything is handled internally by `stats` based on the metrics registry entries. No back-compat checks are needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124936675
##########
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py:
##########
@@ -225,11 +226,14 @@ def _update_orphaned_jobs(self, session: Session) -> bool:
"queue": job.queue,
"state": str(TaskInstanceState.FAILED),
}
- Stats.incr(
- f"edge_worker.ti.finish.{job.queue}.{TaskInstanceState.FAILED}.{job.dag_id}.{job.task_id}",
- tags=tags,
- )
- Stats.incr("edge_worker.ti.finish", tags=tags)
+ if AIRFLOW_V_3_2_1_PLUS:
Review Comment:
I simplified the approach and we don't need any back-compat checks anymore. Everything is handled internally by `stats` and the public interface stays the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3126981676
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
Oh, thanks @ashb fixing it prior release. Totally over-looked it was not released! So the reversion of the handling is great! Problem solved!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3155002859
##########
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml:
##########
@@ -564,8 +564,8 @@ metrics:
- name: "dag_processing.last_duration"
description: "Milliseconds taken to load the given Dag file"
Review Comment:
If you see the diffs, I changed the docs to match the code. The metric stayed the same but only the docs were updated.
https://github.com/apache/airflow/pull/63932/changes/BASE..c0d10c315b15a28da841ae9fe0a2ed3f311d884d#diff-ab04d49f916c0bd418143e70f1c1ac6b335e8a98526c4a705404b76f3c9cdb93L1518-R1524
The metric on the docs didn't exist and therefore we aren't breaking any workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4100031263
> can you summarize the potential backcompat issues?
@dstandish Any potential backcompat issues are with the providers. Because they have different versioning, the latest version could run against some version of `airflow-core` or `task-sdk` prior to 3.2, in which case they will need to import and use the old `Stats...` class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2967662514
##########
airflow-core/src/airflow/observability/stats.py:
##########
@@ -15,8 +15,39 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Re-exports from airflow._shared.observability.metrics.stats for compatibility."""
+"""Stats module — re-exports from airflow._shared.observability.metrics.stats."""
from __future__ import annotations
-from airflow._shared.observability.metrics.stats import Stats as Stats
+from airflow._shared.observability.metrics.stats import (
+ decr,
+ gauge,
+ incr,
+ initialize,
+ normalize_name_for_stats,
+ timer,
+ timing,
+)
+
+
+class Stats:
Review Comment:
I removed this shim.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3093615189
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
On second thought, `AIRFLOW_V_3_2_1_PLUS` will also work. Target version will in any case be `>= 3.2.1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3123202414
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
It was discussed on a previous community call and the conclusion was that if the pattern is
```
try:
...
except ImportError:
...
```
then we are safe to remove it. But if the pattern is
```
if AIRFLOW_V_X_X_PLUS:
...
else:
...
```
then we are locked and we need to support it for 3 minor releases after the version on the check.
During the time of the call, it was with the `try-except` pattern but now I guess we have to maintain it somehow.
Maybe it will be a no-op shim that will underneath call the new code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124926495
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
Hey @jscheffl, I added back the `dual_stats_manager.py` in the same location that it existed. It still gets re-exported from the sdk and can be accessed from the old import. The file has been converted into a shim that calls the new interface.
It was released in the edge3 provider 3.4.0 with a version check
```
if AIRFLOW_V_3_2_PLUS:
```
which means that as long as that provider version is used, it will expect the `DualStatsManager` to exist for any version of `airflow-core` >= 3.2.0. I think the shim can only be removed once the edge3 3.4.0 version stops being supported.
https://github.com/xBis7/airflow/blob/8ea0a2a065213f0a9b7c0c6779501dc3fcf9995d/shared/observability/src/airflow_shared/observability/metrics/dual_stats_manager.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3125311447
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
https://github.com/apache/airflow/pull/65676 to remove it again from Edge provider.
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -180,90 +180,32 @@ def set_metrics(
"free_concurrency",
}
- if AIRFLOW_V_3_2_PLUS:
Review Comment:
https://github.com/apache/airflow/pull/65676 to remove it again from Edge provider as it hasn't made it to any release yet, so by doing it the way it was before Jens's latest PR we remain free to remove DualStatsManager when we want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154612370
##########
airflow-core/src/airflow/plugins_manager.py:
##########
@@ -128,7 +128,8 @@ def __register_plugins(plugin_instances: list[AirflowPlugin], errors: dict[str,
if not settings.LAZY_LOAD_PROVIDERS:
__register_plugins(*_load_providers_plugins())
- log.debug("Loading %d plugin(s) took %.2f ms", len(plugins), timer.duration)
+ if hasattr(timer, "duration"):
Review Comment:
Now we are returning a `TimerProtocol` which doesn't have `duration`. Currently, `timer` is always of type `Timer` and `_DualTimer` which have a duration but the check is just to be safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154935821
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -86,7 +86,10 @@ def _executor_initializer():
try:
from airflow.observability.metrics import stats_utils
- Stats.initialize(factory=stats_utils.get_stats_factory(Stats))
+ Stats.initialize(
+ factory=stats_utils.get_stats_factory(),
+ export_legacy_names=conf.getboolean("metrics", "legacy_names_on"),
Review Comment:
If the config doesn't exist and there is a fallback, then the fallback will be used without an error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb merged PR #63932:
URL: https://github.com/apache/airflow/pull/63932
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2960967041
##########
airflow-core/src/airflow/observability/stats.py:
##########
@@ -15,8 +15,39 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Re-exports from airflow._shared.observability.metrics.stats for compatibility."""
+"""Stats module — re-exports from airflow._shared.observability.metrics.stats."""
from __future__ import annotations
-from airflow._shared.observability.metrics.stats import Stats as Stats
+from airflow._shared.observability.metrics.stats import (
+ decr,
+ gauge,
+ incr,
+ initialize,
+ normalize_name_for_stats,
+ timer,
+ timing,
+)
+
+
+class Stats:
Review Comment:
if this is private, then we don't need backcompat do we?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4158071136
I keep getting random CI failures on the step that prepares the breeze and CI image. It's always the same error
```
The image /mnt/ci-image-save-v3-linux_amd64-3.10.tar does not exist.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011350713
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
@potiuk In case anyone imported
```python
from airflow.stats import Stats
```
he would get a warning that `Stats` has been moved and that
```python
from airflow.observability.stats import Stats
```
should be used instead.
But we only maintain `Stats` as a shim to be used in providers. We don't want to encourage using it in core or sdk. I don't think anyone in core will try to import
```python
from airflow import stats
```
to get the deprecation warning to use
```python
from airflow.observability import stats
```
These are all new imports. So, I'm wondering whether this file should be removed entirely. Unless there is a backwards compatibility scenario here that I'm missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3149770619
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +78,246 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
-
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
-
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
-
- type.__setattr__(cls, "instance", instance)
-
- return getattr(instance, name)
-
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
-
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _export_legacy_names = export_legacy_names
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, creating it on first access."""
+ global _backend
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
Review Comment:
One special thing that we carry as still using StatsD - we have (multiple times) seen the regression that the `.` is the separator in StatsD but also a `.`is used in the task_id if the task is in a task group. We usually patch locally but until then fixed again... can we make a replacement of `.` to `__` for task_id keys?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3153576059
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +78,246 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
-
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
-
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
-
- type.__setattr__(cls, "instance", instance)
-
- return getattr(instance, name)
-
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
-
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _export_legacy_names = export_legacy_names
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, creating it on first access."""
+ global _backend
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
Review Comment:
I see, so for a `task_id` = `my_group.my_task`, it will become `my_group__my_task`.
It makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154564171
##########
airflow-core/tests/unit/jobs/test_scheduler_job.py:
##########
@@ -436,8 +436,10 @@ def test_only_idle_with_dag_exits_after_n_idle_runs(self, caplog, dag_maker, ses
assert total_runs_val > idle_runs_val, "Some runs should not be idle"
@mock.patch("airflow.jobs.scheduler_job_runner.TaskCallbackRequest")
- @mock.patch("airflow.jobs.scheduler_job_runner.Stats.incr")
- def test_process_executor_events(self, mock_stats_incr, mock_task_callback, dag_maker):
+ @mock.patch("airflow._shared.observability.metrics.stats._get_backend")
+ def test_process_executor_events(self, mock_get_backend, mock_task_callback, dag_maker):
+ mock_stats = mock.MagicMock()
Review Comment:
`mock.MagicMock()` without a `spec` silently accepts any attribute or call — if production code calls a misspelled method the test still passes.
I think we can use `spec=StatsLogger` (from `airflow._shared.observability.metrics.base_stats_logger`). Same issue on the other ~10 sites in this file that follow the same pattern.
##########
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml:
##########
@@ -564,8 +564,8 @@ metrics:
- name: "dag_processing.last_duration"
description: "Milliseconds taken to load the given Dag file"
Review Comment:
The legacy name format for `dag_processing.last_duration` changed from `dag_processing.last_duration.{dag_file}` to `dag_processing.last_duration.{bundle_name}.{file_name}`. Anyone currently alerting or dashboarding on the old format (e.g. `dag_processing.last_duration.my_dag_file`) will silently lose those metrics after upgrade — the old name simply stops being emitted. Worth a note in the migration docs or a deprecation warning before the hard switch.
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -86,7 +86,10 @@ def _executor_initializer():
try:
from airflow.observability.metrics import stats_utils
- Stats.initialize(factory=stats_utils.get_stats_factory(Stats))
+ Stats.initialize(
+ factory=stats_utils.get_stats_factory(),
+ export_legacy_names=conf.getboolean("metrics", "legacy_names_on"),
Review Comment:
```suggestion
export_legacy_names=conf.getboolean("metrics", "legacy_names_on", fallback=True),
```
(Not sure about the fallback value) but I think we need _a_ fallback in providers, otherwise if this new provider version is used against an Airflow < 3.2.0 release it will fail with a config not found exc?
##########
airflow-core/src/airflow/plugins_manager.py:
##########
@@ -128,7 +128,8 @@ def __register_plugins(plugin_instances: list[AirflowPlugin], errors: dict[str,
if not settings.LAZY_LOAD_PROVIDERS:
__register_plugins(*_load_providers_plugins())
- log.debug("Loading %d plugin(s) took %.2f ms", len(plugins), timer.duration)
+ if hasattr(timer, "duration"):
Review Comment:
This shouldn't be required anymore? I thought the protocol always has it.
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -230,7 +233,8 @@ def on_running():
if not doc:
doc, doc_type = get_dag_documentation(dag)
- with Stats.timer(f"ol.extract.{event_type}.{operator_name}"):
+ ctx = Stats.timer("ol.extract", tags={"event_type": event_type, "operator_name": operator_name})
+ with ctx:
Review Comment:
Pointless `ctx` variable? Or just a formatting affordance so it doesn't line wrap?
##########
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py:
##########
@@ -225,10 +225,6 @@ def _update_orphaned_jobs(self, session: Session) -> bool:
"queue": job.queue,
"state": str(TaskInstanceState.FAILED),
}
- Stats.incr(
- f"edge_worker.ti.finish.{job.queue}.{TaskInstanceState.FAILED}.{job.dag_id}.{job.task_id}",
- tags=tags,
- )
Stats.incr("edge_worker.ti.finish", tags=tags)
Review Comment:
Removed on purpose? (I think so, just confirming)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154612370
##########
airflow-core/src/airflow/plugins_manager.py:
##########
@@ -128,7 +128,8 @@ def __register_plugins(plugin_instances: list[AirflowPlugin], errors: dict[str,
if not settings.LAZY_LOAD_PROVIDERS:
__register_plugins(*_load_providers_plugins())
- log.debug("Loading %d plugin(s) took %.2f ms", len(plugins), timer.duration)
+ if hasattr(timer, "duration"):
Review Comment:
Now we are returning a `TimerProtocol` which doesn't have `duration`. Currently, `timer` is always of type `Timer` and `_DualTimer` which have a `duration` attribute but the check is just to be safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3066499130
##########
airflow-core/src/airflow/__init__.py:
##########
@@ -89,7 +89,6 @@
# Deprecated lazy imports
"AirflowException": (".exceptions", "AirflowException", True),
"Dataset": (".sdk", "Asset", True),
Review Comment:
Removing the top-level lazy import for `Stats` means `from airflow import Stats` will stop working, which is a backwards-incompatible change compared to earlier versions. If the goal is deprecation rather than an immediate break, consider keeping the lazy import wired to the shim and issuing a deprecation warning (similar to `airflow.stats`).
```suggestion
"Dataset": (".sdk", "Asset", True),
"Stats": (".stats", "Stats", True),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3092005987
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
Yes, this needs to be updated to `AIRFLOW_V_3_3_PLUS` but the target version needed to be bumped first.
https://github.com/apache/airflow/pull/65317
Now that the other PR is merged, I will make the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3120312522
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
Unresolving. No, 3.2.1 is going to be released (in a moment?). So you need to upgrade to name it `AIRFLOW_V_3_2_2_PLUS` assuming it will be back-ported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3133640279
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -174,125 +169,21 @@ def set_metrics(
EdgeWorkerState.MAINTENANCE_EXIT,
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- additional_keys = set(sysinfo or ()) - {
- "status",
- "airflow_version",
- "edge_provider_version",
- "python_version",
- "worker_start_time",
- "concurrency",
- "free_concurrency",
- }
-
- if DualStatsManager is not None:
- try:
- DualStatsManager.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError:
- logger.warning(
- "Failed to set metric edge_worker.status. Mapping is missing in metrics_template.yaml"
- )
-
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- try:
- DualStatsManager.gauge(
- f"edge_worker.{key}",
- value,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
- except ValueError as e:
- logger.warning(
- "Failed to set metric for key %s with value %s: %s",
- key,
- value,
- e,
- )
- else:
- from airflow.providers.common.compat.sdk import Stats
-
- Stats.gauge(f"edge_worker.status.{worker_name}", sysinfo.get("status", logging.NOTSET)) # type: ignore
- Stats.gauge(
- "edge_worker.status",
- sysinfo.get("status", logging.NOTSET), # type: ignore
- tags={"worker_name": worker_name},
- )
-
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
-
- for key in additional_keys:
- value = sysinfo.get(key)
- if isinstance(value, (int, float)):
- Stats.gauge(f"edge_worker.{key}.{worker_name}", value)
- Stats.gauge(f"edge_worker.{key}", value, tags={"worker_name": worker_name})
Review Comment:
Thanks for re-addition! Suspected it is a merge artifact :-D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4371068182
@kacpermuda Sorry for the inconvenience. I didn't put much thought into it because the back compat CI step for 3.2.1 was green.
https://github.com/apache/airflow/actions/runs/25169748334/job/73789176552?pr=63932
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3092190096
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
In certain cases, it's better to go with `try-except` while in other cases, it's easier to go with version checks.
But for this case, `try-except` is straight forward and will avoid the technical debt that the version check will create.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3092190096
##########
providers/edge3/src/airflow/providers/edge3/worker_api/routes/jobs.py:
##########
@@ -92,8 +88,8 @@ def fetch(
session.commit()
# Edge worker does not backport emitted Airflow metrics, so export some metrics
tags = {"dag_id": job.dag_id, "task_id": job.task_id, "queue": job.queue}
- if DualStatsManager is not None:
- DualStatsManager.incr("edge_worker.ti.start", tags=tags)
+ if AIRFLOW_V_3_2_PLUS:
Review Comment:
In certain cases, it's better to go with `try-except` while in other cases, it's easier to go with version checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3097397427
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,71 +71,235 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
- )
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
- if instance is None:
- if factory is None:
- factory = NoStatsLogger
- type.__setattr__(cls, "factory", factory)
- try:
- instance = factory()
- cls._instance_pid = current_pid
- except (socket.gaierror, ImportError) as e:
- log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
- instance = NoStatsLogger()
- cls._instance_pid = current_pid
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
- type.__setattr__(cls, "instance", instance)
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
- return getattr(instance, name)
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
- def initialize(cls, *, factory: Callable) -> None:
- type.__setattr__(cls, "instance", None)
- type.__setattr__(cls, "factory", factory)
+ return _backend
- @classmethod
- def get_constant_tags(cls, *, tags_in_string: str | None) -> list[str]:
- """Get constant DataDog tags to add to all stats."""
- if not tags_in_string:
- return []
- return tags_in_string.split(",")
+def _get_legacy_stat(stat: str, variables: dict[str, Any]) -> str | None:
+ """
+ Look up and format the legacy name for a metric from the registry.
-if TYPE_CHECKING:
- Stats: StatsLogger
-else:
+ Returns the formatted legacy name, or None if the metric has no legacy name.
+ Raises ValueError if the metric is not in the registry or required variables are missing.
+ """
+ stat_from_registry = _registry.get(name=stat)
+
+ if not stat_from_registry:
+ raise ValueError(
+ f"Metric '{stat}' not found in the registry. Add the metric to the YAML file before using it."
+ )
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ if legacy_name == "-":
+ return None
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ missing_vars = set(required_vars) - set(variables.keys())
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided variables: {sorted(variables.keys())}. "
+ f"Provide all required variables."
+ )
+
+ legacy_vars = {k: variables[k] for k in required_vars if k in variables}
+ return legacy_name.format(**legacy_vars)
+
+
+def _defined(**kwargs: Any) -> dict[str, Any]:
+ """Return only kwargs with a meaningful (non-None, non-empty) value."""
+ result = {}
+ for k, v in kwargs.items():
+ if v is None:
+ continue
+ try:
+ if len(v) == 0:
+ continue
+ except TypeError:
+ pass # Numerics and bools don't have len, always include
+ result[k] = v
+ return result
+
+
+def _merged(
+ tags: dict[str, Any] | None,
+ legacy_name_tags: dict[str, Any] | None,
+) -> dict[str, Any] | None:
+ """Merge tags and legacy_name_tags into a single dict, returning None if both are empty."""
+ if not tags and not legacy_name_tags:
+ return None
+ merged: dict[str, Any] = {}
+ if tags:
+ merged.update(tags)
+ if legacy_name_tags:
+ merged.update(legacy_name_tags)
+ return merged or None
+
+
+def _emit_legacy(stat: str, legacy_name_tags: dict[str, Any]) -> str:
+ """Return the formatted legacy stat name, raising ValueError if it has none."""
+ legacy_stat = _get_legacy_stat(stat, legacy_name_tags)
+ if legacy_stat is None:
+ raise ValueError(f"Stat '{stat}' doesn't have a legacy name registered in the YAML file.")
+ return legacy_stat
+
+
+def incr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Increment a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().incr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().incr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def decr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Decrement a counter metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().decr(
+ _emit_legacy(stat, legacy_name_tags), **_defined(count=count, rate=rate, tags=tags)
+ )
+ _get_backend().decr(stat, **_defined(count=count, rate=rate, tags=_merged(tags, legacy_name_tags)))
+
+
+def gauge(
+ stat: str,
+ value: float,
+ rate: int | float | None = None,
+ delta: bool | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Set a gauge metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().gauge(
+ _emit_legacy(stat, legacy_name_tags), value, **_defined(rate=rate, delta=delta, tags=tags)
+ )
+ _get_backend().gauge(
+ stat, value, **_defined(rate=rate, delta=delta, tags=_merged(tags, legacy_name_tags))
+ )
+
+
+def timing(
+ stat: str,
+ dt: DeltaType,
+ *,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+) -> None:
+ """Record a timing metric."""
+ if _export_legacy_names and legacy_name_tags is not None:
+ _get_backend().timing(_emit_legacy(stat, legacy_name_tags), dt, **_defined(tags=tags))
+ _get_backend().timing(stat, dt, **_defined(tags=_merged(tags, legacy_name_tags)))
+
+
+def timer(
+ stat: str | None = None,
+ tags: dict[str, Any] | None = None,
+ legacy_name_tags: dict[str, Any] | None = None,
+ **kwargs,
+) -> Timer:
+ """
+ Context manager that times a block and emits a timer metric.
+
+ When ``stat`` is None, returns the backend timer directly for stopwatch
+ use (duration is available but no metric is emitted).
+
+ When ``legacy_name_tags`` is provided and legacy name export is enabled, both the
+ legacy metric name and the regular name are timed simultaneously via an
+ ExitStack. Otherwise, the backend timer is returned directly, preserving the
+ ability to access ``timer.duration`` on the returned object.
+ """
+ if stat is None:
+ return _get_backend().timer()
+
+ regular_kw: dict[str, Any] = {**kwargs}
+ merged = _merged(tags, legacy_name_tags)
+ if merged:
+ regular_kw["tags"] = merged
+
+ if _export_legacy_names and legacy_name_tags is not None:
+ legacy_kw: dict[str, Any] = {**kwargs}
+ if tags is not None:
+ legacy_kw["tags"] = tags
+
+ stack = ExitStack()
+ stack.enter_context(
+ cast(
+ "AbstractContextManager[Any]",
+ _get_backend().timer(_emit_legacy(stat, legacy_name_tags), **legacy_kw),
+ )
+ )
+ stack.enter_context(_get_backend().timer(stat, **regular_kw))
+ return cast("Timer", stack)
Review Comment:
In the `legacy_name_tags` branch, `timer()` returns an `ExitStack` cast to `Timer`, but `ExitStack` doesn’t implement the `Timer`/`TimerProtocol` API (`start()`, `stop()`, `duration`). This can cause runtime errors if a caller uses the stopwatch-style API (not just `with`). Consider returning a real `Timer`-like wrapper (or widening the return type and updating docs) so the returned object’s API matches the annotation.
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/adapter.py:
##########
@@ -162,18 +158,18 @@ def emit(self, event: RunEvent):
transport_type = f"{self._client.transport.kind}".lower()
try:
- with ExitStack() as stack:
- if DualStatsManager is not None:
- stack.enter_context(
- DualStatsManager.timer(
- "ol.emit.attempts",
- extra_tags={"event_type": event_type, "transport_type": transport_type},
- )
- )
- else:
- stack.enter_context(Stats.timer(f"ol.emit.attempts.{event_type}.{transport_type}"))
- stack.enter_context(Stats.timer("ol.emit.attempts"))
-
+ ctx: AbstractContextManager[object]
+ if AIRFLOW_V_3_2_1_PLUS:
+ ctx = Stats.timer(
+ "ol.emit.attempts",
+ legacy_name_tags={"event_type": event_type, "transport_type": transport_type},
+ )
+ else:
+ stack = ExitStack()
+ stack.enter_context(Stats.timer(f"ol.emit.attempts.{event_type}.{transport_type}"))
+ stack.enter_context(Stats.timer("ol.emit.attempts"))
+ ctx = stack
+ with ctx:
Review Comment:
In the Airflow < 3.2.1 branch, contexts are entered on an `ExitStack` before the stack itself is used as a context manager (`with ctx:`). If any exception occurs while adding contexts (or between construction and `with`), already-entered contexts won’t be cleaned up. This is safer (and simpler) if structured as `with ExitStack() as stack:` and all `enter_context(...)` calls occur inside that block, or by unconditionally using an `ExitStack` for both branches.
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -22,18 +21,28 @@
import re
import socket
from collections.abc import Callable
-from typing import TYPE_CHECKING
+from contextlib import AbstractContextManager, ExitStack
+from typing import TYPE_CHECKING, Any, cast
from .base_stats_logger import NoStatsLogger
+from .metrics_registry import MetricsRegistry
if TYPE_CHECKING:
from .base_stats_logger import StatsLogger
+ from .protocols import DeltaType, Timer
log = logging.getLogger(__name__)
_VALID_STAT_NAME_CHARS_RE = re.compile(r"^[a-zA-Z0-9_.-]+$")
_INVALID_STAT_NAME_CHARS_RE = re.compile(r"[^a-zA-Z0-9_.-]")
+# Module-level singleton state.
+_factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
+_backend: StatsLogger | NoStatsLogger | None = None
+_instance_pid: int | None = None
+_export_legacy_names: bool = True
+_registry: MetricsRegistry = MetricsRegistry()
+
Review Comment:
`_registry` is instantiated at import time (`MetricsRegistry()` reads/parses `metrics_template.yaml` and imports PyYAML). This introduces import-time file I/O and extra import overhead even when legacy-name export is disabled and no metrics are emitted. Consider lazily creating the registry only when legacy-name formatting is actually needed (e.g., on first `_get_legacy_stat` call, or during `initialize()` when `export_legacy_names=True`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #63932:
URL: https://github.com/apache/airflow/pull/63932#issuecomment-4356108682
Wohoo! landed!
#protm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3011350713
##########
airflow-core/src/airflow/stats.py:
##########
@@ -15,17 +15,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Deprecated module - Stats moved to airflow.observability.stats but we have to retain compat."""
+"""Deprecated module - stats moved to airflow.observability.stats."""
from __future__ import annotations
import warnings
-from airflow.sdk.observability.stats import Stats as Stats
Review Comment:
@potiuk In case anyone imported
```python
from airflow.stats import Stats
```
he would get a warning that `Stats` has been moved and that
```python
from airflow.observability.stats import Stats
```
should be used instead.
But we only maintain `Stats` as a shim to be used in providers. We don't want to encourage using it in core or sdk. I don't think anyone in core will try to import
```python
from airflow import stats
```
to get the deprecation warning to use
```python
from airflow.observability import stats
```
These are all new imports. So, I'm wondering whether this file should be removed entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3148600576
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +72,239 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
+
+
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
+
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
+
+ return _backend
+
+
+def _get_registry() -> MetricsRegistry:
+ """Initialize the registry on first use to avoid import-time file I/O."""
+ global _registry
+ if _registry is None:
+ _registry = MetricsRegistry()
+ return _registry
+
+
+def _get_legacy_stat_name_and_tags(
+ stat: str, tags: dict[str, Any] | None
+) -> tuple[str | None, dict[str, Any]]:
+ """
+ Look up and format the legacy name for a metric from the registry.
+
+ Returns (formatted_name, extra_tags) where formatted_name is None when there
+ is no legacy stat to emit, and extra_tags are the tags not consumed as name
+ variables. Raises ValueError if required name variables are missing from tags.
+ """
+ _none: tuple[None, dict[str, Any]] = None, {}
+
+ # If the config flag is enabled/disabled.
+ if not _export_legacy_names:
+ return _none
+
+ stat_from_registry = _get_registry().get(name=stat)
+
+ # If the provided stat exists in the registry.
+ if not stat_from_registry:
+ return _none
+
+ legacy_name = stat_from_registry.get("legacy_name", "-")
+
+ # If the registry stat has a legacy name.
+ if legacy_name == "-":
+ return _none
+
+ required_vars = stat_from_registry.get("name_variables", [])
+ provided_vars = set(tags.keys()) if tags else set()
+ missing_vars = set(required_vars) - provided_vars
+ # If there are specified variables in the YAML file that haven't been provided in the tags param.
+ if missing_vars:
+ raise ValueError(
+ f"Missing required variables for metric '{stat}': {sorted(missing_vars)}. "
+ f"Required variables found in the registry: {required_vars}. "
+ f"Provided tags: {sorted(provided_vars)}. "
+ f"Provide all required variables as tags."
+ )
+
+ # 'required_vars' are the ones found in the registry.
+ # If there is a variable that exists in the tags and not in the registry,
+ # then it's extra, and it will be set as a tag for the legacy stat.
+ name_var_set = set(required_vars)
+ formatted_name = legacy_name.format(**{k: tags[k] for k in required_vars}) # type: ignore[index]
+ extra_tags = {k: v for k, v in (tags or {}).items() if k not in name_var_set}
+ return formatted_name, extra_tags
+
+
+def _defined(**kwargs: Any) -> dict[str, Any]:
+ """Return only kwargs with a meaningful (non-None, non-empty) value."""
+ result = {}
+ for k, v in kwargs.items():
+ if v is None:
+ continue
+ try:
+ if len(v) == 0:
+ continue
+ except TypeError:
+ pass # Numerics and bools don't have len, always include
+ result[k] = v
+ return result
+
+
+def incr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Increment a counter metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().incr(legacy_name, **_defined(count=count, rate=rate, tags=extra_tags))
+ _get_backend().incr(stat, **_defined(count=count, rate=rate, tags=tags))
+
+
+def decr(
+ stat: str,
+ count: int | None = None,
+ rate: int | float | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Decrement a counter metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().decr(legacy_name, **_defined(count=count, rate=rate, tags=extra_tags))
+ _get_backend().decr(stat, **_defined(count=count, rate=rate, tags=tags))
+
+
+def gauge(
+ stat: str,
+ value: float,
+ rate: int | float | None = None,
+ delta: bool | None = None,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Set a gauge metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().gauge(legacy_name, value, **_defined(rate=rate, delta=delta, tags=extra_tags))
+ _get_backend().gauge(stat, value, **_defined(rate=rate, delta=delta, tags=tags))
+
+
+def timing(
+ stat: str,
+ dt: DeltaType,
+ *,
+ tags: dict[str, Any] | None = None,
+) -> None:
+ """Record a timing metric."""
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ _get_backend().timing(legacy_name, dt, **_defined(tags=extra_tags))
+ _get_backend().timing(stat, dt, **_defined(tags=tags))
+
+
+def timer(
+ stat: str | None = None,
+ tags: dict[str, Any] | None = None,
+ **kwargs,
+) -> Timer:
+ """
+ Context manager that times a block and emits a timer metric.
+
+ When ``stat`` is None, returns the backend timer directly for stopwatch
+ use (duration is available but no metric is emitted).
+
+ When legacy name export is enabled and the metric has a legacy name in the
+ registry, both the legacy metric name and the regular name are timed
+ simultaneously via an ExitStack. Otherwise, the backend timer is returned
+ directly, preserving the ability to access ``timer.duration`` on the returned object.
+ """
+ if stat is None:
+ return _get_backend().timer()
+
+ regular_kw: dict[str, Any] = {**kwargs}
+ if tags:
+ regular_kw["tags"] = tags
+
+ legacy_name, extra_tags = _get_legacy_stat_name_and_tags(stat, tags)
+ if legacy_name is not None:
+ legacy_kw: dict[str, Any] = {**kwargs}
+ if extra_tags:
+ legacy_kw["tags"] = extra_tags
+
+ stack = ExitStack()
+ stack.enter_context(
+ cast(
+ "AbstractContextManager[Any]",
+ _get_backend().timer(legacy_name, **legacy_kw),
)
- # Setting the instance to None, will force re-initialization.
- cls.instance = None
- cls._instance_pid = None
+ )
+ real_timer = stack.enter_context(_get_backend().timer(stat, **regular_kw))
+ return _ExitStackTimer(stack, real_timer)
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3124936675
##########
providers/edge3/src/airflow/providers/edge3/executors/edge_executor.py:
##########
@@ -225,11 +226,14 @@ def _update_orphaned_jobs(self, session: Session) -> bool:
"queue": job.queue,
"state": str(TaskInstanceState.FAILED),
}
- Stats.incr(
- f"edge_worker.ti.finish.{job.queue}.{TaskInstanceState.FAILED}.{job.dag_id}.{job.task_id}",
- tags=tags,
- )
- Stats.incr("edge_worker.ti.finish", tags=tags)
+ if AIRFLOW_V_3_2_1_PLUS:
Review Comment:
I simplified the approach and we don't need any back-compat checks anymore. Everything is handled internally by `stats` and the public interface stayed the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3139637275
##########
shared/observability/src/airflow_shared/observability/metrics/stats.py:
##########
@@ -62,72 +72,239 @@ def normalize_name_for_stats(name: str, log_warning: bool = True) -> str:
return normalized
-class _Stats(type):
- factory: Callable[[], StatsLogger | NoStatsLogger] | None = None
- instance: StatsLogger | NoStatsLogger | None = None
- _instance_pid: int | None = None
-
- def __getattr__(cls, name: str) -> str:
- factory = type.__getattribute__(cls, "factory")
- instance = type.__getattribute__(cls, "instance")
-
- # When using OpenTelemetry, some subprocesses are short-lived and
- # often exit before flushing any metrics.
- #
- # The solution is to register a hook that performs a force flush at exit.
- # The atexit hook is registered when initializing the instance.
- #
- # The instance gets initialized once per process. In case a process is forked, then
- # the new subprocess, will inherit the already initialized instance of the parent process.
- #
- # Store the instance pid so that it can be compared with the current pid
- # to decide whether to initialize the instance again or not.
- #
- # So far, all forks are resetting their state to remove anything inherited by the parent.
- # But in the future that might not always be true.
- current_pid = os.getpid()
- if cls.instance and cls._instance_pid != current_pid:
- log.info(
- "Stats instance was created in PID %s but accessed in PID %s. Re-initializing.",
- cls._instance_pid,
- current_pid,
+def initialize(
+ *,
+ factory: Callable[[], StatsLogger | NoStatsLogger],
+ export_legacy_names: bool,
+) -> None:
+ """Initialize the stats module with a backend factory and legacy name configuration."""
+ global _factory, _backend, _instance_pid, _export_legacy_names
+ _factory = factory
+ _backend = None
+ _instance_pid = None
+ _export_legacy_names = export_legacy_names
+
+
+def _get_backend() -> StatsLogger | NoStatsLogger:
+ """Return the current backend, re-initializing if the process has been forked."""
+ global _backend, _instance_pid
+ current_pid = os.getpid()
+
+ if _backend is not None and _instance_pid != current_pid:
+ log.info(
+ "Stats backend was created in PID %s but accessed in PID %s. Re-initializing.",
+ _instance_pid,
+ current_pid,
+ )
+ _backend = None
+ _instance_pid = None
+
+ if _backend is None:
+ factory = _factory if _factory is not None else NoStatsLogger
+ try:
+ _backend = factory()
+ _instance_pid = current_pid
+ except (socket.gaierror, ImportError) as e:
+ log.error("Could not configure StatsClient: %s, using NoStatsLogger instead.", e)
+ _backend = NoStatsLogger()
+ _instance_pid = current_pid
+
+ return _backend
Review Comment:
Good point! I made the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3155002859
##########
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml:
##########
@@ -564,8 +564,8 @@ metrics:
- name: "dag_processing.last_duration"
description: "Milliseconds taken to load the given Dag file"
Review Comment:
If you see the diffs, I changed the docs to match the code. The metric stayed the same but only the docs were updated.
https://github.com/apache/airflow/pull/63932/changes/BASE..c0d10c315b15a28da841ae9fe0a2ed3f311d884d#diff-ab04d49f916c0bd418143e70f1c1ac6b335e8a98526c4a705404b76f3c9cdb93L1518-R1524
The old metric in the docs didn't exist and therefore we aren't breaking any workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154930789
##########
providers/openlineage/src/airflow/providers/openlineage/plugins/listener.py:
##########
@@ -230,7 +233,8 @@ def on_running():
if not doc:
doc, doc_type = get_dag_documentation(dag)
- with Stats.timer(f"ol.extract.{event_type}.{operator_name}"):
+ ctx = Stats.timer("ol.extract", tags={"event_type": event_type, "operator_name": operator_name})
+ with ctx:
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r3154712317
##########
shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml:
##########
@@ -564,8 +564,8 @@ metrics:
- name: "dag_processing.last_duration"
description: "Milliseconds taken to load the given Dag file"
Review Comment:
There was a mismatch between the code and the docs and I didn't know which one to change, so I went with what made more sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove the DualStatsManager and the Stats interfaces [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #63932:
URL: https://github.com/apache/airflow/pull/63932#discussion_r2960981408
##########
providers/edge3/src/airflow/providers/edge3/models/edge_worker.py:
##########
@@ -179,70 +174,36 @@ def set_metrics(
EdgeWorkerState.OFFLINE_MAINTENANCE,
)
- if DualStatsManager is not None:
- DualStatsManager.gauge(
- "edge_worker.connected",
- int(connected),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.maintenance",
- int(maintenance),
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.jobs_active",
- jobs_active,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.concurrency",
- concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.free_concurrency",
- free_concurrency,
- tags={},
- extra_tags={"worker_name": worker_name},
- )
-
- DualStatsManager.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={},
- extra_tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
- else:
- Stats.gauge(f"edge_worker.connected.{worker_name}", int(connected))
- Stats.gauge("edge_worker.connected", int(connected), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.maintenance.{worker_name}", int(maintenance))
- Stats.gauge("edge_worker.maintenance", int(maintenance), tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.jobs_active.{worker_name}", jobs_active)
- Stats.gauge("edge_worker.jobs_active", jobs_active, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.concurrency.{worker_name}", concurrency)
- Stats.gauge("edge_worker.concurrency", concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.free_concurrency.{worker_name}", free_concurrency)
- Stats.gauge("edge_worker.free_concurrency", free_concurrency, tags={"worker_name": worker_name})
-
- Stats.gauge(f"edge_worker.num_queues.{worker_name}", len(queues))
- Stats.gauge(
- "edge_worker.num_queues",
- len(queues),
- tags={"worker_name": worker_name, "queues": ",".join(queues)},
- )
+ Stats.gauge(
+ "edge_worker.connected",
+ int(connected),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.maintenance",
+ int(maintenance),
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.jobs_active",
+ jobs_active,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
+ "edge_worker.concurrency",
+ concurrency,
+ legacy_name_tags={"worker_name": worker_name},
+ )
+ Stats.gauge(
Review Comment:
It's just for providers due to the different versioning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org