[I] Add serviceMonitor for statsd into helm chart [airflow]

["ido177 (via GitHub)" <gi...@apache.org>]

ido177 opened a new issue, #56664:
URL: https://github.com/apache/airflow/issues/56664

   ### Official Helm Chart version
   
   1.18.0 (latest released)
   
   ### Apache Airflow version
   
   3.1.0
   
   ### Kubernetes Version
   
   1.31.6
   
   ### Helm Chart configuration
   
   _No response_
   
   ### Docker Image customizations
   
   _No response_
   
   ### What happened
   
   In the current chart version, we have the option to enable statsd, but not serviceMonitor for metrics collection. 
   
   ### What you think should happen instead
   
   Many production Airflow deployments run in Kubernetes and use Prometheus Operator as the standard for observability. We need to add the ability to enable serviceMonitor for statsd. Otherwise, collecting metrics in Prometheus becomes more difficult.
   
   ### How to reproduce
   
   Simply run the chart
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add serviceMonitor for statsd into helm chart [airflow]

["maorojloo (via GitHub)" <gi...@apache.org>]

maorojloo commented on issue #56664:
URL: https://github.com/apache/airflow/issues/56664#issuecomment-5225379089

   Hi maintainers,
   
   I would like to work on the StatsD `ServiceMonitor` part of this issue.
   
   I am currently running the official Airflow Helm chart with StatsD enabled. The chart exposes Prometheus metrics through the existing StatsD service and its `statsd-scrape` port, but Prometheus Operator users must maintain a separate `ServiceMonitor`.
   
   After reviewing the current Helm chart development guidelines, I understand that resources requiring externally managed CRDs should normally be implemented as Kustomize overlays rather than native chart templates.
   
   I propose the following scope:
   
   * Add an optional Prometheus Operator Kustomize overlay for the existing StatsD service.
   * Include a `ServiceMonitor` that selects the StatsD service using its existing labels and named scrape port.
   * Keep the base Helm chart and its defaults unchanged.
   * Do not install or manage Prometheus Operator CRDs.
   * Add validation/tests and usage documentation, including namespace and selector configuration.
   * Document the required Prometheus Operator prerequisite.
   
   Could you please confirm whether this is the preferred scope before I start implementing it? If native Helm chart support is preferred despite the current chart-routing guidelines, I can adjust the implementation accordingly.
   
   I can test this against:
   
   * Airflow Helm chart: 1.22.0
   * Kubernetes: 1.30.4
   * Prometheus Operator/kube-prometheus-stack: v0.82.2 / 72.9.0
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add serviceMonitor for statsd into helm chart [airflow]

["ido177 (via GitHub)" <gi...@apache.org>]

ido177 closed issue #56664: Add serviceMonitor for statsd into helm chart
URL: https://github.com/apache/airflow/issues/56664


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org