You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/23 07:00:55 UTC

[GitHub] [airflow] dibyo-gintaa opened a new issue #18458: Airflow deployed on Kubernetes cluster NOT showing airflow app metrics in STATSD Exporter

dibyo-gintaa opened a new issue #18458:
URL: https://github.com/apache/airflow/issues/18458


   ### Apache Airflow version
   
   2.1.1
   
   ### Operating System
   
   PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian
   
   ### Versions of Apache Airflow Providers
   
   airflow:
     extraPipPackages: 
      - apache-airflow[statsd]
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   We have deployed the airflow as a helm release in a Kubernetes cluster.
   So airflow web server is running as a pod in kubernetes cluster having statsd metrics in its values.yaml.
   
   We have tried to follow this link: https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html
   
   We also deployed statsd exporter as a helm release.
   Our flow is like this:    airflow app--> statsd config in values.yaml --> running statsd exporter where it will receive airflow metrics --> send the metrics to Prometheus server.
   We have given the follow config in airflow values.yaml:
   [metrics]
   statsd_on = True
   statsd_host = localhost
   statsd_port = 8125
   statsd_prefix = airflow
   
   
   
   ### What happened
   
   We can not see any airflow metrics running in Host: 8125 or Host:125
   When we do a local port forward of the statsd exporter pod we see only default statsd metric but not any airflow related metrics in statsd
   
   ### What you expected to happen
   
   _No response_
   
   ### How to reproduce
   
   1) Deploy airflow cluster as helm release using values.yaml
   helm install poc-cluster-airflow  airflow-stable/airflow  --namespace airflow  --version 8.4.1  --values values-custom.yaml
   values-custom.yaml having: 
    # Config settings to go into the mounted airflow.cfg
   config:
     metrics:
       #statsd_on: '{{ ternary "True" "False" .Values.statsd.enabled }}'
       statsd_on: True
       statsd_port: 8125
       statsd_prefix: airflow
       statsd_host: '{{ printf "%s-statsd" .Release.Name }}'
   
   
   2) helm --kubeconfig kube_config_cluster.yml install statsd-release prometheus-community/prometheus-statsd-exporter --namespace airflow
   
   3) Try to port forward the statsd exporter and see metrics in localhost:9102/metrics. 
   No airflow metrics is coming up
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18458: Airflow deployed on Kubernetes cluster NOT showing airflow app metrics in STATSD Exporter

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #18458:
URL: https://github.com/apache/airflow/issues/18458#issuecomment-925551609


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jedcunningham closed issue #18458: Airflow deployed on Kubernetes cluster NOT showing airflow app metrics in STATSD Exporter

Posted by GitBox <gi...@apache.org>.
jedcunningham closed issue #18458:
URL: https://github.com/apache/airflow/issues/18458


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jedcunningham closed issue #18458: Airflow deployed on Kubernetes cluster NOT showing airflow app metrics in STATSD Exporter

Posted by GitBox <gi...@apache.org>.
jedcunningham closed issue #18458:
URL: https://github.com/apache/airflow/issues/18458


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org