[GitHub] [airflow] DibyoGit opened a new issue, #22899: Airflow not showing StatsD metrics

[GitBox <gi...@apache.org>]

DibyoGit opened a new issue, #22899:
URL: https://github.com/apache/airflow/issues/22899

   ### Apache Airflow version
   
   2.1.0
   
   ### What happened
   
   We are running the Airflow via docker-compose, connecting to a MySQL host. It hs Statd Exporter config in it but we can not see ny airflow metrics in the StatsD metrics
   
   ### What you think should happen instead
   
   StatsD metrics should have shown airlfow scheduler, web metrics
   
   ### How to reproduce
   
   Run docker compsoe fil by docker compsoe up and access statsd metrics in IP:9123 url and search for airflow metrics
   
   ### Operating System
   
   Oracle Linux 7
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   Sharing the docker-compose:
   
   version: '3'
   x-airflow-common:
     &airflow-common
     image: apache/airflow:2.1.0
     environment:
       - LOAD_EX=n
       - POSTGRES_USER=airflow
       - POSTGRES_PASSWORD=airflow
       - POSTGRES_DB=airflow
       - REDIS_HOST=redis
       - REDIS_PASSWORD=redis
       - AIRFLOW__SCHEDULER__STATSD_ON=True
       - AIRFLOW__SCHEDULER__STATSD_HOST=statsd-exporter
       - AIRFLOW__SCHEDULER__STATSD_PORT=8125
       - AIRFLOW__SCHEDULER__STATSD_PREFIX=airflow
       - AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql+mysqlconnector://airflowuserpoc:airflowpass@10.1.91.225:3306/airflow_custom_image_db
       - AIRFLOW__CELERY__BROKER_URL=redis://:redis@redis:6379/1
       - AIRFLOW__CELERY__RESULT_BACKEND=db+postgresql://airflow:airflow@postgres/airflow
       - AIRFLOW__CORE__FERNET_KEY=pMrhjIcqUNHMYRk_ZOBmMptWR6o1DahCXCKn5lEMpzM=
       - AIRFLOW__CORE__EXECUTOR=CeleryExecutor
       - AIRFLOW_HOME=/opt/airflow
       - AIRFLOW_CONFIG=/opt/airflow/airflow.cfg
       - AIRFLOW__CORE__LOAD_EXAMPLES=False
       - AIRFLOW__CORE__LOAD_DEFAULT_CONNECTIONS=False
       - AIRFLOW__WEBSERVER__WORKERS=2
       - AIRFLOW__WEBSERVER__WORKER_REFRESH_INTERVAL=1800
   
   services:
       airflow-init:
           << : *airflow-common
           container_name: airflow-init
           entrypoint: /bin/bash
           command:
           - -c
           - airflow users list || ( airflow db init &&
               airflow users create
               --role Admin
               --username admin
               --password password
               --email airflow@airflow.com
               --firstname first
               --lastname last )
           restart: on-failure
   
       webserver:
           << : *airflow-common
           container_name: airflow-webserver
           restart: always
           depends_on:
               - statsd-exporter
               - airflow-init
           ports:
               - "8080:8080"
           command: airflow webserver
           healthcheck:
               test: ["CMD-SHELL", "[ -f /opt/airflow/airflow-webserver.pid ]"]
               interval: 30s
               timeout: 30s
               retries: 3
   
       flower:
           << : *airflow-common
           container_name: airflow-flower
           restart: always
           depends_on:
               - statsd-exporter
               - airflow-init
           ports:
               - "5555:5555"
           command: airflow celery flower
   
       scheduler:
           << : *airflow-common
           container_name: airflow-scheduler
           restart: always
           depends_on:
               - webserver
               - airflow-init
           command: airflow scheduler
   
       worker:
           << : *airflow-common
           container_name: airflow-worker
           restart: always
           sysctls:
               - net.ipv4.tcp_keepalive_time=200
               - net.ipv4.tcp_keepalive_intvl=200
               - net.ipv4.tcp_keepalive_probes=5
           depends_on:
               - scheduler
               - airflow-init
           ports:
               - "8793"
           command: airflow celery worker
   
       statsd-exporter:
           image: prom/statsd-exporter
           container_name: airflow-statsd-exporter
           command: "--statsd.listen-udp=:8125 --web.listen-address=:9102"
           ports:
               - 9123:9102
               - 8125:8125/udp
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #22899: Airflow not showing StatsD metrics

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #22899:
URL: https://github.com/apache/airflow/issues/22899#issuecomment-1094710514

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org