You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "joekohlsdorf (via GitHub)" <gi...@apache.org> on 2023/02/03 18:53:18 UTC

[GitHub] [airflow] joekohlsdorf opened a new issue, #29363: Statsd: Allow configuring custom tag for instance identification

joekohlsdorf opened a new issue, #29363:
URL: https://github.com/apache/airflow/issues/29363

   ### Description
   
   Add optional configuration option to StatsD metrics which allows an operator to configure a custom tag which is sent with all metrics to allow instance identification when running multiple Airflow instances.
   
   ### Use case/motivation
   
   When running multiple Airflow instances on the same host/K8s cluster, there is no good way to filter StatsD metrics by instance.
   The only option which currently exists is to modify the `statsd_prefix` which however makes it impossible to reuse the same dashboards / alerts for several Airflow instances in monitoring tools like Datadog.
   
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29363: Statsd: Allow configuring custom tag for instance identification

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.

boring-cyborg[bot] commented on issue #29363:
URL: https://github.com/apache/airflow/issues/29363#issuecomment-1416268791

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] joekohlsdorf commented on issue #29363: Statsd: Allow configuring custom tag for instance identification

Posted by "joekohlsdorf (via GitHub)" <gi...@apache.org>.

joekohlsdorf commented on issue #29363:
URL: https://github.com/apache/airflow/issues/29363#issuecomment-1419539560

   Already available for Datadog and not applicable for other systems.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] joekohlsdorf closed issue #29363: Statsd: Allow configuring custom tag for instance identification

Posted by "joekohlsdorf (via GitHub)" <gi...@apache.org>.

joekohlsdorf closed issue #29363: Statsd: Allow configuring custom tag for instance identification
URL: https://github.com/apache/airflow/issues/29363


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] hussein-awala commented on issue #29363: Statsd: Allow configuring custom tag for instance identification

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.

hussein-awala commented on issue #29363:
URL: https://github.com/apache/airflow/issues/29363#issuecomment-1418202085

   Indeed this is available now only for datadog metrics via the config [statsd_datadog_tags](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#statsd-datadog-tags), where for the other monitoring tools, we are using `statsd.StatsClient` or any custom client which extend it, and unfortunately, this client doesn't support tagging ([doc](https://statsd.readthedocs.io/en/v3.3/tags.html))
   > Tagged metrics—such as those used by [Datadog](https://www.datadoghq.com/) and [Telegraf](https://github.com/influxdata/telegraf)—are explicitly outside the scope of this library. [Alternatives](https://pypi.org/project/statsd-tags/) exist and are recommended. This document lays out the reasons to avoid support for tags.
   
   I'm testing alternative solutions to support tagging on all the backends in order to generalize #28961, and this will include what you are proposing. If you have any idea to do that, feel free to propose it or open a new PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org