You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Kamil Bregula (Jira)" <ji...@apache.org> on 2020/01/28 19:50:00 UTC

[jira] [Comment Edited] (AIRFLOW-2906) DataDog Integration for Airflow

    [ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025395#comment-17025395 ] 

Kamil Bregula edited comment on AIRFLOW-2906 at 1/28/20 7:49 PM:
-----------------------------------------------------------------

Hook allows you to send additional metrics if you use it in your operator. This PR talks about sending StatsD metrics. [https://airflow.readthedocs.io/en/latest/metrics.html]  Airflow does not yet have direct integration with Datadog, but this can be achieved by DogStatsD - [https://docs.datadoghq.com/developers/dogstatsd/?tab=python].


was (Author: kamil.bregula):
Hook allows you to send additional metrics if someone uses it in your operator. This PR talks about sending StatsD metrics. [https://airflow.readthedocs.io/en/latest/metrics.html]  Airflow does not yet have direct integration with Datadog, but this can be achieved by DogStatsD - [https://docs.datadoghq.com/developers/dogstatsd/?tab=python].

> DataDog Integration for Airflow
> -------------------------------
>
>                 Key: AIRFLOW-2906
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2906
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: utils
>    Affects Versions: 1.8.0
>            Reporter: Austin Hsu
>            Assignee: Chandu Kavar
>            Priority: Minor
>              Labels: metrics
>
> Add functionality to Airflow to enable sending of metrics to DataDog.  DataDog provides support for tags which allows us to aggregate data more easily and visualize it.  We can utilize the [Datadog python library|https://github.com/DataDog/datadogpy] python library and the [Datadog ThreadStats module|https://datadogpy.readthedocs.io/en/latest/#datadog-threadstats-module] to send metrics directly to DataDog without needing to spin up an agent to forward the metrics.  The current implementation in 1.8 uses the statsd library to send the metrics which provides us with much less control to filter our data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)