You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Claude M <cl...@gmail.com> on 2021/04/09 02:47:23 UTC

Flink Metrics emitted from a Kubernetes Application Cluster

Hello,

I've setup Flink as an Application Cluster in Kubernetes.  Now I'm looking
into monitoring the Flink cluster in Datadog.  This is what is configured
in the flink-conf.yaml to emit metrics:

metrics.scope.jm: flink.jobmanager
metrics.scope.jm.job: flink.jobmanager.job
metrics.scope.tm: flink.taskmanager
metrics.scope.tm.job: flink.taskmanager.job
metrics.scope.task: flink.task
metrics.scope.operator: flink.operator
metrics.reporter.dghttp.class:
org.apache.flink.metrics.datadog.DatadogHttpReporter
metrics.reporter.dghttp.apikey: {{ datadog_api_key }}
metrics.reporter.dghttp.tags: environment: {{ environment }}

When it gets to Datadog though, the metrics for the flink.jobmanager and
flink.taskmanager is filtered by the host which is the Pod IP.  However, I
would like it to use the pod name.  How can this be accomplished?


Thanks

Re: Flink Metrics emitted from a Kubernetes Application Cluster

Posted by Chesnay Schepler <ch...@apache.org>.

This is currently not possible. See also FLINK-8358

On 4/9/2021 4:47 AM, Claude M wrote:
> Hello,
>
> I've setup Flink as an Application Cluster in Kubernetes. Now I'm 
> looking into monitoring the Flink cluster in Datadog. This is what is 
> configured in the flink-conf.yaml to emit metrics:
>
> metrics.scope.jm <http://metrics.scope.jm>: flink.jobmanager
> metrics.scope.jm.job: flink.jobmanager.job
> metrics.scope.tm <http://metrics.scope.tm>: flink.taskmanager
> metrics.scope.tm.job: flink.taskmanager.job
> metrics.scope.task: flink.task
> metrics.scope.operator: flink.operator
> metrics.reporter.dghttp.class: 
> org.apache.flink.metrics.datadog.DatadogHttpReporter
> metrics.reporter.dghttp.apikey: {{ datadog_api_key }}
> metrics.reporter.dghttp.tags: environment: {{ environment }}
>
> When it gets to Datadog though, the metrics for the flink.jobmanager 
> and flink.taskmanager is filtered by the host which is the Pod IP.  
> However, I would like it to use the pod name.  How can this be 
> accomplished?
>
>
> Thanks