You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Lars Almgren Schwartz <la...@tink.com> on 2020/03/05 11:22:52 UTC

Metrics for unbounded Apache Flink pipeline

We're running a pipeline which we previously ran with a local Spark runner
but now have switched to instead use a local Flink runner. With the Spark
runner we have been getting Beam metrics by using
the org.apache.beam.runners.extensions.metrics.MetricsGraphiteSink. When we
switched over to instead using Flink, we stopped getting the metrics from
Beam. Is there a way around this problem or can someone point me in a
direction where I can find some info on how to set it up?

For some background; our set up is in Kubernetes where we have a sidecar
container running Graphite where our Beam-pipeline is running in another
container in the same pod, which is pushing metrics over localhost to the
Graphite container.

// Lars