Applying Custom metrics

[Piper Piper <pi...@gmail.com>]

Hi

I have questions regarding making my own custom metrics.

When exactly is the class RichMapFunction’s map(value) method
called/invoked, and what “value” will be passed/expected as an argument to
this map(value) method?

Does the RichMapFunction’s map() method have any relation to the
transformation map() method, or are they completely different?

Once defined, how do I put the custom metric onto a specific source,
specific operator, and onto a specific sink in my job’s DAG?

Thank you,

Piper

Re: Applying Custom metrics

[Till Rohrmann <tr...@apache.org>]

Hi Piper,

the RichMapFunction's map function is called for every record you are
processing in your DataStream/DataSet. The RichMapFunction is the
definition of the map function you are applying to every record. Hence, it
is basically what you pass to the DataStream.map(MapFunction myMapFunction)
in order to describe the transformation you want to do.

If you want to define your own metrics, then you need to define a
RichMapFunction and use its getRuntimeContext() method to call

counter = getRuntimeContext()
  .getMetricGroup()
  .addGroup("MyMetrics")
  .counter("myCounter");

in order to register a custom counter. For more information please take a
look at [1].

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#user-scope

Cheers,
Till

On Tue, Oct 6, 2020 at 1:05 AM Piper Piper <pi...@gmail.com> wrote:

> Hi
>
> I have questions regarding making my own custom metrics.
>
> When exactly is the class RichMapFunction’s map(value) method
> called/invoked, and what “value” will be passed/expected as an argument to
> this map(value) method?
>
> Does the RichMapFunction’s map() method have any relation to the
> transformation map() method, or are they completely different?
>
> Once defined, how do I put the custom metric onto a specific source,
> specific operator, and onto a specific sink in my job’s DAG?
>
> Thank you,
>
> Piper
>