You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Felipe Gutierrez <fe...@gmail.com> on 2020/03/05 17:17:40 UTC
How do I get the value of 99th latency inside an operator?
Hi community,
where from the Dlink code I can get the value of 99th percentile latency
(flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_index_latency{operator_id="93352199ce18d8917f20fdf82cedb1b4",quantile="0.99"})?
Probably I will have to hack the Flink source code to export those values
to my own operator. Nevertheless, it is what I need.
Kind Regards,
Felipe
*--*
*-- Felipe Gutierrez*
*-- skype: felipe.o.gutierrez*
*--* *https://felipeogutierrez.blogspot.com
<https://felipeogutierrez.blogspot.com>*
Re: How do I get the value of 99th latency inside an operator?
Posted by Arvid Heise <ar...@ververica.com>.
Hi Felipe,
could you use the JMX metrics reporter and tap into the reported values?
The proposed hacks are obviously unstable over time.
On Fri, Mar 6, 2020 at 1:06 PM Aljoscha Krettek <al...@apache.org> wrote:
> Hi,
>
> I'm afraid you're correct, this is currently not exposed and you would
> have to hack around some things and/or use reflection.
>
> AbstractStreamOperator has a field latencyStats [1], which is what holds
> the metrics. This is being updated from method
> reportOrUpdateLatencyMarker [2].
>
> I hope that helps somewhat.
>
> Best,
> Aljoscha
>
> [1]
>
> https://github.com/apache/flink/blob/ab642cb616fb909893e2c650b0b4c2aa10407e6d/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java#L154
> [2]
>
> https://github.com/apache/flink/blob/ab642cb616fb909893e2c650b0b4c2aa10407e6d/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java#L702
>
> On 05.03.20 18:17, Felipe Gutierrez wrote:
> > Hi community,
> >
> > where from the Dlink code I can get the value of 99th percentile latency
> >
> (flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_index_latency{operator_id="93352199ce18d8917f20fdf82cedb1b4",quantile="0.99"})?
> >
> > Probably I will have to hack the Flink source code to export those values
> > to my own operator. Nevertheless, it is what I need.
> >
> > Kind Regards,
> > Felipe
> >
> > *--*
> > *-- Felipe Gutierrez*
> >
> > *-- skype: felipe.o.gutierrez*
> > *--* *https://felipeogutierrez.blogspot.com
> > <https://felipeogutierrez.blogspot.com>*
> >
>
Re: How do I get the value of 99th latency inside an operator?
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I'm afraid you're correct, this is currently not exposed and you would
have to hack around some things and/or use reflection.
AbstractStreamOperator has a field latencyStats [1], which is what holds
the metrics. This is being updated from method
reportOrUpdateLatencyMarker [2].
I hope that helps somewhat.
Best,
Aljoscha
[1]
https://github.com/apache/flink/blob/ab642cb616fb909893e2c650b0b4c2aa10407e6d/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java#L154
[2]
https://github.com/apache/flink/blob/ab642cb616fb909893e2c650b0b4c2aa10407e6d/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java#L702
On 05.03.20 18:17, Felipe Gutierrez wrote:
> Hi community,
>
> where from the Dlink code I can get the value of 99th percentile latency
> (flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_index_latency{operator_id="93352199ce18d8917f20fdf82cedb1b4",quantile="0.99"})?
>
> Probably I will have to hack the Flink source code to export those values
> to my own operator. Nevertheless, it is what I need.
>
> Kind Regards,
> Felipe
>
> *--*
> *-- Felipe Gutierrez*
>
> *-- skype: felipe.o.gutierrez*
> *--* *https://felipeogutierrez.blogspot.com
> <https://felipeogutierrez.blogspot.com>*
>