You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Gyula Fóra <gy...@gmail.com> on 2017/08/10 09:10:57 UTC
Using latency markers
Hi all!
Does anyone have a working example of using the latency markers to test for
the topology latency?
We are using Flink 1.3.2 and it seems like however we tune it, whatever job
we use all we get is NaN in the metrics. Maybe we are completely missing
something...
Thanks!
Gyula
Re: Using latency markers
Posted by Marvin777 <xy...@gmail.com>.
Hi all!
I'm having the same issues. I get NaN in the Task metrics. The version is
1.3.1.
Maybe I'm completely missing something...
There are questions about FLINK-7692, it can Solve the problem above? What
should I do in my version.
Thanks!
2017-10-11 23:30 GMT+08:00 Aljoscha Krettek <al...@apache.org>:
> This is the Jira issue: https://issues.apache.org/jira/browse/FLINK-7608
>
>
> On 27. Sep 2017, at 12:22, Martin Eden <ma...@gmail.com> wrote:
>
> Any follow-up on this? Jira? PR?
>
> On Wed, Sep 13, 2017 at 11:30 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org
> > wrote:
>
>> Hi Aitozi,
>>
>> Yes, I think we haven’t really pin-pointed out the actual cause of the
>> problem, but if you have a fix for that and can provide a PR we can
>> definitely look at it! That would be helpful.
>> Before opening a PR, also make sure to first open a JIRA for the issue (I
>> don’t think there is one yet for this issue).
>>
>> Cheers,
>> Gordon
>>
>> On 13 September 2017 at 12:14:42 PM, aitozi (gjying1314@gmail.com) wrote:
>>
>> Hi, Aljoscha,
>>
>> the dashboard shown NAN is just because the value of the latencyGague is
>> not
>> numerical， so it can't be shown in dashboard, i removed the other
>> latencydescprition except the sink, so i can see the latency in
>> dashboard,
>> do i need to post a pr?
>>
>> thanks,
>> Aitozi
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.
>> nabble.com/
>>
>>
>
>
Re: Using latency markers
Posted by Aljoscha Krettek <al...@apache.org>.
This is the Jira issue: https://issues.apache.org/jira/browse/FLINK-7608
> On 27. Sep 2017, at 12:22, Martin Eden <ma...@gmail.com> wrote:
>
> Any follow-up on this? Jira? PR?
>
> On Wed, Sep 13, 2017 at 11:30 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org <ma...@apache.org>> wrote:
> Hi Aitozi,
>
> Yes, I think we haven’t really pin-pointed out the actual cause of the problem, but if you have a fix for that and can provide a PR we can definitely look at it! That would be helpful.
> Before opening a PR, also make sure to first open a JIRA for the issue (I don’t think there is one yet for this issue).
>
> Cheers,
> Gordon
>
> On 13 September 2017 at 12:14:42 PM, aitozi (gjying1314@gmail.com <ma...@gmail.com>) wrote:
>
>> Hi, Aljoscha,
>>
>> the dashboard shown NAN is just because the value of the latencyGague is not
>> numerical， so it can't be shown in dashboard, i removed the other
>> latencydescprition except the sink, so i can see the latency in dashboard,
>> do i need to post a pr?
>>
>> thanks,
>> Aitozi
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>
Re: Using latency markers
Posted by Martin Eden <ma...@gmail.com>.
Any follow-up on this? Jira? PR?
On Wed, Sep 13, 2017 at 11:30 AM, Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:
> Hi Aitozi,
>
> Yes, I think we haven’t really pin-pointed out the actual cause of the
> problem, but if you have a fix for that and can provide a PR we can
> definitely look at it! That would be helpful.
> Before opening a PR, also make sure to first open a JIRA for the issue (I
> don’t think there is one yet for this issue).
>
> Cheers,
> Gordon
>
> On 13 September 2017 at 12:14:42 PM, aitozi (gjying1314@gmail.com) wrote:
>
> Hi, Aljoscha,
>
> the dashboard shown NAN is just because the value of the latencyGague is
> not
> numerical， so it can't be shown in dashboard, i removed the other
> latencydescprition except the sink, so i can see the latency in dashboard,
> do i need to post a pr?
>
> thanks,
> Aitozi
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/
>
>
Re: Using latency markers
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi Aitozi,
Yes, I think we haven’t really pin-pointed out the actual cause of the problem, but if you have a fix for that and can provide a PR we can definitely look at it! That would be helpful.
Before opening a PR, also make sure to first open a JIRA for the issue (I don’t think there is one yet for this issue).
Cheers,
Gordon
On 13 September 2017 at 12:14:42 PM, aitozi (gjying1314@gmail.com) wrote:
Hi, Aljoscha,
the dashboard shown NAN is just because the value of the latencyGague is not
numerical， so it can't be shown in dashboard, i removed the other
latencydescprition except the sink, so i can see the latency in dashboard,
do i need to post a pr?
thanks,
Aitozi
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Using latency markers
Posted by aitozi <gj...@gmail.com>.
Hi, Aljoscha,
the dashboard shown NAN is just because the value of the latencyGague is not
numerical， so it can't be shown in dashboard, i removed the other
latencydescprition except the sink, so i can see the latency in dashboard,
do i need to post a pr?
thanks,
Aitozi
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Using latency markers
Posted by Aljoscha Krettek <al...@apache.org>.
Ok, I can also confirm that it doesn't work. I'll look into this and will let you know what I find.
Best,
Aljoscha
> On 11. Aug 2017, at 14:57, Kien Truong <du...@gmail.com> wrote:
>
> Yes, we also tried changing the tracking interval to no avail, still no latency metric.
> Kien
>
> On 8/11/2017 7:26 PM, Gyula Fóra wrote:
>> Yes, they are enabled by default I think.
>>
>> Gyula
>>
>>
>> On Fri, Aug 11, 2017, 14:14 Aljoscha Krettek <aljoscha@apache.org <ma...@apache.org>> wrote:
>> It seems you have to enable latency tracking via ExecutionConfig.setLatencyTrackingInterval(...). This will make the sources emit latency tokens, which then in turn update the latency metric: https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking <https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking>.
>>
>> Did you enable that?
>>
>> Best,
>> Aljoscha
>>
>>
>>> On 10. Aug 2017, at 18:36, Kien Truong <duckientruong@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> I just want to say we're having the same issues.
>>>
>>> There's no metric for latency when we attempted to export the metrics through graphite either.
>>>
>>>
>>> Regards,
>>>
>>> Kien
>>>
>>>
>>> On 8/10/2017 7:36 PM, Aljoscha Krettek wrote:
>>>> Hi,
>>>>
>>>> I must admit that I've never used this but I'll try and look into it.
>>>>
>>>> Best,
>>>> Aljoscha
>>>>
>>>>> On 10. Aug 2017, at 11:10, Gyula Fóra <gyula.fora@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Hi all!
>>>>>
>>>>> Does anyone have a working example of using the latency markers to test for the topology latency?
>>>>> We are using Flink 1.3.2 and it seems like however we tune it, whatever job we use all we get is NaN in the metrics. Maybe we are completely missing something...
>>>>>
>>>>> Thanks!
>>>>> Gyula
>>>
>>
>
Re: Using latency markers
Posted by Kien Truong <du...@gmail.com>.
Yes, we also tried changing the tracking interval to no avail, still no
latency metric.
Kien
On 8/11/2017 7:26 PM, Gyula Fóra wrote:
>
> Yes, they are enabled by default I think.
>
> Gyula
>
>
> On Fri, Aug 11, 2017, 14:14 Aljoscha Krettek <aljoscha@apache.org
> <ma...@apache.org>> wrote:
>
> It seems you have to enable latency tracking
> via ExecutionConfig.setLatencyTrackingInterval(...). This will
> make the sources emit latency tokens, which then in turn update
> the latency metric:
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking.
>
>
> Did you enable that?
>
> Best,
> Aljoscha
>
>
>> On 10. Aug 2017, at 18:36, Kien Truong <duckientruong@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> I just want to say we're having the same issues.
>>
>> There's no metric for latency when we attempted to export the
>> metrics through graphite either.
>>
>>
>> Regards,
>>
>> Kien
>>
>>
>> On 8/10/2017 7:36 PM, Aljoscha Krettek wrote:
>>> Hi,
>>>
>>> I must admit that I've never used this but I'll try and look
>>> into it.
>>>
>>> Best,
>>> Aljoscha
>>>
>>>> On 10. Aug 2017, at 11:10, Gyula Fóra <gyula.fora@gmail.com
>>>> <ma...@gmail.com>> wrote:
>>>>
>>>> Hi all!
>>>>
>>>> Does anyone have a working example of using the latency markers
>>>> to test for the topology latency?
>>>> We are using Flink 1.3.2 and it seems like however we tune it,
>>>> whatever job we use all we get is NaN in the metrics. Maybe we
>>>> are completely missing something...
>>>>
>>>> Thanks!
>>>> Gyula
>>
>
Re: Using latency markers
Posted by Gyula Fóra <gy...@gmail.com>.
Yes, they are enabled by default I think.
Gyula
On Fri, Aug 11, 2017, 14:14 Aljoscha Krettek <al...@apache.org> wrote:
> It seems you have to enable latency tracking
> via ExecutionConfig.setLatencyTrackingInterval(...). This will make the
> sources emit latency tokens, which then in turn update the latency metric:
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking
> .
>
> Did you enable that?
>
> Best,
> Aljoscha
>
>
> On 10. Aug 2017, at 18:36, Kien Truong <du...@gmail.com> wrote:
>
> Hi,
>
> I just want to say we're having the same issues.
>
> There's no metric for latency when we attempted to export the metrics
> through graphite either.
>
>
> Regards,
>
> Kien
>
>
> On 8/10/2017 7:36 PM, Aljoscha Krettek wrote:
>
> Hi,
>
> I must admit that I've never used this but I'll try and look into it.
>
> Best,
> Aljoscha
>
> On 10. Aug 2017, at 11:10, Gyula Fóra <gy...@gmail.com> wrote:
>
> Hi all!
>
> Does anyone have a working example of using the latency markers to test
> for the topology latency?
> We are using Flink 1.3.2 and it seems like however we tune it, whatever
> job we use all we get is NaN in the metrics. Maybe we are completely
> missing something...
>
> Thanks!
> Gyula
>
>
>
>
Re: Using latency markers
Posted by Aljoscha Krettek <al...@apache.org>.
It seems you have to enable latency tracking via ExecutionConfig.setLatencyTrackingInterval(...). This will make the sources emit latency tokens, which then in turn update the latency metric: https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking <https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#latency-tracking>.
Did you enable that?
Best,
Aljoscha
> On 10. Aug 2017, at 18:36, Kien Truong <du...@gmail.com> wrote:
>
> Hi,
>
> I just want to say we're having the same issues.
>
> There's no metric for latency when we attempted to export the metrics through graphite either.
>
>
> Regards,
>
> Kien
>
>
> On 8/10/2017 7:36 PM, Aljoscha Krettek wrote:
>> Hi,
>>
>> I must admit that I've never used this but I'll try and look into it.
>>
>> Best,
>> Aljoscha
>>
>>> On 10. Aug 2017, at 11:10, Gyula Fóra <gy...@gmail.com> wrote:
>>>
>>> Hi all!
>>>
>>> Does anyone have a working example of using the latency markers to test for the topology latency?
>>> We are using Flink 1.3.2 and it seems like however we tune it, whatever job we use all we get is NaN in the metrics. Maybe we are completely missing something...
>>>
>>> Thanks!
>>> Gyula
>
Re: Using latency markers
Posted by Kien Truong <du...@gmail.com>.
Hi,
I just want to say we're having the same issues.
There's no metric for latency when we attempted to export the metrics
through graphite either.
Regards,
Kien
On 8/10/2017 7:36 PM, Aljoscha Krettek wrote:
> Hi,
>
> I must admit that I've never used this but I'll try and look into it.
>
> Best,
> Aljoscha
>
>> On 10. Aug 2017, at 11:10, Gyula Fóra <gy...@gmail.com> wrote:
>>
>> Hi all!
>>
>> Does anyone have a working example of using the latency markers to test for the topology latency?
>> We are using Flink 1.3.2 and it seems like however we tune it, whatever job we use all we get is NaN in the metrics. Maybe we are completely missing something...
>>
>> Thanks!
>> Gyula
Re: Using latency markers
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I must admit that I've never used this but I'll try and look into it.
Best,
Aljoscha
> On 10. Aug 2017, at 11:10, Gyula Fóra <gy...@gmail.com> wrote:
>
> Hi all!
>
> Does anyone have a working example of using the latency markers to test for the topology latency?
> We are using Flink 1.3.2 and it seems like however we tune it, whatever job we use all we get is NaN in the metrics. Maybe we are completely missing something...
>
> Thanks!
> Gyula