You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by aitozi <gj...@gmail.com> on 2017/09/05 02:32:42 UTC
LatencyMarker
i am doubt that whether the backpressure will increase the latency of the
LatencyMarker? does the latency number can be used to monitor the
backpressure
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: LatencyMarker
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
So far I don't have any updates but I still want to look at this.
Best,
Aljoscha
> On 13. Sep 2017, at 11:46, Tzu-Li (Gordon) Tai <tz...@apache.org> wrote:
>
> Hi,
>
> There is actually latency metrics in the Web UI, but I think there was also some previously reported problem on that [1].
> Is there a JIRA for the patch work you did that you mentioned? If you have a fix for that we could definitely have a look at it.
>
> Looping in Aljoscha also, who I think was looking into the issue of latency metrics not showing and may have some status updates on that.
>
> Cheers,
> Gordon
>
> [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Using-latency-markers-td14791.html#a14799 <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Using-latency-markers-td14791.html#a14799>
>
> On 10 September 2017 at 8:39:46 AM, aitozi (gjying1314@gmail.com <ma...@gmail.com>) wrote:
>
>> Hi,
>>
>> You are right, i have proved that the backpressure will increase the
>> latency,and in have another question, now the value of the latency is not
>> Visualization in the dashboard，is there any plan to do this, i have add this
>> by remove the other operator and only keep the source -> end latency and
>> then i can see the latency in dashboard, if the community accept the patch
>>
>> Thanks.
>>
>> and now the
>> Tzu-Li (Gordon) Tai wrote
>> > Hi!
>> >
>> > Yes, backpressure should also increase the latency value calculated from
>> > LatencyMarkers.
>> > LatencyMarkers are special events that flow along with the actual stream
>> > records, so they should also be affected by backpressure.
>> >
>> > Are you asking because you observed otherwise?
>> >
>> > Cheers,
>> > Gordon
>> >
>> >
>> >
>> > --
>> > Sent from:
>> > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: LatencyMarker
Posted by aitozi <gj...@gmail.com>.
Hi，
I have looked into the link you posted, the dashboard shown NAN is just
because the value of the latencyGague is not numerical， so it can't be shown
in dashboard. another day i will post a pr about this .
Now i have a another pr on the issue:
https://issues.apache.org/jira/browse/FLINK-7611.
https://github.com/apache/flink/pull/4665
about the num of lost data, can you take a look at this, and give me some
suggestion,
thank you!
Aitozi
Tzu-Li (Gordon) Tai wrote
> Hi,
>
> There is actually latency metrics in the Web UI, but I think there was
> also some previously reported problem on that [1].
> Is there a JIRA for the patch work you did that you mentioned? If you have
> a fix for that we could definitely have a look at it.
>
> Looping in Aljoscha also, who I think was looking into the issue of
> latency metrics not showing and may have some status updates on that.
>
> Cheers,
> Gordon
>
> [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Using-latency-markers-td14791.html#a14799
>
> On 10 September 2017 at 8:39:46 AM, aitozi (
> gjying1314@
> ) wrote:
>
> Hi,
>
> You are right, i have proved that the backpressure will increase the
> latency,and in have another question, now the value of the latency is not
> Visualization in the dashboard，is there any plan to do this, i have add
> this
> by remove the other operator and only keep the source -> end latency and
> then i can see the latency in dashboard, if the community accept the patch
>
> Thanks.
>
> and now the
> Tzu-Li (Gordon) Tai wrote
>> Hi!
>>
>> Yes, backpressure should also increase the latency value calculated from
>> LatencyMarkers.
>> LatencyMarkers are special events that flow along with the actual stream
>> records, so they should also be affected by backpressure.
>>
>> Are you asking because you observed otherwise?
>>
>> Cheers,
>> Gordon
>>
>>
>>
>> --
>> Sent from:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: LatencyMarker
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi,
There is actually latency metrics in the Web UI, but I think there was also some previously reported problem on that [1].
Is there a JIRA for the patch work you did that you mentioned? If you have a fix for that we could definitely have a look at it.
Looping in Aljoscha also, who I think was looking into the issue of latency metrics not showing and may have some status updates on that.
Cheers,
Gordon
[1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Using-latency-markers-td14791.html#a14799
On 10 September 2017 at 8:39:46 AM, aitozi (gjying1314@gmail.com) wrote:
Hi,
You are right, i have proved that the backpressure will increase the
latency,and in have another question, now the value of the latency is not
Visualization in the dashboard，is there any plan to do this, i have add this
by remove the other operator and only keep the source -> end latency and
then i can see the latency in dashboard, if the community accept the patch
Thanks.
and now the
Tzu-Li (Gordon) Tai wrote
> Hi!
>
> Yes, backpressure should also increase the latency value calculated from
> LatencyMarkers.
> LatencyMarkers are special events that flow along with the actual stream
> records, so they should also be affected by backpressure.
>
> Are you asking because you observed otherwise?
>
> Cheers,
> Gordon
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: LatencyMarker
Posted by aitozi <gj...@gmail.com>.
Hi,
You are right, i have proved that the backpressure will increase the
latency,and in have another question, now the value of the latency is not
Visualization in the dashboard，is there any plan to do this, i have add this
by remove the other operator and only keep the source -> end latency and
then i can see the latency in dashboard, if the community accept the patch
Thanks.
and now the
Tzu-Li (Gordon) Tai wrote
> Hi!
>
> Yes, backpressure should also increase the latency value calculated from
> LatencyMarkers.
> LatencyMarkers are special events that flow along with the actual stream
> records, so they should also be affected by backpressure.
>
> Are you asking because you observed otherwise?
>
> Cheers,
> Gordon
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: LatencyMarker
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi!
Yes, backpressure should also increase the latency value calculated from
LatencyMarkers.
LatencyMarkers are special events that flow along with the actual stream
records, so they should also be affected by backpressure.
Are you asking because you observed otherwise?
Cheers,
Gordon
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/