You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by yuvraj singh <19...@gmail.com> on 2019/11/15 08:47:11 UTC
slow checkpoints
Hi all ,
I am facing one issue , when i have high back pressure my checkpoints start
failing . please let me know how to deal with this kind of situations .
Thanks
Yubraj Singh .
[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
11/15/19,
02:15:52 PM
Re: slow checkpoints
Posted by vino yang <ya...@gmail.com>.
Hi Yubraj,
So the frequent job failure is the root reason, you need to fix it. Yes,
when too many messages are squashed into the message system. If the
messages can not be consumed normally, there would exist catchup consuming
which will cause your streaming system more pressure than usual.
Best,
Vino
yuvraj singh <19...@gmail.com> 于2019年11月15日周五 下午5:25写道：
> @Congxian , back pressure is due to job failure , some times job can fail
> and we need to catch up .
>
> Thanks
> Yubraj Singh
>
> [image: Mailtrack]
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> Sender
> notified by
> Mailtrack
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> 11/15/19,
> 02:53:23 PM
>
> On Fri, Nov 15, 2019 at 2:39 PM Congxian Qiu <qc...@gmail.com>
> wrote:
>
>> Hi
>>
>> Currently, checkpoint may be faile in high back pressure scenario,
>> because the barrier alignment can't be done in expected time, you should
>> fix the back pressure problem first. There is a FLIP[1] that wants to fix
>> this issue.
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints
>> Best,
>> Congxian
>>
>>
>> yuvraj singh <19...@gmail.com> 于2019年11月15日周五 下午4:47写道：
>>
>>>
>>> Hi all ,
>>>
>>> I am facing one issue , when i have high back pressure my checkpoints
>>> start failing . please let me know how to deal with this kind of
>>> situations .
>>>
>>>
>>> Thanks
>>> Yubraj Singh .
>>>
>>>
>>> [image: Mailtrack]
>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> Sender
>>> notified by
>>> Mailtrack
>>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> 11/15/19,
>>> 02:15:52 PM
>>>
>>
Re: slow checkpoints
Posted by yuvraj singh <19...@gmail.com>.
@Congxian , back pressure is due to job failure , some times job can fail
and we need to catch up .
Thanks
Yubraj Singh
[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
11/15/19,
02:53:23 PM
On Fri, Nov 15, 2019 at 2:39 PM Congxian Qiu <qc...@gmail.com> wrote:
> Hi
>
> Currently, checkpoint may be faile in high back pressure scenario, because
> the barrier alignment can't be done in expected time, you should fix the
> back pressure problem first. There is a FLIP[1] that wants to fix this
> issue.
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints
> Best,
> Congxian
>
>
> yuvraj singh <19...@gmail.com> 于2019年11月15日周五 下午4:47写道：
>
>>
>> Hi all ,
>>
>> I am facing one issue , when i have high back pressure my checkpoints
>> start failing . please let me know how to deal with this kind of
>> situations .
>>
>>
>> Thanks
>> Yubraj Singh .
>>
>>
>> [image: Mailtrack]
>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> Sender
>> notified by
>> Mailtrack
>> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> 11/15/19,
>> 02:15:52 PM
>>
>
Re: slow checkpoints
Posted by Congxian Qiu <qc...@gmail.com>.
Hi
Currently, checkpoint may be faile in high back pressure scenario, because
the barrier alignment can't be done in expected time, you should fix the
back pressure problem first. There is a FLIP[1] that wants to fix this
issue.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints
Best,
Congxian
yuvraj singh <19...@gmail.com> 于2019年11月15日周五 下午4:47写道：
>
> Hi all ,
>
> I am facing one issue , when i have high back pressure my checkpoints
> start failing . please let me know how to deal with this kind of
> situations .
>
>
> Thanks
> Yubraj Singh .
>
>
> [image: Mailtrack]
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> Sender
> notified by
> Mailtrack
> <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&> 11/15/19,
> 02:15:52 PM
>