You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Alexis Sarda-Espinosa <sa...@gmail.com> on 2022/11/16 14:39:28 UTC
Savepoint restore mode for the Kubernetes operator
Hello,
Is there a recommended configuration for the restore mode of jobs managed
by the operator?
Since the documentation states that the operator keeps a savepoint history
to perform cleanup, I imagine restore mode should always be NO_CLAIM, but I
just want to confirm.
Regards,
Alexis.
Re: Savepoint restore mode for the Kubernetes operator
Posted by Alexis Sarda-Espinosa <sa...@gmail.com>.
Just to be clear, I don't think the operator must have special logic to
find out if a savepoint was used as base for an incremental checkpoint,
however, the operator logic might want to completely disable savepoint
cleanup for a deployment if the user enabled CLAIM mode for it. At least
that sounds like the safer option to me.
Regards,
Alexis.
On Tue, 29 Nov 2022, 10:31 Gyula Fóra, <gy...@gmail.com> wrote:
> The operator might call dispose on an old savepoint that’s true, but I am
> not sure if the dispose api call would actually corrupt it.
>
> Gyula
>
> On Tue, 29 Nov 2022 at 09:28, Alexis Sarda-Espinosa <
> sarda.espinosa@gmail.com> wrote:
>
>> Hi Hangxiang,
>>
>> but, if I understand correctly, setting restore mode to CLAIM means that
>> the job might create a new incremental checkpoint based on the savepoint,
>> right? And if the operator then decides to clean up the savepoint, the
>> checkpoint would be corrupted, no?
>>
>> Regards,
>> Alexis.
>>
>> Am Mo., 28. Nov. 2022 um 05:17 Uhr schrieb Hangxiang Yu <
>> masteryhx@gmail.com>:
>>
>>> Hi, Alexis.
>>> IIUC, There is no conflict between savepoint history and restore mode.
>>> Restore mode cares about whether/how we manage the savepoint of old job.
>>> Savepoint management in operator only cares about savepoint history of
>>> new job.
>>> In other words, savepoint cleanup should not clean the savepoint from
>>> the old job which should only be controlled by restore mode.
>>> So I think you could also set restore mode according to your needs.
>>>
>>>
>>> On Wed, Nov 16, 2022 at 10:41 PM Alexis Sarda-Espinosa <
>>> sarda.espinosa@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> Is there a recommended configuration for the restore mode of jobs
>>>> managed by the operator?
>>>>
>>>> Since the documentation states that the operator keeps a savepoint
>>>> history to perform cleanup, I imagine restore mode should always be
>>>> NO_CLAIM, but I just want to confirm.
>>>>
>>>> Regards,
>>>> Alexis.
>>>>
>>>
>>>
>>> --
>>> Best,
>>> Hangxiang.
>>>
>>
Re: Savepoint restore mode for the Kubernetes operator
Posted by Gyula Fóra <gy...@gmail.com>.
The operator might call dispose on an old savepoint that’s true, but I am
not sure if the dispose api call would actually corrupt it.
Gyula
On Tue, 29 Nov 2022 at 09:28, Alexis Sarda-Espinosa <
sarda.espinosa@gmail.com> wrote:
> Hi Hangxiang,
>
> but, if I understand correctly, setting restore mode to CLAIM means that
> the job might create a new incremental checkpoint based on the savepoint,
> right? And if the operator then decides to clean up the savepoint, the
> checkpoint would be corrupted, no?
>
> Regards,
> Alexis.
>
> Am Mo., 28. Nov. 2022 um 05:17 Uhr schrieb Hangxiang Yu <
> masteryhx@gmail.com>:
>
>> Hi, Alexis.
>> IIUC, There is no conflict between savepoint history and restore mode.
>> Restore mode cares about whether/how we manage the savepoint of old job.
>> Savepoint management in operator only cares about savepoint history of
>> new job.
>> In other words, savepoint cleanup should not clean the savepoint from the
>> old job which should only be controlled by restore mode.
>> So I think you could also set restore mode according to your needs.
>>
>>
>> On Wed, Nov 16, 2022 at 10:41 PM Alexis Sarda-Espinosa <
>> sarda.espinosa@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Is there a recommended configuration for the restore mode of jobs
>>> managed by the operator?
>>>
>>> Since the documentation states that the operator keeps a savepoint
>>> history to perform cleanup, I imagine restore mode should always be
>>> NO_CLAIM, but I just want to confirm.
>>>
>>> Regards,
>>> Alexis.
>>>
>>
>>
>> --
>> Best,
>> Hangxiang.
>>
>
Re: Savepoint restore mode for the Kubernetes operator
Posted by Alexis Sarda-Espinosa <sa...@gmail.com>.
Hi Hangxiang,
but, if I understand correctly, setting restore mode to CLAIM means that
the job might create a new incremental checkpoint based on the savepoint,
right? And if the operator then decides to clean up the savepoint, the
checkpoint would be corrupted, no?
Regards,
Alexis.
Am Mo., 28. Nov. 2022 um 05:17 Uhr schrieb Hangxiang Yu <masteryhx@gmail.com
>:
> Hi, Alexis.
> IIUC, There is no conflict between savepoint history and restore mode.
> Restore mode cares about whether/how we manage the savepoint of old job.
> Savepoint management in operator only cares about savepoint history of new
> job.
> In other words, savepoint cleanup should not clean the savepoint from the
> old job which should only be controlled by restore mode.
> So I think you could also set restore mode according to your needs.
>
>
> On Wed, Nov 16, 2022 at 10:41 PM Alexis Sarda-Espinosa <
> sarda.espinosa@gmail.com> wrote:
>
>> Hello,
>>
>> Is there a recommended configuration for the restore mode of jobs managed
>> by the operator?
>>
>> Since the documentation states that the operator keeps a savepoint
>> history to perform cleanup, I imagine restore mode should always be
>> NO_CLAIM, but I just want to confirm.
>>
>> Regards,
>> Alexis.
>>
>
>
> --
> Best,
> Hangxiang.
>
Re: Savepoint restore mode for the Kubernetes operator
Posted by Hangxiang Yu <ma...@gmail.com>.
Hi, Alexis.
IIUC, There is no conflict between savepoint history and restore mode.
Restore mode cares about whether/how we manage the savepoint of old job.
Savepoint management in operator only cares about savepoint history of new
job.
In other words, savepoint cleanup should not clean the savepoint from the
old job which should only be controlled by restore mode.
So I think you could also set restore mode according to your needs.
On Wed, Nov 16, 2022 at 10:41 PM Alexis Sarda-Espinosa <
sarda.espinosa@gmail.com> wrote:
> Hello,
>
> Is there a recommended configuration for the restore mode of jobs managed
> by the operator?
>
> Since the documentation states that the operator keeps a savepoint history
> to perform cleanup, I imagine restore mode should always be NO_CLAIM, but I
> just want to confirm.
>
> Regards,
> Alexis.
>
--
Best,
Hangxiang.