You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Salva Alcántara <sa...@gmail.com> on 2024/06/12 03:26:50 UTC

Changing TTL config without breaking state

I have some jobs where I can configure the TTL duration for certain
operator state. The problem I'm noticing is that when I make changes in the
TTL configuration the new state descriptor becomes incompatible and I
cannot restart my jobs from current savepoints. Is that expected?

More precisely, I'm using this when myConfig.durationTTL() > 0:

```java
newBuilder(Time.fromDuration(myConfig.durationTTL()))
          .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
          .setStateVisibility(StateTtlConfig.StateVisibility.NeverReturnExpired)
          .build();
```

and this otherwise for the disabled case (myConfig.durationTTL() == 0):

```java
newBuilder(Time.milliseconds(Long.MAX_VALUE))
          .setUpdateType(StateTtlConfig.UpdateType.Disabled)
          .build();
```

What I'd like is to never break the state regardless of the duration value.
Is it possible to reconcile these cases?

Re: Changing TTL config without breaking state

Posted by xiangyu feng <xi...@gmail.com>.
Hi Salva,

Unfortunately, the state is currently incompatible between enabling TTL and
disabling TTL. This issue is tracked by this jira(
https://issues.apache.org/jira/browse/FLINK-32955) and not resolved yet.

Usually, we need to find a way to reaccumulate the state after enabled the
state ttl.

Regards,
Xiangyu Feng


Salva Alcántara <sa...@gmail.com> 于2024年6月12日周三 15:31写道：

> I have some jobs where I can configure the TTL duration for certain
> operator state. The problem I'm noticing is that when I make changes in the
> TTL configuration the new state descriptor becomes incompatible and I
> cannot restart my jobs from current savepoints. Is that expected?
>
> More precisely, I'm using this when myConfig.durationTTL() > 0:
>
> ```java
> newBuilder(Time.fromDuration(myConfig.durationTTL()))
>           .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>           .setStateVisibility(StateTtlConfig.StateVisibility.NeverReturnExpired)
>           .build();
> ```
>
> and this otherwise for the disabled case (myConfig.durationTTL() == 0):
>
> ```java
> newBuilder(Time.milliseconds(Long.MAX_VALUE))
>           .setUpdateType(StateTtlConfig.UpdateType.Disabled)
>           .build();
> ```
>
> What I'd like is to never break the state regardless of the duration
> value. Is it possible to reconcile these cases?
>