Possible race in MapState#asyncPutAll on ForSt: chained reads see keys but null values

[Francis Altomare <fr...@helpscout.com>]

Hi all,

We hit what looks like a race condition in the ForSt state backend on Flink 2.2 with async state enabled, and I filed FLINK-40157 for it:

https://issues.apache.org/jira/browse/FLINK-40157

Short version: in a KeyedProcessFunction with a MapState<String, String>, writing with asyncPutAll and then chaining reads on the returned StateFuture gives back the newly written keys with null values. asyncKeys() returns everything, asyncValues() returns nothing, and asyncEntries() produces entries whose values are all null. The data is not lost. We inspected a checkpoint from the job and both keys and values are there, so the write reaches the backend. It looks like the future returned by asyncPutAll completes before the values are readable.

This only shows up on ForSt. Our MiniCluster integration tests do not reproduce it, and neither does the official docker image in single instance mode.

The workaround that fixed it for us was replacing asyncPutAll with individual asyncPut calls combined with StateFutureUtils.combineAll. With that change the chained reads return correct values.

A minimal example and full details are in the ticket. Happy to provide more info or help test a fix.

Thanks,
Francis

Re: Possible race in MapState#asyncPutAll on ForSt: chained reads see keys but null values

[Zakelly Lan <za...@gmail.com>]

Hi Francis,

Thanks for the detailed investigation! I found the `asyncPutAll` uses the
wrong serialized form without the null flag, so when reading it gives the
wrong.
I'll fix this soon.


Best,
Zakelly

On Thu, Jul 16, 2026 at 3:54 PM Francis Altomare <
francisaltomare@helpscout.com> wrote:

> Hi all,
>
> We hit what looks like a race condition in the ForSt state backend on
> Flink 2.2 with async state enabled, and I filed FLINK-40157 for it:
>
> https://issues.apache.org/jira/browse/FLINK-40157
>
> Short version: in a KeyedProcessFunction with a MapState<String, String>,
> writing with asyncPutAll and then chaining reads on the returned
> StateFuture gives back the newly written keys with null values. asyncKeys()
> returns everything, asyncValues() returns nothing, and asyncEntries()
> produces entries whose values are all null. The data is not lost. We
> inspected a checkpoint from the job and both keys and values are there, so
> the write reaches the backend. It looks like the future returned by
> asyncPutAll completes before the values are readable.
>
> This only shows up on ForSt. Our MiniCluster integration tests do not
> reproduce it, and neither does the official docker image in single instance
> mode.
>
> The workaround that fixed it for us was replacing asyncPutAll with
> individual asyncPut calls combined with StateFutureUtils.combineAll. With
> that change the chained reads return correct values.
>
> A minimal example and full details are in the ticket. Happy to provide
> more info or help test a fix.
>
> Thanks,
> Francis