You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Lasse Nedergaard <la...@gmail.com> on 2025/08/11 13:16:40 UTC
Re: Unit testing KeyedProcessFunction that uses async state
Hi
I have updated to Flink 2.1 and I can see the changes you mention.
I hope you can help me here as I can’t see how to use the new functions.
I’m implementing TwoInputNoneBroadcastEventTimeStreamProcesaFunction.
And I can’t figure out how to get testharness to work on interfaces like this.
So any help appreciated
Med venlig hilsen / Best regards
Lasse Nedergaard
> Den 24. jun. 2025 kl. 05.48 skrev Han Yin <al...@gmail.com>:
>
>
> Hi Nate, Hi Lasse,
>
> FYI, if you're still interested in the test harness for KeyFunctions with
> State V2, please note that the ticket [1] mentioned earlier has been
> resolved. Although it hasn't been officially released yet, you can try out
> the new harness methods. These methods can be found in the recent commit [2]
> on the master branch of Flink.
>
> [1] <https://issues.apache.org/jira/browse/FLINK-37944>
>
> [2]
> <https://github.com/apache/flink/commit/a4958171f1d4a06c4031e4adf1b6ea25e072d005>
>
>
>
>
>
>
>
>> 2025年6月18日 20:08，Lasse Nedergaard
<[lassenedergaardflink@gmail.com](mailto:lassenedergaardflink@gmail.com)> 写道：
>>
>>
>
>>
>> Hi
>>
>>
>
>>
>> So the conclusion is that it isn’t possible at all to test key functions
using the new DataStream api as it requires state V2 and test support is
missing.
>>
>>
>
>>
>> And it isn’t possible to declare and use state V1 together with the new
DataStream api as a temporary solution because you don’t have access to the
RuntimeContext.
>>
>>
>
>>
>> Right?
>>
>>
>
>>
>> Best regards
>
>>
>> Lasse Nedergaard
>>
>>
>
>>
>>
>
>>
>>> Den 11. jun. 2025 kl. 12.31 skrev Zakelly Lan
<[zakelly.lan@gmail.com](mailto:zakelly.lan@gmail.com)>:
>
>
>>
>>>
>>>
>>> Hi Nate,
>>>
>>>
>
>>>
>>> Glad to see you trying out the new API.
>>>
>>>
>
>>>
>>> Well, you're right that the `ProcessFunctionTestHarnesses` doesn't support
the new state API yet. There are operators tailored for async state access, as
well the operator test harness. But unfortunately we can't tell in advance if
a `KeyedProcessFunction` is using the new state api or not, so current
`forKeyedProcessFunction` still uses the operator without async state support.
I've created a ticket[1] and will add a method for the case.
>>>
>>>
>
>>>
>>> [1] <https://issues.apache.org/jira/browse/FLINK-37944>
>>>
>>>
>
>>>
>>>
>
>>>
>>> Best,
>>>
>>> Zakelly
>>>
>>>
>
>>>
>>> On Wed, Jun 11, 2025 at 3:29 AM Nate Drake
<[ndrake@gmail.com](mailto:ndrake@gmail.com)> wrote:
>
>>>
>>>> Hi,
>
> I'm attempting to unit test a KeyedProcessFunction that uses v2 Async State
> API using "ProcessFunctionTestHarnesses.forKeyedProcessFunction()". When I
> run my test it fails with:
>
> "Caused by: java.lang.IllegalStateException: Current operator integrates the
> async processing logic, thus only supports state v2 APIs. Please use
> StateDescriptor under 'org.apache.flink.runtime.state.v2'."
>
> I ran into a similar issue when attempting to run integration tests for the
> function even though I confirmed I'm using the .v2. packages and properly
> calling ".enableAsyncState()". After finding these two issues in Jira:
>
> <https://issues.apache.org/jira/browse/FLINK-37621>
> <https://issues.apache.org/jira/browse/FLINK-37623>
>
> I updated my project to use Flink 2.1.0-SNAPSHOT. That fixed my integration
> test, but my unit test is still failing.
>
> I'm not sure if I'm doing something wrong in my test or if maybe
> ProcessFunctionTestHarnesses doesn't support Async State yet.
>
> Any ideas?
>
> Thanks!
>
> Nate
>
>
>