You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dongwon Kim <ea...@gmail.com> on 2022/04/19 07:28:02 UTC
[State Processor API] unable to load the state of a trigger attached to a session window
Hi,
I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the state
of a stateful trigger attached to a session window.
I found that the following data become available in WindowReaderFunction:
- the state defined in the ProcessWindowFunction
- the registered timers of the stateful trigger attached to the session
window
- all the elements of the window
, but the state of the stateful trigger attached to the session window is
not available when using State Processor API.
In other words, the following code always returns null when used with
session windows:
> ReducingState<Long> state =
> context.triggerState(triggerCountDesc);
> Long val = state.get();
>
On the other hand, the above code snippet returns expected data when used
with sliding and tumbling windows.
To explain the problem, I made up an example in a similar spirit to
o.a.f.state.api.SavepointWindowReaderITCase.
Here you can find three test cases each with different types of event-time
windows: Session, Sliding, and Tumbling.
With sliding and tumbling windows, I can read the state of the trigger
attached to the windows in WindowReaderFunction.
However, with a session window, I cannot read the state of the trigger in
WindowReaderFunction.
Is it a bug, or did I miss something?
Best,
Dongwon
Re: [State Processor API] unable to load the state of a trigger attached to a session window
Posted by Dongwon Kim <ea...@gmail.com>.
Is the library, State Processor API, not widely used and no longer
maintained?
On Mon, Apr 25, 2022 at 3:16 PM Dongwon Kim <ea...@gmail.com> wrote:
> Can anyone help me with this?
>
> Thanks in advance,
>
> On Tue, Apr 19, 2022 at 4:28 PM Dongwon Kim <ea...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the
>> state of a stateful trigger attached to a session window.
>> I found that the following data become available in WindowReaderFunction:
>> - the state defined in the ProcessWindowFunction
>> - the registered timers of the stateful trigger attached to the session
>> window
>> - all the elements of the window
>> , but the state of the stateful trigger attached to the session window is
>> not available when using State Processor API.
>> In other words, the following code always returns null when used with
>> session windows:
>>
>>> ReducingState<Long> state =
>>> context.triggerState(triggerCountDesc);
>>> Long val = state.get();
>>>
>> On the other hand, the above code snippet returns expected data when used
>> with sliding and tumbling windows.
>>
>> To explain the problem, I made up an example in a similar spirit to
>> o.a.f.state.api.SavepointWindowReaderITCase.
>> Here you can find three test cases each with different types of
>> event-time windows: Session, Sliding, and Tumbling.
>> With sliding and tumbling windows, I can read the state of the trigger
>> attached to the windows in WindowReaderFunction.
>> However, with a session window, I cannot read the state of the trigger in
>> WindowReaderFunction.
>>
>> Is it a bug, or did I miss something?
>>
>> Best,
>>
>> Dongwon
>>
>>
Re: [State Processor API] unable to load the state of a trigger attached to a session window
Posted by Dongwon Kim <ea...@gmail.com>.
Can anyone help me with this?
Thanks in advance,
On Tue, Apr 19, 2022 at 4:28 PM Dongwon Kim <ea...@gmail.com> wrote:
> Hi,
>
> I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the
> state of a stateful trigger attached to a session window.
> I found that the following data become available in WindowReaderFunction:
> - the state defined in the ProcessWindowFunction
> - the registered timers of the stateful trigger attached to the session
> window
> - all the elements of the window
> , but the state of the stateful trigger attached to the session window is
> not available when using State Processor API.
> In other words, the following code always returns null when used with
> session windows:
>
>> ReducingState<Long> state =
>> context.triggerState(triggerCountDesc);
>> Long val = state.get();
>>
> On the other hand, the above code snippet returns expected data when used
> with sliding and tumbling windows.
>
> To explain the problem, I made up an example in a similar spirit to
> o.a.f.state.api.SavepointWindowReaderITCase.
> Here you can find three test cases each with different types of event-time
> windows: Session, Sliding, and Tumbling.
> With sliding and tumbling windows, I can read the state of the trigger
> attached to the windows in WindowReaderFunction.
> However, with a session window, I cannot read the state of the trigger in
> WindowReaderFunction.
>
> Is it a bug, or did I miss something?
>
> Best,
>
> Dongwon
>
>
Re: [State Processor API] unable to load the state of a trigger attached to a session window
Posted by Dongwon Kim <ea...@gmail.com>.
Hi Juntao,
Thanks a lot for taking a look at this.
After a little inspection, I found that elements (window state) are stored
> in namespace TimeWindow{start=1,end=11}, in your case, and trigger count
> (trigger state) is stored in namespace TimeWindow{start=1,end=15}, but
> WindowReaderOperator only tries to find keys and namespaces related to
> window state.
So is there any chance for me to get the trigger state using the correct
name space? Or should it be considered as a bug to be fixed?
Best,
Dongwon
On Thu, May 12, 2022 at 5:51 PM Juntao Hu <ma...@gmail.com> wrote:
> Sorry to make the previous mail private.
> My response reposted here:
> "
> After a little inspection, I found that elements (window state) are stored
> in namespace TimeWindow{start=1,end=11}, in your case, and trigger count
> (trigger state) is stored in namespace TimeWindow{start=1,end=15}, but
> WindowReaderOperator only tries to find keys and namespaces related to
> window state.
> "
>
> Juntao Hu <ma...@gmail.com> 于2022年5月12日周四 11:47写道：
>
>> After a little inspection, I found that elements (window state) are
>> stored in namespace TimeWindow{start=1,end=11}, in your case, and trigger
>> count (trigger state) is stored in namespace TimeWindow{start=1,end=15},
>> but WindowReaderOperator only tries to find keys and namespaces related to
>> window state.
>>
>> Dongwon Kim <ea...@gmail.com> 于2022年4月19日周二 15:29写道：
>>
>>> Hi,
>>>
>>> I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the
>>> state of a stateful trigger attached to a session window.
>>> I found that the following data become available in WindowReaderFunction:
>>> - the state defined in the ProcessWindowFunction
>>> - the registered timers of the stateful trigger attached to the session
>>> window
>>> - all the elements of the window
>>> , but the state of the stateful trigger attached to the session window
>>> is not available when using State Processor API.
>>> In other words, the following code always returns null when used with
>>> session windows:
>>>
>>>> ReducingState<Long> state =
>>>> context.triggerState(triggerCountDesc);
>>>> Long val = state.get();
>>>>
>>> On the other hand, the above code snippet returns expected data when
>>> used with sliding and tumbling windows.
>>>
>>> To explain the problem, I made up an example in a similar spirit to
>>> o.a.f.state.api.SavepointWindowReaderITCase.
>>> Here you can find three test cases each with different types of
>>> event-time windows: Session, Sliding, and Tumbling.
>>> With sliding and tumbling windows, I can read the state of the trigger
>>> attached to the windows in WindowReaderFunction.
>>> However, with a session window, I cannot read the state of the trigger
>>> in WindowReaderFunction.
>>>
>>> Is it a bug, or did I miss something?
>>>
>>> Best,
>>>
>>> Dongwon
>>>
>>>
Re: [State Processor API] unable to load the state of a trigger attached to a session window
Posted by Juntao Hu <ma...@gmail.com>.
Sorry to make the previous mail private.
My response reposted here:
"
After a little inspection, I found that elements (window state) are stored
in namespace TimeWindow{start=1,end=11}, in your case, and trigger count
(trigger state) is stored in namespace TimeWindow{start=1,end=15}, but
WindowReaderOperator only tries to find keys and namespaces related to
window state.
"
Juntao Hu <ma...@gmail.com> 于2022年5月12日周四 11:47写道：
> After a little inspection, I found that elements (window state) are stored
> in namespace TimeWindow{start=1,end=11}, in your case, and trigger count
> (trigger state) is stored in namespace TimeWindow{start=1,end=15}, but
> WindowReaderOperator only tries to find keys and namespaces related to
> window state.
>
> Dongwon Kim <ea...@gmail.com> 于2022年4月19日周二 15:29写道：
>
>> Hi,
>>
>> I'm using Flink-1.14.4 and failed to load in WindowReaderFunction the
>> state of a stateful trigger attached to a session window.
>> I found that the following data become available in WindowReaderFunction:
>> - the state defined in the ProcessWindowFunction
>> - the registered timers of the stateful trigger attached to the session
>> window
>> - all the elements of the window
>> , but the state of the stateful trigger attached to the session window is
>> not available when using State Processor API.
>> In other words, the following code always returns null when used with
>> session windows:
>>
>>> ReducingState<Long> state =
>>> context.triggerState(triggerCountDesc);
>>> Long val = state.get();
>>>
>> On the other hand, the above code snippet returns expected data when used
>> with sliding and tumbling windows.
>>
>> To explain the problem, I made up an example in a similar spirit to
>> o.a.f.state.api.SavepointWindowReaderITCase.
>> Here you can find three test cases each with different types of
>> event-time windows: Session, Sliding, and Tumbling.
>> With sliding and tumbling windows, I can read the state of the trigger
>> attached to the windows in WindowReaderFunction.
>> However, with a session window, I cannot read the state of the trigger in
>> WindowReaderFunction.
>>
>> Is it a bug, or did I miss something?
>>
>> Best,
>>
>> Dongwon
>>
>>