You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zakelly Lan (Jira)" <ji...@apache.org> on 2024/12/13 04:07:00 UTC

[jira] [Commented] (FLINK-36897) Error executing processElement when inheriting from AbstractAsyncStateStreamOperator

    [ https://issues.apache.org/jira/browse/FLINK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17905361#comment-17905361 ] 

Zakelly Lan commented on FLINK-36897:
-------------------------------------

I had a glance at your commit, in `AsyncStateMiniBatchGroupAggFunction#finishBundle` you should not do `ctx.setCurrentKey(currentKey);` which is a sync processing method. The `AbstractAsyncStateStreamOperator#asyncProcessWithKey` is prepared for this scenario. 

Given the minibatch is obviously more difficult to process with async state, my advice is to hold back the specific sql implementations until there is a unified framework for minibatch with async state. 

> Error executing processElement when inheriting from AbstractAsyncStateStreamOperator
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-36897
>                 URL: https://issues.apache.org/jira/browse/FLINK-36897
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Runtime
>    Affects Versions: 2.0.0
>            Reporter: Wang Qilong
>            Priority: Major
>
> When I created the AbstractAsynchronous StateMapBundleOperator and inherited it from the AbstractAsynchronous StateStreamOperator, there was an error in the data passed into the element by the processElement of the AbstractAsynchronous StateMapBundleOperator itself
> The inheritance relationship between asynchronous synchronization and two classes is:
> AbstractMapBundleOperator->AbstractStreamOperator
> AbstractAsyncStateMapBundleOperator->AbstractAsyncStateStreamOperator->AbstractStreamOperator
> The reason for creating this class is to enable KeyedMapBundleOperator to support asynchronous running capability
> Example of incorrect information: For example, the original data format was:
> val data = new mutable.MutableList[(String, Long)]
> data.+=(("x", 1L))
> data.+=(("x", 2L))
> data.+=(("x", 3L))
> data.+=(("y", 1L))
> data.+=(("y", 2L))
> data.+=(("z", 3L))
> So the result of data transmission becomes:
> val data = new mutable.MutableList[(String, Long)]
> data.+=(("x", 1L))
> data.+=(("x", 2L))
> data.+=(("x", 3L))
> data.+=(("x", 1L))
> data.+=(("x", 2L))
> data.+=(("x", 3L))
> How to reproduce:
> Run testOverloadedAccumulator() in sql/AggregateITCase.java in [1]
>  
> [1]  [https://github.com/Au-Miner/flink/tree/FLINK-36882]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)