[jira] [Resolved] (SPARK-35800) Improving testability of GroupState in streaming flatMapGroupsWithState

["Jungtaek Lim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-35800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-35800.
----------------------------------
    Fix Version/s: 3.2.0
       Resolution: Fixed

Looks like Li Zhang didn't create a ASF Jira account. I'll leave the assignee as unassigned.

> Improving testability of GroupState in streaming flatMapGroupsWithState
> -----------------------------------------------------------------------
>
>                 Key: SPARK-35800
>                 URL: https://issues.apache.org/jira/browse/SPARK-35800
>             Project: Spark
>          Issue Type: New Feature
>          Components: Structured Streaming
>    Affects Versions: 3.1.2
>            Reporter: Tathagata Das
>            Priority: Major
>             Fix For: 3.2.0
>
>
> GroupStateImpl is the internal implementation of the GroupState interface which mean to be not exposed. Thus, it only has a private constructor. Such access control does benefit encapsulation, however, this introduces difficulties for unit tests and the users are calling the engine to construct such GroupState instances in order to test their customized state transition functions.
> The solution is to introduce new interfaces that allow users to create instances of GroupState but also access internal values of what they have set (for example, has to state been updated, or removed). This would allow them to write unit tests of the state transition function with custom GroupState objects and then verifying whether the state was updated in an expected way. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org