[jira] [Resolved] (SPARK-47960) Support Chaining Stateful Operators in TransformWithState

["Jungtaek Lim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-47960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-47960.
----------------------------------
    Resolution: Fixed

Issue resolved by pull request 45376
[https://github.com/apache/spark/pull/45376]

> Support Chaining Stateful Operators in TransformWithState
> ---------------------------------------------------------
>
>                 Key: SPARK-47960
>                 URL: https://issues.apache.org/jira/browse/SPARK-47960
>             Project: Spark
>          Issue Type: New Feature
>          Components: Structured Streaming
>    Affects Versions: 4.0.0
>            Reporter: Bhuwan Sahni
>            Assignee: Bhuwan Sahni
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>
> This issue tracks adding support to chain stateful operators after the Arbitrary State API, transformWithState. In order to support chaining, we need to allow the user to specify the new eventTimeColumn in the output from StatefulProcessor. Any watermark evaluation expressions downstream after transformWithState would use the user specified eventTimeColumn.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org