You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Boyang Jerry Peng (Jira)" <ji...@apache.org> on 2022/06/24 20:58:00 UTC

[jira] [Updated] (SPARK-39585) Multiple Stateful Operators in Structured Streaming

     [ https://issues.apache.org/jira/browse/SPARK-39585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Boyang Jerry Peng updated SPARK-39585:
--------------------------------------
    Summary: Multiple Stateful Operators in Structured Streaming  (was: Multiple Stateful Operators)

> Multiple Stateful Operators in Structured Streaming
> ---------------------------------------------------
>
>                 Key: SPARK-39585
>                 URL: https://issues.apache.org/jira/browse/SPARK-39585
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 3.3.0
>            Reporter: Boyang Jerry Peng
>            Priority: Major
>
> Currently, Structured Streaming only supports one stateful operator per pipeline.  This constraint excludes Structured Streaming to be used for many use cases such ones involving chained time window aggregations, chained stream-stream outer equality join, stream-stream time interval join followed by time window aggregation.  Enabling multiple stateful operators within a pipeline in Structured Streaming will open up many additional use cases for its usage as well as be on par with other stream processing engines.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org