You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2026/06/02 16:46:00 UTC

[jira] [Commented] (SPARK-55122) Allow seamless and sequential source switching for streaming queries

    [ https://issues.apache.org/jira/browse/SPARK-55122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18085544#comment-18085544 ] 

Wenchen Fan commented on SPARK-55122:
-------------------------------------

re-target to 4.3

> Allow seamless and sequential source switching for streaming queries
> --------------------------------------------------------------------
>
>                 Key: SPARK-55122
>                 URL: https://issues.apache.org/jira/browse/SPARK-55122
>             Project: Spark
>          Issue Type: Epic
>          Components: Structured Streaming
>    Affects Versions: 4.3.0
>            Reporter: Eric Marnadi
>            Priority: Major
>
> Users may have multiple sources of data, for example a backfill and live source, that they want to process one after the other. We propose the `.followedBy` API where the user can specify multiple stateless streaming DFs that will execute one after the other, which will hydrate the same downstream stateful operators with no need for manual checkpoint intervention.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org