You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2026/03/26 20:54:00 UTC

[jira] [Assigned] (SPARK-55630) Skip updating matched flag on non-outer side of stream-stream join

     [ https://issues.apache.org/jira/browse/SPARK-55630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim reassigned SPARK-55630:
------------------------------------

    Assignee: Jungtaek Lim

> Skip updating matched flag on non-outer side of stream-stream join
> ------------------------------------------------------------------
>
>                 Key: SPARK-55630
>                 URL: https://issues.apache.org/jira/browse/SPARK-55630
>             Project: Spark
>          Issue Type: Task
>          Components: Structured Streaming
>    Affects Versions: 4.2.0
>            Reporter: Jungtaek Lim
>            Assignee: Jungtaek Lim
>            Priority: Major
>              Labels: pull-request-available
>
> Updating matched flag in stream-stream join triggers write(s) during retrieval of the matching values. While this is required for outer join side to track the unmatched output, it's unnecessary to track the matched flag for non-outer join side - the cost of updating matched flag is significant. For non-outer join side, we can skip updating the matched flag.
> Applying this optimization would break the case of changing join type during restarting of the query, but we do not have an explicit support of it, so this optimization does not break the actual use case.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org