You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2026/02/22 07:26:00 UTC

[jira] [Created] (SPARK-55630) Skip updating matched flag on non-outer side of stream-stream join

Jungtaek Lim created SPARK-55630:
------------------------------------

             Summary: Skip updating matched flag on non-outer side of stream-stream join
                 Key: SPARK-55630
                 URL: https://issues.apache.org/jira/browse/SPARK-55630
             Project: Spark
          Issue Type: Task
          Components: Structured Streaming
    Affects Versions: 4.2.0
            Reporter: Jungtaek Lim


Updating matched flag in stream-stream join triggers write(s) during retrieval of the matching values. While this is required for outer join side to track the unmatched output, it's unnecessary to track the matched flag for non-outer join side - the cost of updating matched flag is significant. For non-outer join side, we can skip updating the matched flag.

Applying this optimization would break the case of changing join type during restarting of the query, but we do not have an explicit support of it, so this optimization does not break the actual use case.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org