You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2026/01/23 05:45:00 UTC
[jira] [Created] (SPARK-55147) Improve the retrieval operation in new stream-stream stateful operator to scope the timestamp range for time-interval join
Jungtaek Lim created SPARK-55147:
------------------------------------
Summary: Improve the retrieval operation in new stream-stream stateful operator to scope the timestamp range for time-interval join
Key: SPARK-55147
URL: https://issues.apache.org/jira/browse/SPARK-55147
Project: Spark
Issue Type: Task
Components: Structured Streaming
Affects Versions: 4.2.0
Reporter: Jungtaek Lim
With SPARK-55144, the state format of the primary store would support for "prefix scan + range scan on event time". That allows us to perform the retrieval operation of time interval join more efficiently, via only scanning through the timestamp range which is constructed with "time interval condition".
This ticket tracks the effort of modifying the state store API and state manager, and modifying stream-stream join operator to integrate the change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org