[jira] [Created] (SPARK-36480) SessionWindowStateStoreSaveExec should not filter input rows against watermark

["Jungtaek Lim (Jira)" <ji...@apache.org>]

Jungtaek Lim created SPARK-36480:
------------------------------------

             Summary: SessionWindowStateStoreSaveExec should not filter input rows against watermark
                 Key: SPARK-36480
                 URL: https://issues.apache.org/jira/browse/SPARK-36480
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 3.2.0
            Reporter: Jungtaek Lim


SessionWindowStateStoreSaveExec receives all sessions including existing sessions into input rows and stores as they are. That said, we should not filter out input rows before storing into state store, but we do. 

Fortunately it hasn't showed any actual problem due to the nature how we deal with watermark against micro-batch and it seems hard to come up with the broken case, but it should be better to fix it before someone succeeds to touch the possible edge case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org