You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/03/29 04:24:00 UTC

[jira] [Assigned] (SPARK-34889) Introduce MergingSessionsIterator merging elements directly which belong to the same session

     [ https://issues.apache.org/jira/browse/SPARK-34889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-34889:
------------------------------------

    Assignee:     (was: Apache Spark)

> Introduce MergingSessionsIterator merging elements directly which belong to the same session
> --------------------------------------------------------------------------------------------
>
>                 Key: SPARK-34889
>                 URL: https://issues.apache.org/jira/browse/SPARK-34889
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Structured Streaming
>    Affects Versions: 3.2.0
>            Reporter: Jungtaek Lim
>            Priority: Major
>
> This issue tracks effort on introducing MergingSessionsIterator, which enables to merge elements belong to the same session directly. This would be quite performant compared to UpdatingSessionIterator. Note that MergingSessionsIterator can only apply to the cases aggregation can be applied altogether, so there're still rooms for UpdatingSessionIterator to be used.
> This issue also introduces MergingSessionsExec which is the physical node on leveraging MergingSessionsIterator to sort the input rows and aggregate rows according to the session windows.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org