You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Li Yuanjian (JIRA)" <ji...@apache.org> on 2018/09/28 03:24:00 UTC
[jira] [Comment Edited] (SPARK-22565) Session-based windowing
[ https://issues.apache.org/jira/browse/SPARK-22565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631298#comment-16631298 ]
Li Yuanjian edited comment on SPARK-22565 at 9/28/18 3:23 AM:
--------------------------------------------------------------
Thanks for reporting this. Actually we also met this problem in our usage, we have an implement about session window in internal folk to resolve this. After steady running online for real product env, we want to contribute to community within the next few days. We implemented this by a build-in function named session_window and corresponding support for window merge in Structure Streaming. The usage of dataframe api and SQL can be quickly browsing by the test:
!screenshot-1.png!
was (Author: xuanyuan):
Thanks for reporting this. Actually we also met this problem in our usage, we have an implement about session window in internal folk to resolve this. After steady running online for real product env, we want to contribute to community within the next few days. We implemented this by a build-in function named session_window. The usage of dataframe api and SQL can be quickly browsing by the test:
!screenshot-1.png!
> Session-based windowing
> -----------------------
>
> Key: SPARK-22565
> URL: https://issues.apache.org/jira/browse/SPARK-22565
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.2.0
> Reporter: Richard Xin
> Priority: Major
> Attachments: screenshot-1.png
>
>
> I came across a requirement to support session-based windowing. for example, user activity comes in from kafka, we want to create window per user session (if the time gap of activity from the same user exceeds the predefined value, a new window will be created).
> I noticed that Flink does support this kind of support, any plan/schedule for spark for this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org