You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "L. C. Hsieh (Jira)" <ji...@apache.org> on 2021/02/13 04:01:00 UTC
[jira] [Comment Edited] (SPARK-34427) Session window support in SS
[ https://issues.apache.org/jira/browse/SPARK-34427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284076#comment-17284076 ]
L. C. Hsieh edited comment on SPARK-34427 at 2/13/21, 4:00 AM:
---------------------------------------------------------------
Thanks for the quick feedback. I didn't find the previous PRs although I did a quick search of session window + spark structured streaming on Internet. Roughly I think the requirement should be as the same as what Flink provides: static and dynamic gap. But I can confirm again with the customer.
Seems to me there were two PRs working on the session window feature. I took a quick look at [~kabhwan]
's and the PR looks pretty complicated than what I have in my mind. From the code size, that PR is much larger than another. I'm not sure if from feature perspective they are the same.
As it comes to the weekend, I can take another look at the previous two PRs.
From my side, I'd like to push this feature as we have real use case and requirement. But I'm not sure if we want to follow up with previous PRs.
For some implementations on flatMapGroupsWithState, seems to me it doesn't actually achieve the same feature as real session window feature. Maybe I miss it, but I'm willing to and looking some flatMapGroupsWithState example in previous JIRA.
was (Author: viirya):
Thanks for the quick feedback. I didn't find the previous PRs although I did a quick search of session window + spark structured streaming on Internet. Roughly I think the requirement should be as the same as what Flink provides: static and dynamic gap. But I can confirm again with the customer.
Seems to me there were two PRs working on the session window feature. I took a quick look at [~kabhwan]
's and the PR looks pretty complicated than what I have in my mind. From the code size, that PR is much larger than another. I'm not sure if from feature perspective they are the same.
As it comes to the weekend, I can take another look at the previous two PRs.
From my side, I'd like to push this feature as we have real use case and requirement. But I'm not sure if we want to follow up with previous PRs.
For some implementations on flatMapGroupsWithState, seems to me it doesn't actually achieve the same feature as real session window feature.
> Session window support in SS
> ----------------------------
>
> Key: SPARK-34427
> URL: https://issues.apache.org/jira/browse/SPARK-34427
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Affects Versions: 3.2.0
> Reporter: L. C. Hsieh
> Assignee: L. C. Hsieh
> Priority: Major
>
> Currently structured streaming supports two kinds of windows: tumbling window and sliding window. Another useful window function is session window. Which is not supported by SS. We have user requirement to use session window. We'd like to have this support in the upstream.
> About session window, there is some info: https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#session-windows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org