[jira] [Commented] (BEAM-84) Add support for Session Windows - Beam Sessions

["Amit Sela (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242594#comment-15242594 ] 

Amit Sela commented on BEAM-84:
-------------------------------

Wait with this one until integration with Spark 2.0 since this is a streaming feature, and the runner is going through a refactor to work with Datasets - which will support streaming (Structured Streaming) as of 2.0

> Add support for Session Windows - Beam Sessions
> -----------------------------------------------
>
>                 Key: BEAM-84
>                 URL: https://issues.apache.org/jira/browse/BEAM-84
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-spark
>            Reporter: Amit Sela
>
> Implement Beam Sessions, or Session Windows, by maintaining the session state. Spark 1.6 presents mapWithState as an improvement to updateStateByKey so It'd be better to use it.
> See "Session Windows" in Dataflow documentation here: https://cloud.google.com/dataflow/model/windowing#Functions
> Also, this blog post from Databricks would be a good place to start: https://databricks.com/blog/2016/02/01/faster-stateful-stream-processing-in-spark-streaming.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)