You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ismaël Mejía (Jira)" <ji...@apache.org> on 2020/05/08 16:44:00 UTC
[jira] [Commented] (BEAM-9933) Support Streaming in Spark
Structured Streaming Runner
[ https://issues.apache.org/jira/browse/BEAM-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102724#comment-17102724 ]
Ismaël Mejía commented on BEAM-9933:
------------------------------------
I did an experiment to reproduce the aggregation issue in pure Spark here
https://github.com/iemejia/spark-playground
> Support Streaming in Spark Structured Streaming Runner
> ------------------------------------------------------
>
> Key: BEAM-9933
> URL: https://issues.apache.org/jira/browse/BEAM-9933
> Project: Beam
> Issue Type: New Feature
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: Major
> Labels: structured-streaming
>
> The current version of the Spark Structured Streaming Runner only support Bounded IO. This ticket is to address the support of full Streaming.
> This issue is blocked because of issues on Spark with multiple aggregations
> https://issues.apache.org/jira/browse/SPARK-26655
--
This message was sent by Atlassian Jira
(v8.3.4#803005)