You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:04:39 UTC
[jira] [Updated] (SPARK-16454) Consider adding a per-batch
transform for structured streaming
[ https://issues.apache.org/jira/browse/SPARK-16454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-16454:
---------------------------------
Labels: bulk-closed (was: )
> Consider adding a per-batch transform for structured streaming
> --------------------------------------------------------------
>
> Key: SPARK-16454
> URL: https://issues.apache.org/jira/browse/SPARK-16454
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: holdenk
> Priority: Major
> Labels: bulk-closed
>
> The new structured streaming API lacks the DStream functionality of transform (which allowed one to mix in existing RDD transformation logic). It would be useful to be able to do per-batch (even without any specific gaurantees about the batch being complete provided you eventually get called with the "catch up" records) processing as was done in the DStream API.
> This might be useful for implementing Streaming Machine Learning on Structured Streaming.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org