[jira] [Created] (SPARK-16454) Consider adding a per-batch transform for structured streaming

["holdenk (JIRA)" <ji...@apache.org>]

holdenk created SPARK-16454:
-------------------------------

             Summary: Consider adding a per-batch transform for structured streaming
                 Key: SPARK-16454
                 URL: https://issues.apache.org/jira/browse/SPARK-16454
             Project: Spark
          Issue Type: Improvement
          Components: SQL, Streaming
            Reporter: holdenk


The new structured streaming API lacks the DStream functionality of transform (which allowed one to mix in existing RDD transformation logic). It would be useful to be able to do per-batch (even without any specific gaurantees about the batch being complete provided you eventually get called with the "catch up" records) processing as was done in the DStream API.

This might be useful for implementing Streaming Machine Learning on Structured Streaming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org