You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com> on 2017/10/23 22:11:01 UTC

[discuss] SPIP: Continuous Processing Mode for Structured Streaming

Please take a look at the attached PDF for the SPIP: Continuous Processing
Mode for Structured Streaming

https://issues.apache.org/jira/browse/SPARK-20928

It is meant to be a very small, surgical change to Structured Streaming to
enable ultra-low latency. This is great timing because we are also
designing and implementing data source API v2. If designed properly, we can
have the same data source API working for both streaming and batch.