You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiao Li (JIRA)" <ji...@apache.org> on 2018/07/13 05:27:01 UTC
[jira] [Updated] (SPARK-24234) create the bottom-of-task RDD with
row buffer
[ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li updated SPARK-24234:
----------------------------
Fix Version/s: (was: 3.0.0)
2.4.0
> create the bottom-of-task RDD with row buffer
> ---------------------------------------------
>
> Key: SPARK-24234
> URL: https://issues.apache.org/jira/browse/SPARK-24234
> Project: Spark
> Issue Type: Sub-task
> Components: Structured Streaming
> Affects Versions: 2.4.0
> Reporter: Jose Torres
> Assignee: Jose Torres
> Priority: Major
> Fix For: 2.4.0
>
>
> [https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65AYOfE/edit#heading=h.8t3ci57f7uii]
>
> This probably ought to be an abstraction of ContinuousDataSourceRDD and ContinuousQueuedDataReader. These classes do pretty much exactly what's needed, except the buffer is filled from the remote data source instead of a remote Spark task.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org