You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Michael Armbrust (JIRA)" <ji...@apache.org> on 2016/11/02 01:20:58 UTC

[jira] [Closed] (SPARK-17345) Prerequisites for Kafka 0.10 support in Structured Streaming

     [ https://issues.apache.org/jira/browse/SPARK-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Armbrust closed SPARK-17345.
------------------------------------
    Resolution: Fixed

> Prerequisites for Kafka 0.10 support in Structured Streaming
> ------------------------------------------------------------
>
>                 Key: SPARK-17345
>                 URL: https://issues.apache.org/jira/browse/SPARK-17345
>             Project: Spark
>          Issue Type: Sub-task
>          Components: DStreams
>            Reporter: Frederick Reiss
>
> This issue covers any API changes, refactoring, and utility classes/methods that are necessary to make it possible to implement support for Kafka 0.10 sources and sinks in Structured Streaming.
> At a minimum, the changes in SPARK-16963 will be needed in order for the Kafka commit protocol to work. Given that KIP-33 ("Add a time based log index") is not yet in place, it may be necessary to make additional API changes in Spark for commit to work efficiently. Some refactoring of the existing KafkaRDD class is probably in order also.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org