You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cody Koeninger (JIRA)" <ji...@apache.org> on 2016/10/12 23:10:21 UTC
[jira] [Resolved] (SPARK-5718) Add native offset management for
ReliableKafkaReceiver
[ https://issues.apache.org/jira/browse/SPARK-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cody Koeninger resolved SPARK-5718.
-----------------------------------
Resolution: Fixed
Fix Version/s: 2.0.0
SPARK-12177 added support for the native kafka offset commit api
> Add native offset management for ReliableKafkaReceiver
> ------------------------------------------------------
>
> Key: SPARK-5718
> URL: https://issues.apache.org/jira/browse/SPARK-5718
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.3.0
> Reporter: Saisai Shao
> Fix For: 2.0.0
>
>
> Kafka 0.8.2 supports native offsets management instead of ZK, this will get better performance, for now in ReliableKafkaReceiver, we rely on ZK to manage the offsets, this potentially will be a bottleneck if the injection rate is high (once per 200ms by default), so here in order to get better performance as well as keeping consistent with Kafka, add native offset management for ReliableKafkaReceiver.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org