You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Igor Kuzmenko <f1...@gmail.com> on 2017/02/20 16:15:32 UTC
Kafka Spout enable.auto.commit=false
Hello, I'd like to understand difference between auto commit mode
true/false in new KafkaSpout.
With auto.commit.enabled = false KafkaSpout will move my offset relying on
acked tuples, it seems easy.
But what happens if I turn auto commit on?
How Kafka make decision which offset to commit?
Re: Kafka Spout enable.auto.commit=false
Posted by Igor Kuzmenko <f1...@gmail.com>.
Thanks, Hugo. That's all I want to know abot this.
On Tue, Feb 21, 2017 at 9:01 PM, Hugo Da Cruz Louro <hl...@hortonworks.com>
wrote:
> As per KafkaConsumer documentation
> <https://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html>
> :
>
> “ Setting enable.auto.commit=true means that offsets are committed
> automatically with a frequency controlled by the config
> auto.commit.interval.ms. “
>
> That means that the Spout disregards if the Tuple is acked or not; every
> time interval (as controlled by auto.commit.interval.ms ) the offset will
> be committed. This can have have an impact on the delivery guarantees,
> because an offset may be committed, yet the tuple may fail.
>
> On Feb 20, 2017, at 8:15 AM, Igor Kuzmenko <f1...@gmail.com> wrote:
>
> Hello, I'd like to understand difference between auto commit mode
> true/false in new KafkaSpout.
>
> With auto.commit.enabled = false KafkaSpout will move my offset relying on
> acked tuples, it seems easy.
>
> But what happens if I turn auto commit on?
> How Kafka make decision which offset to commit?
>
>
>
Re: Kafka Spout enable.auto.commit=false
Posted by Hugo Da Cruz Louro <hl...@hortonworks.com>.
As per KafkaConsumer documentation<https://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html>:
“ Setting enable.auto.commit=true means that offsets are committed automatically with a frequency controlled by the config auto.commit.interval.ms. “
That means that the Spout disregards if the Tuple is acked or not; every time interval (as controlled by auto.commit.interval.ms ) the offset will be committed. This can have have an impact on the delivery guarantees, because an offset may be committed, yet the tuple may fail.
On Feb 20, 2017, at 8:15 AM, Igor Kuzmenko <f1...@gmail.com>> wrote:
Hello, I'd like to understand difference between auto commit mode true/false in new KafkaSpout.
With auto.commit.enabled = false KafkaSpout will move my offset relying on acked tuples, it seems easy.
But what happens if I turn auto commit on?
How Kafka make decision which offset to commit?