You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Justin Borromeo <ju...@gmail.com> on 2019/03/15 20:07:58 UTC

Preventing blocking upon producer buffer full in Kafka 2.1

Hi all,

I was wondering how I could prevent blocking when using KafkaProducer to
send records with a full buffer.  I noticed from the v0.9 docs that there
was a block.on.buffer.full config that could be set to false to achieve
that behaviour...however, that was deprecated and is unavailable in v2.1.
Would setting max.block.ms to 0 behave the same?  If not, what's the best
way to do so?

Thanks,
Justin Borromeo

Re: Preventing blocking upon producer buffer full in Kafka 2.1

Posted by Justin Borromeo <ju...@gmail.com>.

Thanks for the explanation.  Is there any way to replicate the
block.on.buffer.full=false behaviour on Kafka 2.x or was the option
completely eliminated in every form?

Thanks,
Justin

On Thu, Mar 21, 2019 at 10:18 AM 星月夜 <10...@qq.com> wrote:

> hi,
> you cannot set max.block.ms to 0 because the max.block.ms is the max time
> producer.send（）and partitionsfor() will block. This block time contains
> metadata fetch time, buffer full block time, serilization time and
> partitioning time, not just buffer full block time. You should set it to a
> considerate value.
> From: "Justin Borromeo"<ju...@gmail.com>
> Date: Sat, Mar 16, 2019 04:08 AM
> To: "users"<us...@kafka.apache.org>;
> Subject: Preventing blocking upon producer buffer full in Kafka 2.1
>
>
> Hi all,
>
> I was wondering how I could prevent blocking when using KafkaProducer to
> send records with a full buffer.  I noticed from the v0.9 docs that there
> was a block.on.buffer.full config that could be set to false to achieve
> that behaviour...however, that was deprecated and is unavailable in v2.1.
> Would setting max.block.ms to 0 behave the same?  If not, what's the best
> way to do so?
>
> Thanks,
> Justin Borromeo

Re: Preventing blocking upon producer buffer full in Kafka 2.1

Posted by 星月夜 <10...@qq.com>.

hi,
you cannot set max.block.ms to 0 because the max.block.ms is the max time producer.send（）and partitionsfor() will block. This block time contains metadata fetch time, buffer full block time, serilization time and partitioning time, not just buffer full block time. You should set it to a considerate value.
From: "Justin Borromeo"<ju...@gmail.com>
Date: Sat, Mar 16, 2019 04:08 AM
To: "users"<us...@kafka.apache.org>;
Subject: Preventing blocking upon producer buffer full in Kafka 2.1

Hi all,

I was wondering how I could prevent blocking when using KafkaProducer to
send records with a full buffer.  I noticed from the v0.9 docs that there
was a block.on.buffer.full config that could be set to false to achieve
that behaviour...however, that was deprecated and is unavailable in v2.1.
Would setting max.block.ms to 0 behave the same?  If not, what's the best
way to do so?

Thanks,
Justin Borromeo