You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Haojin Gui <gu...@gmail.com> on 2014/12/04 01:03:47 UTC

How to create a reliable and fast Kafka Producer in a web server?

Hi,

I am using Playframework to build a web server that needs to handle very
heavy traffic. And it has a Kafka Producer sending messages to Kafka
brokers.

Currently I am using the synchronized Kafka producer API and seems like
it's affecting the performance of the web server. I am considering
switching to async Producer API but not sure how reliable it is(I heard it
might loss message). Are there any other options to minimize the Producer's
affect to the server.

Thanks,
Haojin

Re: How to create a reliable and fast Kafka Producer in a web server?

Posted by Haojin Gui <gu...@gmail.com>.
Thank you, Guozhang!

On Thu, Dec 4, 2014 at 9:14 AM, Guozhang Wang <wa...@gmail.com> wrote:

> Haojin,
>
> With the old "synchronized" API you can also enable asynchrony with this
> config "producer.type" in the producer configs:
>
> http://kafka.apache.org/documentation.html#producerconfigs
>
> The new "asynchronized" producer improves performance by using pipelining.
>
> Guozhang
>
>
>
> On Wed, Dec 3, 2014 at 4:03 PM, Haojin Gui <gu...@gmail.com> wrote:
>
> > Hi,
> >
> > I am using Playframework to build a web server that needs to handle very
> > heavy traffic. And it has a Kafka Producer sending messages to Kafka
> > brokers.
> >
> > Currently I am using the synchronized Kafka producer API and seems like
> > it's affecting the performance of the web server. I am considering
> > switching to async Producer API but not sure how reliable it is(I heard
> it
> > might loss message). Are there any other options to minimize the
> Producer's
> > affect to the server.
> >
> > Thanks,
> > Haojin
> >
>
>
>
> --
> -- Guozhang
>

Re: How to create a reliable and fast Kafka Producer in a web server?

Posted by Guozhang Wang <wa...@gmail.com>.
Haojin,

With the old "synchronized" API you can also enable asynchrony with this
config "producer.type" in the producer configs:

http://kafka.apache.org/documentation.html#producerconfigs

The new "asynchronized" producer improves performance by using pipelining.

Guozhang



On Wed, Dec 3, 2014 at 4:03 PM, Haojin Gui <gu...@gmail.com> wrote:

> Hi,
>
> I am using Playframework to build a web server that needs to handle very
> heavy traffic. And it has a Kafka Producer sending messages to Kafka
> brokers.
>
> Currently I am using the synchronized Kafka producer API and seems like
> it's affecting the performance of the web server. I am considering
> switching to async Producer API but not sure how reliable it is(I heard it
> might loss message). Are there any other options to minimize the Producer's
> affect to the server.
>
> Thanks,
> Haojin
>



-- 
-- Guozhang