You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sivananda Reddy <si...@gmail.com> on 2015/07/17 18:35:46 UTC
New producer in production
Hi,
Kafka document ion says that the new producer is in Beta state, how safe is
it to use the new producer in production?. This is the first time I am
using Kafka for my application messaging needs. Please let me know.
Thank you,
Siva.
Re: New producer in production
Posted by yuto sasaki <ye...@gmail.com>.
Hi,
Is there any progress on this doc bug?
This discourages us from using the new producer.
Thanks
On 2015/07/18 5:23, Sivananda Reddy wrote:
> Hi Jay,
>
> I found it here:
> http://kafka.apache.org/documentation.html#newproducerconfigs, the same
> link is reported by James.
>
> @Joel: Thanks a lot for the info, I will use new producer
>
> Regards,
> Siva.
>
> On Fri, Jul 17, 2015 at 12:02 PM, James Cheng <jc...@tivo.com> wrote:
>
>> http://kafka.apache.org/documentation.html, Section 3.4.
>>
>>> 3.4 New Producer Configs
>>>
>>> We are working on a replacement for our existing producer. The code is
>> available in trunk now and can be considered beta quality. Below is the
>> configuration for the new producer.
>>
>> Sivananda might have seen it elsewhere, but this is where I found it.
>>
>> -James
>>
>> On Jul 17, 2015, at 9:49 AM, Jay Kreps <ja...@confluent.io> wrote:
>>
>>> Hey Sivananda,
>>>
>>> That's actually no longer true and likely a documentation bug. Where did
>>> you see that?
>>>
>>> -Jay
>>>
>>> On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <
>> sivananda2510@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Kafka document ion says that the new producer is in Beta state, how
>> safe is
>>>> it to use the new producer in production?. This is the first time I am
>>>> using Kafka for my application messaging needs. Please let me know.
>>>>
>>>> Thank you,
>>>> Siva.
>>>>
>>
>>
>
Re: New producer in production
Posted by Sivananda Reddy <si...@gmail.com>.
Hi James,
Thanks for the info. For my use case I have a custom partition key
assignment to the producers.
Regards,
Siva.
On Fri, Jul 17, 2015 at 1:36 PM, James Cheng <jc...@tivo.com> wrote:
> Be aware that the old producer and new producer have different
> partitioning algorithms:
>
> http://mail-archives.apache.org/mod_mbox/kafka-users/201504.mbox/%3C4A6E5F2C-232E-4288-8F9F-3F9D9AE05718%40tivo.com%3E
>
> This could bite you if you rely on key-based partitioning and are
> switching from the old producer to the new producer.
>
> Siva, this is less important for you, since you are starting to use kafka
> for the first time. And this is not important at all if you provide your
> own partitioning algorithm.
>
> -James
>
> On Jul 17, 2015, at 1:23 PM, Sivananda Reddy <si...@gmail.com>
> wrote:
>
> > Hi Jay,
> >
> > I found it here:
> > http://kafka.apache.org/documentation.html#newproducerconfigs, the same
> > link is reported by James.
> >
> > @Joel: Thanks a lot for the info, I will use new producer
> >
> > Regards,
> > Siva.
> >
> > On Fri, Jul 17, 2015 at 12:02 PM, James Cheng <jc...@tivo.com> wrote:
> >
> >> http://kafka.apache.org/documentation.html, Section 3.4.
> >>
> >>> 3.4 New Producer Configs
> >>>
> >>> We are working on a replacement for our existing producer. The code is
> >> available in trunk now and can be considered beta quality. Below is the
> >> configuration for the new producer.
> >>
> >> Sivananda might have seen it elsewhere, but this is where I found it.
> >>
> >> -James
> >>
> >> On Jul 17, 2015, at 9:49 AM, Jay Kreps <ja...@confluent.io> wrote:
> >>
> >>> Hey Sivananda,
> >>>
> >>> That's actually no longer true and likely a documentation bug. Where
> did
> >>> you see that?
> >>>
> >>> -Jay
> >>>
> >>> On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <
> >> sivananda2510@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Kafka document ion says that the new producer is in Beta state, how
> >> safe is
> >>>> it to use the new producer in production?. This is the first time I am
> >>>> using Kafka for my application messaging needs. Please let me know.
> >>>>
> >>>> Thank you,
> >>>> Siva.
> >>>>
> >>
> >>
>
>
Re: New producer in production
Posted by James Cheng <jc...@tivo.com>.
Be aware that the old producer and new producer have different partitioning algorithms:
http://mail-archives.apache.org/mod_mbox/kafka-users/201504.mbox/%3C4A6E5F2C-232E-4288-8F9F-3F9D9AE05718%40tivo.com%3E
This could bite you if you rely on key-based partitioning and are switching from the old producer to the new producer.
Siva, this is less important for you, since you are starting to use kafka for the first time. And this is not important at all if you provide your own partitioning algorithm.
-James
On Jul 17, 2015, at 1:23 PM, Sivananda Reddy <si...@gmail.com> wrote:
> Hi Jay,
>
> I found it here:
> http://kafka.apache.org/documentation.html#newproducerconfigs, the same
> link is reported by James.
>
> @Joel: Thanks a lot for the info, I will use new producer
>
> Regards,
> Siva.
>
> On Fri, Jul 17, 2015 at 12:02 PM, James Cheng <jc...@tivo.com> wrote:
>
>> http://kafka.apache.org/documentation.html, Section 3.4.
>>
>>> 3.4 New Producer Configs
>>>
>>> We are working on a replacement for our existing producer. The code is
>> available in trunk now and can be considered beta quality. Below is the
>> configuration for the new producer.
>>
>> Sivananda might have seen it elsewhere, but this is where I found it.
>>
>> -James
>>
>> On Jul 17, 2015, at 9:49 AM, Jay Kreps <ja...@confluent.io> wrote:
>>
>>> Hey Sivananda,
>>>
>>> That's actually no longer true and likely a documentation bug. Where did
>>> you see that?
>>>
>>> -Jay
>>>
>>> On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <
>> sivananda2510@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Kafka document ion says that the new producer is in Beta state, how
>> safe is
>>>> it to use the new producer in production?. This is the first time I am
>>>> using Kafka for my application messaging needs. Please let me know.
>>>>
>>>> Thank you,
>>>> Siva.
>>>>
>>
>>
Re: New producer in production
Posted by Sivananda Reddy <si...@gmail.com>.
Hi Jay,
I found it here:
http://kafka.apache.org/documentation.html#newproducerconfigs, the same
link is reported by James.
@Joel: Thanks a lot for the info, I will use new producer
Regards,
Siva.
On Fri, Jul 17, 2015 at 12:02 PM, James Cheng <jc...@tivo.com> wrote:
> http://kafka.apache.org/documentation.html, Section 3.4.
>
> > 3.4 New Producer Configs
> >
> > We are working on a replacement for our existing producer. The code is
> available in trunk now and can be considered beta quality. Below is the
> configuration for the new producer.
>
> Sivananda might have seen it elsewhere, but this is where I found it.
>
> -James
>
> On Jul 17, 2015, at 9:49 AM, Jay Kreps <ja...@confluent.io> wrote:
>
> > Hey Sivananda,
> >
> > That's actually no longer true and likely a documentation bug. Where did
> > you see that?
> >
> > -Jay
> >
> > On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <
> sivananda2510@gmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> Kafka document ion says that the new producer is in Beta state, how
> safe is
> >> it to use the new producer in production?. This is the first time I am
> >> using Kafka for my application messaging needs. Please let me know.
> >>
> >> Thank you,
> >> Siva.
> >>
>
>
Re: New producer in production
Posted by James Cheng <jc...@tivo.com>.
http://kafka.apache.org/documentation.html, Section 3.4.
> 3.4 New Producer Configs
>
> We are working on a replacement for our existing producer. The code is available in trunk now and can be considered beta quality. Below is the configuration for the new producer.
Sivananda might have seen it elsewhere, but this is where I found it.
-James
On Jul 17, 2015, at 9:49 AM, Jay Kreps <ja...@confluent.io> wrote:
> Hey Sivananda,
>
> That's actually no longer true and likely a documentation bug. Where did
> you see that?
>
> -Jay
>
> On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <si...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Kafka document ion says that the new producer is in Beta state, how safe is
>> it to use the new producer in production?. This is the first time I am
>> using Kafka for my application messaging needs. Please let me know.
>>
>> Thank you,
>> Siva.
>>
Re: New producer in production
Posted by Jay Kreps <ja...@confluent.io>.
Hey Sivananda,
That's actually no longer true and likely a documentation bug. Where did
you see that?
-Jay
On Fri, Jul 17, 2015 at 9:35 AM, Sivananda Reddy <si...@gmail.com>
wrote:
> Hi,
>
> Kafka document ion says that the new producer is in Beta state, how safe is
> it to use the new producer in production?. This is the first time I am
> using Kafka for my application messaging needs. Please let me know.
>
> Thank you,
> Siva.
>
Re: New producer in production
Posted by Joel Koshy <jj...@gmail.com>.
It is pretty safe :) and adopting it now will save you the trouble
of migrating later.
At LinkedIn, we are using the new producer for all the producers that
the Kafka team directly own - this means mirror-maker pipelines, REST
proxies and some auditing tools. We have not yet migrated other
producers, but will likely proceed with that after
https://issues.apache.org/jira/browse/KAFKA-2120 is done. That is
probably not an extremely critical issue though if your Kafka
deployments are smaller and do not have hardware failures as often as
ours (which is mainly because we just happen to have so many brokers
some of which are just older machines).
Thanks,
Joel
On Fri, Jul 17, 2015 at 09:35:46AM -0700, Sivananda Reddy wrote:
> Hi,
>
> Kafka document ion says that the new producer is in Beta state, how safe is
> it to use the new producer in production?. This is the first time I am
> using Kafka for my application messaging needs. Please let me know.
>
> Thank you,
> Siva.