Change default min.insync.replicas for cluster

[Bryan Baugher <bj...@gmail.com>]

Hi everyone,

I'm currently looking to upgrade my cluster from 0.8.1.1 to 0.8.2.1. Is it
possible to change the default min.insync.replicas at the cluster level? It
is not listed as a broker config but only as a topic config.

Similarly my producers are currently using request.required.acks to 2 with
our topics having a replication factor of 3 to kind of get the same thing
as min.insync.replicas = 2. I know this is no longer allowed but will my
producers fail to write with request.required.acks = 2 in a 0.8.2.1 cluster?

Bryan

Re: Change default min.insync.replicas for cluster

[Bryan Baugher <bj...@gmail.com>]

Sure, https://issues.apache.org/jira/browse/KAFKA-2114

On Sat, Apr 11, 2015 at 3:16 PM Gwen Shapira <gs...@cloudera.com> wrote:

> mmmm.... this seems like a bug. We expect broker settings to set
> defaults for topics. Perhaps open a JIRA?
>
> On Fri, Apr 10, 2015 at 1:32 PM, Bryan Baugher <bj...@gmail.com> wrote:
> > To answer my own question via testing, setting min.insync.replicas on the
> > broker does not change the default. The only way I can find to change
> this
> > is by editing the topic config.
> >
> > On Fri, Apr 10, 2015 at 12:57 PM Kamal C <ka...@gmail.com> wrote:
> >
> >> >
> >> >
> >> >
> >> > Similarly my producers are currently using request.required.acks to 2
> >> with
> >> > our topics having a replication factor of 3 to kind of get the same
> thing
> >> > as min.insync.replicas = 2. I know this is no longer allowed but will
> my
> >> > producers fail to write with request.required.acks = 2 in a 0.8.2.1
> >> > cluster?
> >> >
> >> >
> >> There was a https://issues.apache.org/jira/browse/KAFKA-1697 JIRA
> filed to
> >> remove this support. Kafka 0.8.2.1 will log a warning if ack>1. In the
> new
> >> Shiny Producer 0.8.2.1, you can wait on the returned future to know
> whether
> >> the message / record successfully reached to Kafka broker or not.
> >>
> >> producer.send(new ProducerRecord<byte[], byte[]>("the-topic",
> >> "key".getBytes(), "value".getBytes()).get();
> >>
> >>
> >> --
> >> Kamal
> >>
>

Re: Change default min.insync.replicas for cluster

[Gwen Shapira <gs...@cloudera.com>]

mmmm.... this seems like a bug. We expect broker settings to set
defaults for topics. Perhaps open a JIRA?

On Fri, Apr 10, 2015 at 1:32 PM, Bryan Baugher <bj...@gmail.com> wrote:
> To answer my own question via testing, setting min.insync.replicas on the
> broker does not change the default. The only way I can find to change this
> is by editing the topic config.
>
> On Fri, Apr 10, 2015 at 12:57 PM Kamal C <ka...@gmail.com> wrote:
>
>> >
>> >
>> >
>> > Similarly my producers are currently using request.required.acks to 2
>> with
>> > our topics having a replication factor of 3 to kind of get the same thing
>> > as min.insync.replicas = 2. I know this is no longer allowed but will my
>> > producers fail to write with request.required.acks = 2 in a 0.8.2.1
>> > cluster?
>> >
>> >
>> There was a https://issues.apache.org/jira/browse/KAFKA-1697 JIRA filed to
>> remove this support. Kafka 0.8.2.1 will log a warning if ack>1. In the new
>> Shiny Producer 0.8.2.1, you can wait on the returned future to know whether
>> the message / record successfully reached to Kafka broker or not.
>>
>> producer.send(new ProducerRecord<byte[], byte[]>("the-topic",
>> "key".getBytes(), "value".getBytes()).get();
>>
>>
>> --
>> Kamal
>>

Re: Change default min.insync.replicas for cluster

[Bryan Baugher <bj...@gmail.com>]

To answer my own question via testing, setting min.insync.replicas on the
broker does not change the default. The only way I can find to change this
is by editing the topic config.

On Fri, Apr 10, 2015 at 12:57 PM Kamal C <ka...@gmail.com> wrote:

> >
> >
> >
> > Similarly my producers are currently using request.required.acks to 2
> with
> > our topics having a replication factor of 3 to kind of get the same thing
> > as min.insync.replicas = 2. I know this is no longer allowed but will my
> > producers fail to write with request.required.acks = 2 in a 0.8.2.1
> > cluster?
> >
> >
> There was a https://issues.apache.org/jira/browse/KAFKA-1697 JIRA filed to
> remove this support. Kafka 0.8.2.1 will log a warning if ack>1. In the new
> Shiny Producer 0.8.2.1, you can wait on the returned future to know whether
> the message / record successfully reached to Kafka broker or not.
>
> producer.send(new ProducerRecord<byte[], byte[]>("the-topic",
> "key".getBytes(), "value".getBytes()).get();
>
>
> --
> Kamal
>

Re: Change default min.insync.replicas for cluster

[Kamal C <ka...@gmail.com>]

>
>
>
> Similarly my producers are currently using request.required.acks to 2 with
> our topics having a replication factor of 3 to kind of get the same thing
> as min.insync.replicas = 2. I know this is no longer allowed but will my
> producers fail to write with request.required.acks = 2 in a 0.8.2.1
> cluster?
>
>
There was a https://issues.apache.org/jira/browse/KAFKA-1697 JIRA filed to
remove this support. Kafka 0.8.2.1 will log a warning if ack>1. In the new
Shiny Producer 0.8.2.1, you can wait on the returned future to know whether
the message / record successfully reached to Kafka broker or not.

producer.send(new ProducerRecord<byte[], byte[]>("the-topic",
"key".getBytes(), "value".getBytes()).get();


--
Kamal