batching causes replica out of sync

[xiaoyu wang <xi...@gmail.com>]

Hi all,

We previously have replica.max.lag.message set to 4000 and use sync
producer to send data to kafka, one message at a time. With this, we don't
see many unclean leader election.

Recently, we switched to use sync producer and batch messages. After that,
we see unclean leader election more often. Did you observe the same?

Do you have recommendations about how broker configuration should be
adjusted based on the batch size?

Thanks,

RE: batching causes replica out of sync

[Aditya Auradkar <aa...@linkedin.com.INVALID>]

Xiaoyu,

Just FYI - Here's a discussion on this issue if you are interested.
https://issues.apache.org/jira/browse/KAFKA-1546

Aditya
________________________________________
From: Mayuresh Gharat [gharatmayuresh15@gmail.com]
Sent: Thursday, March 05, 2015 4:41 PM
To: users@kafka.apache.org
Subject: Re: batching causes replica out of sync

Whats the size of the batch that you are sending?
If the messages produced are more that 4000 at any point of time there is a
chance of the replica falling behind more than 4000 messages and being
kicked out of the ISR. This happens because the thread that checks for in
sync replicas is asynchronous. The replica gets added back once it catches
up. We at Linkedin have seen this in our test clusters.

If this happens and you loose the leader broker at this point you will have
unclean leader election. If this is the case you should increase the
replica.max.lag.message parameter value.

Thanks,

Mayuresh

On Thu, Mar 5, 2015 at 2:50 PM, xiaoyu wang <xi...@gmail.com> wrote:

> Hi all,
>
> We previously have replica.max.lag.message set to 4000 and use sync
> producer to send data to kafka, one message at a time. With this, we don't
> see many unclean leader election.
>
> Recently, we switched to use sync producer and batch messages. After that,
> we see unclean leader election more often. Did you observe the same?
>
> Do you have recommendations about how broker configuration should be
> adjusted based on the batch size?
>
> Thanks,
>



--
-Regards,
Mayuresh R. Gharat
(862) 250-7125

Re: batching causes replica out of sync

[Mayuresh Gharat <gh...@gmail.com>]

Whats the size of the batch that you are sending?
If the messages produced are more that 4000 at any point of time there is a
chance of the replica falling behind more than 4000 messages and being
kicked out of the ISR. This happens because the thread that checks for in
sync replicas is asynchronous. The replica gets added back once it catches
up. We at Linkedin have seen this in our test clusters.

If this happens and you loose the leader broker at this point you will have
unclean leader election. If this is the case you should increase the
replica.max.lag.message parameter value.

Thanks,

Mayuresh

On Thu, Mar 5, 2015 at 2:50 PM, xiaoyu wang <xi...@gmail.com> wrote:

> Hi all,
>
> We previously have replica.max.lag.message set to 4000 and use sync
> producer to send data to kafka, one message at a time. With this, we don't
> see many unclean leader election.
>
> Recently, we switched to use sync producer and batch messages. After that,
> we see unclean leader election more often. Did you observe the same?
>
> Do you have recommendations about how broker configuration should be
> adjusted based on the batch size?
>
> Thanks,
>



-- 
-Regards,
Mayuresh R. Gharat
(862) 250-7125