[jira] [Created] (KAFKA-3588) Document how Producer Settings affect In Sync Replicas

["Alan Braithwaite (JIRA)" <ji...@apache.org>]

Alan Braithwaite created KAFKA-3588:
---------------------------------------

             Summary: Document how Producer Settings affect In Sync Replicas
                 Key: KAFKA-3588
                 URL: https://issues.apache.org/jira/browse/KAFKA-3588
             Project: Kafka
          Issue Type: Improvement
            Reporter: Alan Braithwaite
            Priority: Minor


One of the big changes from 0.8.x to 0.9.x was of course the way that Kafka handles ISRs and replication.  The blog post [1] goes into detail about some of the issues (and actually hits on the point I'm going to make) but misses a key point about replication.

If the window sizes (timeouts, min/max fetch sizes) between the replica consumer and the producer for the topic are not consistent, then you're likely to see replicas fall out of sync.

e.g. If you've got a producer with a window (linger.ms/batch.size) which is very small and a replica window which is very large (replica.fetch.min.bytes	/replica.lag.time.max.ms), then the amount of time for the replica fetch to actually cause the replica to join the ISR is very small.

I'll include more examples when I have them, but I'm making this now because I didn't want to forget to do it.

[1] http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)