Kafka Synchronous Producer Recommended Configuration

[Tom Dearman <to...@gmail.com>]

Our application that we use with kafka requires that we know the request has reliably reached the leader and 1 insync node (we usemin.insync.replicas=2 and producer acks=all) before we continue with our db transaction.  To do this we call the future objects get with a timeout of 5 seconds  and roll back the request and db transaction if the request times out.  When we take down one of our 3 kafka nodes (particularly if it is the active controller) we find that lots of messages time out.  A lost of the default settings for kafka such as producer request time out (30000 ms) don’t look that sensible for our set up, are there any recommendations for broker and producer settings  for a ‘synchronous’ usage like ours.

Tom