You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Ahmed khan <io...@hotmail.com> on 2014/10/15 13:11:44 UTC

Review request - Producer managed offsets

We have a requirement where in only one producer should be writing to a partition at once.The producer will be managing it's own offsets, what it does is at the time of startup gets latest offsets from Kafka and while producing the messages it assigns this offsets to the message.For this we had to write our own producer (which will be open sourced soon).And also create this patch to validate the offsets which are sent by the producer.The patch will just check for this produce request if it's coming from our own producer then it'll validate the offset and returns error which the producer will recover from. 
If possible can any kafka developer do a quick review of it please? The only concern here is can it be possible that there'll by any gap in offsets assigned to messages by Kafka. We will never be enabling compression on our topics.  
https://github.com/makfair/kafka
Diff here:https://github.com/makfair/kafka/commit/241f5a2175b1dcaca00e663e896b5c9a52a75212 Thanks,Ahmed