total.memory.bytes vs buffer.memory in 0.8.2.x

[Abdoulaye Diallo <ab...@gmail.com>]

Hi there,

I would like to get some help regarding the use of the two producer
properties mentioned above.

1. The API documentation of Producer.send
<http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html#send(org.apache.kafka.clients.producer.ProducerRecord,%20org.apache.kafka.clients.producer.Callback)>(...)
says this.

The producer manages a buffer of records waiting to be sent. This buffer
has a hard limit on it's size, which is controlled by the configuration
*totoal.memory.bytes*. If send() is called faster than the I/O thread can
transfer data to the brokers the buffer will eventually run out of space. The
default behavior in this case is to block the send call until the I/O
thread catches up and more buffer space is available. However in cases
where non-blocking usage is desired the setting block.on.buffer.full=false

2. The project documentation
<http://kafka.apache.org/documentation.html#newproducerconfigs> says

*buffer.memory*:The total bytes of memory the producer can use to buffer
records waiting to be sent to the server. If records are sent faster than
they can be delivered to the server the producer will either block or throw
an exception based on the preference specified by block.on.buffer.full


I want to know which one of these two is really used to limit the
bufferpool size. And what is the remaining one for. It's a bit confusing,
they seems to be doing the same thing and yet called differently.


Thank you in advance,

-- 
Abdoul