Garbage free Kafka client

[Mikael Ståldal <mi...@magine.com>]

Hi.

I am committer on the Apache Log4j 2 project. We are working on providing
garbage-free logging, as described here:
http://logging.apache.org/log4j/2.x/manual/garbagefree.html

I work on the Kafka appender in Log4j 2:
http://logging.apache.org/log4j/2.x/manual/appenders.html#KafkaAppender
https://github.com/apache/logging-log4j2/tree/master/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/mom/kafka

I am now looking into if it is feasible to support garbage-free operation
with the Kafka appender.

This would require some changes in the Kafka client library. The first step
would be to make it possible to send a message without creating a new
ProducerRecord object each time. Maybe an additional "low-level" send
method can be added to the Producer interface with unrolled arguments, like
this:

/**
 * Send the given record asynchronously and return a future which will
eventually contain the response information.
 *
 * @param topic The topic the record will be appended to
 * @param key The key that will be included in the record, or
{@literal null} for no key
 * @param value The record contents
 *
 * @return A future which will eventually contain the response information
 */
public Future<RecordMetadata> send(String topic, byte[] key, byte[] value);

Perhaps it would be good to bypass key and value serializers in this case.

-- 
[image: MagineTV]

*Mikael Ståldal*
Senior software developer

*Magine TV*
mikael.staldal@magine.com
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.