[jira] [Commented] (LOG4J2-1733) Add SyncSend attribute to KafkaAppender (as in KafkaLog4jAppender)

["Vincent Tieleman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LOG4J2-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834462#comment-15834462 ] 

Vincent Tieleman commented on LOG4J2-1733:
------------------------------------------

Extensive testing has shown that when syncSend is set to false, any errors that occur while sending (i.e. exceptions that are thrown) are not propagated to log4j2 anymore. This means messages may be lost without notice, in my opinion a serious offence.

What needs to be done is add a callback method in the send method of the KafkaManager class that catches any asynchronous exceptions that occur and pass them on to log4j2. For instance:

    public void send(final byte[] msg) throws ExecutionException, InterruptedException, TimeoutException {
        if (producer != null) {
            ProducerRecord<byte[], byte[]> newRecord =
                new ProducerRecord<byte[], byte[]>(topic, msg);
            if (syncSend) {
                Future<RecordMetadata> response = producer.send(newRecord);
                response.get(timeoutMillis, TimeUnit.MILLISECONDS);
            } else {
                producer.send(newRecord, new Callback() {
                    public void onCompletion(RecordMetadata metadata, Exception e) {
                        if (e != null) {
                            e.printStackTrace();
                        }
                    }
                });
            }
        }
    }

However, I am not sure how log4j2 is expected to behave when one of its appenders has an exception. Should the exception be passed on to log4j2, or should it simply be printed as in the above code example?

Second, I would like to know whether I should open another ticket for this issue or whether we simply re-open the existing one.



> Add SyncSend attribute to KafkaAppender (as in KafkaLog4jAppender)
> ------------------------------------------------------------------
>
>                 Key: LOG4J2-1733
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-1733
>             Project: Log4j 2
>          Issue Type: Improvement
>          Components: Appenders
>    Affects Versions: 2.5
>         Environment: Kafka 0.10.1.0
>            Reporter: Vincent Tieleman
>            Assignee: Mikael Ståldal
>             Fix For: 2.8
>
>
> The KafkaLog4jAppender (shipped with Kafka and usable with log4j), has a syncsend property that improves performance significantly by not blocking a send to Kafka. I've tried many other configuration settings and setups, but find that none of these approaches improved the logging performance to Kafka as significantly as setting the syncsend property to false.
> Unfortunately, the syncsend property is not supported by the KafkaAppender shipped with log4j2 and the KafkaLog4jAppender only supports log4j, so I am stuck with forking the code and making the change myself.
> Could you please introduce this property in the KafkaAppender in log4j2?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org