[jira] [Commented] (KAFKA-1351) String.format is very expensive in Scala

["Fabian Lange (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243223#comment-14243223 ] 

Fabian Lange commented on KAFKA-1351:
-------------------------------------

Hi,
I was about to create a new issue, but figured this might host the comment almost as good.
As William outlined here: http://www.autoletics.com/posts/quick-performance-hotspot-analysis-apache-kafka
the Log.append function (https://github.com/apache/kafka/blob/7847e9c703f3a0b70519666cdb8a6e4c8e37c3a7/core/src/main/scala/kafka/log/Log.scala#L230)
is by far the biggest contributor to response time.
There are a few reasons for this:
A) it has a long synchronized block. I do not know much about the code, so at first glance I do not see much of an option to fix something
B) it has a trace logging call. Not sure if that trace call needs to be in synchronized though.

The trace call is implemented by
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/Logging.scala#L34

As you might notice this does doe a slow String.format for trace. a log level which is very likely to be off.
The above mentioned Log4j2 pattern would help there.

However there are more options, like implemented by typesafe:
https://github.com/typesafehub/scalalogging/tree/master/scalalogging-log4j/src/main/scala/com/typesafe/scalalogging/log4j
they use a macro, which would transform the code so that the actual string format would be moved into the log level checking. I think thats quite nifty, but maybe too much work to integrate.

So if you would check out that trace call, I would appreciate that :)

> String.format is very expensive in Scala
> ----------------------------------------
>
>                 Key: KAFKA-1351
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1351
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7.2, 0.8.0, 0.8.1
>            Reporter: Neha Narkhede
>              Labels: newbie
>             Fix For: 0.8.3
>
>         Attachments: KAFKA-1351.patch, KAFKA-1351_2014-04-07_18:02:18.patch, KAFKA-1351_2014-04-09_15:40:11.patch
>
>
> As found in KAFKA-1350, logging is causing significant overhead in the performance of a Kafka server. There are several info statements that use String.format which is particularly expensive. We should investigate adding our own version of String.format that merely uses string concatenation under the covers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)