[jira] [Commented] (SOLR-15477) Logging Performance

["Mark Robert Miller (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371453#comment-17371453 ] 

Mark Robert Miller commented on SOLR-15477:
-------------------------------------------

The best performance from the log4j2 async loggers is realized when you also try to take as much advantage as you can of the garbage free logging features.

Some of additional configuration, much of really just cascading off disabling auto webapp detection:

 


||SysProp||Desc||
|log4j2.is.webapp=false|Do not use auto detection for determining webapp behavior. This behavior assumes a webapp in a general container that does not own the system logging, and given that restraint, some other optimizations we want to enable are not used.|
|log4j2.garbagefreeThreadContextMap|Enable garbage free mode for log4j2's MDC support. Not currently likely to do much, but log4j2 MDC efficiency takes a hit going through SLF4j and it's possible it may be used directly in the future IMO.|
|log4j2.enableDirectEncoders|if "true" (the default) log events are converted to text and this text is converted to bytes without creating temporary objects. Note: _synchronous_ logging performance may be worse for multi-threaded applications in this mode due to synchronization on the shared buffer. If your application is multi-threaded and logging performance is important, consider using Async Loggers.|
|log4j2.enable.threadlocals|if "true" (the default for non-web applications) objects are stored in ThreadLocal fields and reused, otherwise new objects are created for each log event.|
|log4j2.disable.jmx|Given that JMX is not a robust, high scale metrics reporting setup, no need to pay for this for those using prometheus or other performant solutions.|

Beyond that, the conversion pattern must use only approved no gc supported options (https://logging.apache.org/log4j/2.x/manual/garbagefree.html). In general, our current patterns largely already do this, although regex shortening or conversion can cost and exception logging will cost, but hopefully be exceptional anyway.

 

> Logging Performance
> -------------------
>
>                 Key: SOLR-15477
>                 URL: https://issues.apache.org/jira/browse/SOLR-15477
>             Project: Solr
>          Issue Type: Sub-task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Mark Robert Miller
>            Priority: Major
>
> Once you remove almost all performance issues and unnecessary garbage generation, logging starts to sit at the top of method and gc perf profiling (well, xml config still competes on gc - you can improve that 90% and it's still fights to be king of the hill) during test runs. This is the case even with fairly reduced logging and configuration optimizations.
> At that point, exception logging is the big killer (no garbage free option), so in a mostly clean running system this will likely be less the case.
> There are a variety of configuration options that can boost the logging impact. Fairly critical to it all is telling log4j2 that no, Solr is not a webapp, please dont punish.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org