You are viewing a plain text version of this content. The canonical link for it is here.

Posted to log4j-dev@logging.apache.org by Ralph Goers <ra...@dslextreme.com> on 2016/04/26 23:23:50 UTC

Performance page

Remko,

The performance page hasn’t been updated in quite some time.  It really could use some of your nice graphs.  Also, it isn’t linked from the main menu. The only way I know of to find it is by clicking a link embedded in the About page.  Do you think you could enhance that page?   It would be great to see a variation of the chart you made for garbage free logging, although I am just thinking of a few charts comparing Log4j 1, Log4j 2, Logback and JUL in various scenarios.  Something similar to https://www.loggly.com/blog/benchmarking-java-logging-frameworks/ <https://www.loggly.com/blog/benchmarking-java-logging-frameworks/> would be nice, but with similar configurations instead of relying on defaults.

 I am still aware that we may need to make improvements to the SocketAppender. I still intend to create a version that uses Netty, 

Ralph

Re: Performance page

Posted by Ralph Goers <ra...@dslextreme.com>.

Not the whole site. The source and download links will point to a snapshot.  However, we can update individual pages.

Ralph

> On May 7, 2016, at 8:06 PM, Remko Popma <re...@gmail.com> wrote:
> 
> You're asking if it's possible to change the site without doing a release. I'm not sure, maybe Ralph knows. 
> 
> Sent from my iPhone
> 
>> On 2016/05/08, at 11:01, Matt Sicker <bo...@gmail.com> wrote:
>> 
>> For the "nice to have" items, can we rebuild the site after 2.6 and just commit that to svn? Or do site releases always need to match up with the artifacts?
>> 
>>> On 7 May 2016 at 19:32, Remko Popma <re...@gmail.com> wrote:
>>> Understood about Rewrite. 
>>> 
>>> I figured that the performance page is the last remaining thing for the 2.6 release. 
>>> 
>>> I had a week off from work and made reasonable progress but it is surprisingly time consuming to create the data and then analyze it and do a write up. 
>>> 
>>> Still to do "must have":
>>> - JMH benchmarks for comparing performance of log4j2's Filter to Logback's TurboFilters  
>>> - rewrite Async Logger latency section
>>> - Log4j2: compare File, RandomAccessFile, MemoryMappedFile, Console and Rewrite appenders (remove RandomAccessFile stuff from Async Logger page)
>>> 
>>> "Nice to have":
>>> - Compare socket/syslog appenders between logging libraries
>>> - log4j2: Cost of various APIs/wrappers (SLF4J, Log4j1, JUL, Commons Logging)
>>> - log4j2: Compare performance all layouts (CSV, Gelf, HTML, JSON, Pattern, RFC-5424, Serialized, Syslog, XML). 
>>> 
>>> The full todo list is here: https://issues.apache.org/jira/browse/LOG4J2-1179
>>> 
>>> Please do "mvn site", take a look at the current state of the Performance page and let me know if you find any issues. 
>>> 
>>> I might postpone some or all of the "nice to have" items to later, unless someone disagrees. 
>>> 
>>> Sent from my iPhone
>>> 
>>>> On 2016/05/08, at 4:02, Ralph Goers <ra...@dslextreme.com> wrote:
>>>> 
>>>> No, I have nothing particular in mind for that.
>>>> 
>>>> Just to let you know - and not to put any pressure on - I am waiting for this page before we do a release. I think it is pretty important, especially with the detailed results you are producing.
>>>> 
>>>> Ralph
>>>> 
>>>>> On May 7, 2016, at 9:33 AM, Remko Popma <re...@gmail.com> wrote:
>>>>> 
>>>>> Ralph, 
>>>>> 
>>>>> Started to work on a benchmark to compare Log4j2's File, RandomAccessFile, MemoryMappedFile, Console and Rewrite appenders.
>>>>> Is there any particular scenario you want to test with Rewrite appenders?
>>>>> 
>>>>> Remko 
>>>>> 
>>>>>> On Mon, May 2, 2016 at 11:51 PM, Remko Popma <re...@gmail.com> wrote:
>>>>>> I will flesh things out further in this ticket: https://issues.apache.org/jira/browse/LOG4J2-1179
>>>>>> 
>>>>>> 
>>>>>>> On Mon, May 2, 2016 at 1:42 AM, Ralph Goers <ra...@dslextreme.com> wrote:
>>>>>>> This all sounds really good!
>>>>>>> 
>>>>>>> Yes, the filter comparison isn’t as clear as it should be. Yes, it is how long it takes to deny the events in nanoseconds. The results were generated by the FilterPerformanceComparison test class.  At the time I tested with 2 Filters - a Marker filter and the MDC filter.  If you look at BasicMarker in SLF4J you will notice that the hasReferences method is synchronized. In multi-threaded cases this can cause huge performance issues if you have a Marker with references to other markers (I personally experienced this).  Also, the referenceList is a Vector, so the contains methods can throw ConcurrentModificationExceptions if Markers are modified during execution, even though the add and remove methods are synchronized the contains method is not.
>>>>>>> 
>>>>>>> The test is currently set up to compare the MDC/ThreadContext Filters during the build. However, I am not sure it is correct. I don’t actually see it setting the value in the thread context, so it wouldn’t be denying the events.
>>>>>>> 
>>>>>>> Ralph
>>>>>>> 
>>>>>>>> On May 1, 2016, at 12:44 AM, Remko Popma <re...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I've started to work on this.
>>>>>>>> 
>>>>>>>> Question: 
>>>>>>>> the current page has a section comparing the performance Log4j 2 Filters to Logback Filters. Can you point me to the test that generated these results? What is being tested? I am guessing this is how long it takes to call logger.log(level, msg) when the filter DENIES the event, is this correct? Is the unit for the result numbers nanosecond/operation?
>>>>>>>> 
>>>>>>>> I'm thinking to convert the above filter test to a JMH benchmark. This would allow people to quickly verify the results on their own machine.
>>>>>>>> 
>>>>>>>> I need to think a bit on the items to put on the performance page (and to create benchmarks for).
>>>>>>>> 
>>>>>>>> * One theme will be a comparison of Log4j 2 to other logging libraries like Logback/Log4j1/JUL. One thing to measure is synchronized logging throughput. I'll use this chart for the performance page and use a simpler graph on the garbage-free manual page.
>>>>>>>> * Comparing File/RandomAccessFile/MemoryMappedFile appenders
>>>>>>>> * Documenting the performance of TCP/UDP appenders
>>>>>>>> * Compare parameterized logging to string concatenation
>>>>>>>> * Perhaps investigate/document the overhead of using the adapter/bridge modules (Log4j1-api, commons-logging, slf4j, JUL adapter)?
>>>>>>>> * Lock-free async logging versus blocking async logging (just a paragraph with link to the async logging manual page)
>>>>>>>> * Any Layouts to compare?
>>>>>>>> 
>>>>>>>> I plan to do most of the performance testing with JMH. The response time test I'm using to test garbage-free latency gives a more nuanced picture but is more work to execute, gather results, analyse etc.
>>>>>>>> 
>>>>>>>> I'll flesh this out more over the coming week. Feedback welcome as always.
>>>>>>>> 
>>>>>>>> Remko
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Wed, Apr 27, 2016 at 10:09 AM, Ralph Goers <ra...@dslextreme.com> wrote:
>>>>>>>>> Either in the manual or a link above it
>>>>>>>>> 
>>>>>>>>> Sent from my iPhone
>>>>>>>>> 
>>>>>>>>>> On Apr 26, 2016, at 4:41 PM, Remko Popma <re...@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Will do. Was on my TODO-eventually list but I'll move it up. :-)
>>>>>>>>>> Shall we included it in the manual then?
>>>>>>>>>> 
>>>>>>>>>>> On Wed, Apr 27, 2016 at 6:23 AM, Ralph Goers <ra...@dslextreme.com> wrote:
>>>>>>>>>>> Remko,
>>>>>>>>>>> 
>>>>>>>>>>> The performance page hasn’t been updated in quite some time.  It really could use some of your nice graphs.  Also, it isn’t linked from the main menu. The only way I know of to find it is by clicking a link embedded in the About page.  Do you think you could enhance that page?   It would be great to see a variation of the chart you made for garbage free logging, although I am just thinking of a few charts comparing Log4j 1, Log4j 2, Logback and JUL in various scenarios.  Something similar to https://www.loggly.com/blog/benchmarking-java-logging-frameworks/ would be nice, but with similar configurations instead of relying on defaults.
>>>>>>>>>>> 
>>>>>>>>>>>  I am still aware that we may need to make improvements to the SocketAppender. I still intend to create a version that uses Netty, 
>>>>>>>>>>> 
>>>>>>>>>>> Ralph
>> 
>> 
>> 
>> -- 
>> Matt Sicker <bo...@gmail.com>