You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Volkan Yazıcı <vo...@yazi.ci> on 2024/01/15 12:32:27 UTC

[log4j] Performance figures

*TLDR:* I want to remove performance figures from the Log4j website,
because they don't serve any practical value anymore.

Log4j website shares performance figures in several pages; Performance
<https://logging.apache.org/log4j/2.x/performance.html#benchmarks>,
Asynchronous
Logging
<https://logging.apache.org/log4j/2.x/manual/async.html#asynchronous-logging-performance>,
Garbage-free Logging
<https://logging.apache.org/log4j/2.x/manual/garbagefree.html> are among
the well-known ones. I want to remove all performance figures and only keep
pragmatic recommendations due to following reasons:

   - *Insufficient relevancy* – Shared figures were mostly produced using
   Log4j version `2.5` and `2.6`. These releases date back from late 2016 and
   *a lot* has changed since then.
   - *Insufficient reliability* – There were many occasions PMC members
   stated they weren't able to reproduce these figures.
   - *Error prone* – As tipped in the website, measuring performance
   correctly is very difficult
   <https://www.infoq.com/presentations/latency-response-time>.
   - *Context dependent* – Performance is an extremely subjective term. It
   requires context. What kind of an application? What is the application
   load? What is the logging load? What is the logging setup? etc. Sharing a
   meaningful figure here that users can benefit in any way is, IMHO,
   impossible.
   - *2.x vs 3.x* – We are ramping down for the `3.0.0` release. I doubt if
   any of the existing performance figures (produced using ~8 year old Log4j)
   are applicable to Log4j 3.

Will we have no performance figures at all? AFAIC, we need to have a
continuous performance testbed that would not only give users an indication
about performance of Log4j over time (in a reproducible way!), but also,
maybe more importantly, guide maintainers on making changes affecting
performance. Some of you might recall: I already had implemented some stuff
on this subject and had a "a continuous performance testbed"  project in my
first STF projects draft. But we needed to drop it due to other pressing
issues and insufficient budget. We can bring it up again if need (and
budget?) arises. Let me know if you and/or your employer would be
interested in funding such an effort.

Re: [log4j] Performance figures

Posted by Gary Gregory <ga...@gmail.com>.
Hi Remko! Just saying "Hi".

Gary

On Tue, Jan 16, 2024, 1:40 AM Remko Popma <re...@gmail.com> wrote:

> I’m open to improvements in this area.
>
> Before going into details or specifics though, I’m curious:
>
> What do we (users, developers and PMC members) consider to be the “value
> proposition” of Log4j? Why should people choose Log4j over the
> alternatives?
>
> This is a positioning question; what are the strengths and weaknesses of
> Log4j and how should Log4j position itself in the market of logging
> solutions?
>
> Remko
>
>
> > On Jan 15, 2024, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
> >
> > We should IMO keep this information available _somewhere_, maybe in a
> new
> > stable historical-archival section of the site. I'm not a fan of using
> the
> > wiki because that's yet another place to look for information and it
> feels
> > transitory, unstable (as far as information permanance), and more like
> > something we should use (if at all) as a scratch pad.
> >
> > Gary
> >
> >> On Mon, Jan 15, 2024, 7:34 AM Volkan Yazıcı <vo...@yazi.ci> wrote:
> >>
> >> *TLDR:* I want to remove performance figures from the Log4j website,
> >> because they don't serve any practical value anymore.
> >>
> >> Log4j website shares performance figures in several pages; Performance
> >> <https://logging.apache.org/log4j/2.x/performance.html#benchmarks>,
> >> Asynchronous
> >> Logging
> >> <
> >>
> https://logging.apache.org/log4j/2.x/manual/async.html#asynchronous-logging-performance
> >>> ,
> >> Garbage-free Logging
> >> <https://logging.apache.org/log4j/2.x/manual/garbagefree.html> are
> among
> >> the well-known ones. I want to remove all performance figures and only
> keep
> >> pragmatic recommendations due to following reasons:
> >>
> >>   - *Insufficient relevancy* – Shared figures were mostly produced using
> >>   Log4j version `2.5` and `2.6`. These releases date back from late 2016
> >> and
> >>   *a lot* has changed since then.
> >>   - *Insufficient reliability* – There were many occasions PMC members
> >>   stated they weren't able to reproduce these figures.
> >>   - *Error prone* – As tipped in the website, measuring performance
> >>   correctly is very difficult
> >>   <https://www.infoq.com/presentations/latency-response-time>.
> >>   - *Context dependent* – Performance is an extremely subjective term.
> It
> >>   requires context. What kind of an application? What is the application
> >>   load? What is the logging load? What is the logging setup? etc.
> Sharing
> >> a
> >>   meaningful figure here that users can benefit in any way is, IMHO,
> >>   impossible.
> >>   - *2.x vs 3.x* – We are ramping down for the `3.0.0` release. I doubt
> if
> >>   any of the existing performance figures (produced using ~8 year old
> >> Log4j)
> >>   are applicable to Log4j 3.
> >>
> >> Will we have no performance figures at all? AFAIC, we need to have a
> >> continuous performance testbed that would not only give users an
> indication
> >> about performance of Log4j over time (in a reproducible way!), but also,
> >> maybe more importantly, guide maintainers on making changes affecting
> >> performance. Some of you might recall: I already had implemented some
> stuff
> >> on this subject and had a "a continuous performance testbed"  project
> in my
> >> first STF projects draft. But we needed to drop it due to other pressing
> >> issues and insufficient budget. We can bring it up again if need (and
> >> budget?) arises. Let me know if you and/or your employer would be
> >> interested in funding such an effort.
> >>
>

Re: [log4j] Performance figures

Posted by Remko Popma <re...@gmail.com>.
I’m open to improvements in this area. 

Before going into details or specifics though, I’m curious:

What do we (users, developers and PMC members) consider to be the “value proposition” of Log4j? Why should people choose Log4j over the alternatives? 

This is a positioning question; what are the strengths and weaknesses of Log4j and how should Log4j position itself in the market of logging solutions?

Remko


> On Jan 15, 2024, at 22:05, Gary Gregory <ga...@gmail.com> wrote:
> 
> We should IMO keep this information available _somewhere_, maybe in a new
> stable historical-archival section of the site. I'm not a fan of using the
> wiki because that's yet another place to look for information and it feels
> transitory, unstable (as far as information permanance), and more like
> something we should use (if at all) as a scratch pad.
> 
> Gary
> 
>> On Mon, Jan 15, 2024, 7:34 AM Volkan Yazıcı <vo...@yazi.ci> wrote:
>> 
>> *TLDR:* I want to remove performance figures from the Log4j website,
>> because they don't serve any practical value anymore.
>> 
>> Log4j website shares performance figures in several pages; Performance
>> <https://logging.apache.org/log4j/2.x/performance.html#benchmarks>,
>> Asynchronous
>> Logging
>> <
>> https://logging.apache.org/log4j/2.x/manual/async.html#asynchronous-logging-performance
>>> ,
>> Garbage-free Logging
>> <https://logging.apache.org/log4j/2.x/manual/garbagefree.html> are among
>> the well-known ones. I want to remove all performance figures and only keep
>> pragmatic recommendations due to following reasons:
>> 
>>   - *Insufficient relevancy* – Shared figures were mostly produced using
>>   Log4j version `2.5` and `2.6`. These releases date back from late 2016
>> and
>>   *a lot* has changed since then.
>>   - *Insufficient reliability* – There were many occasions PMC members
>>   stated they weren't able to reproduce these figures.
>>   - *Error prone* – As tipped in the website, measuring performance
>>   correctly is very difficult
>>   <https://www.infoq.com/presentations/latency-response-time>.
>>   - *Context dependent* – Performance is an extremely subjective term. It
>>   requires context. What kind of an application? What is the application
>>   load? What is the logging load? What is the logging setup? etc. Sharing
>> a
>>   meaningful figure here that users can benefit in any way is, IMHO,
>>   impossible.
>>   - *2.x vs 3.x* – We are ramping down for the `3.0.0` release. I doubt if
>>   any of the existing performance figures (produced using ~8 year old
>> Log4j)
>>   are applicable to Log4j 3.
>> 
>> Will we have no performance figures at all? AFAIC, we need to have a
>> continuous performance testbed that would not only give users an indication
>> about performance of Log4j over time (in a reproducible way!), but also,
>> maybe more importantly, guide maintainers on making changes affecting
>> performance. Some of you might recall: I already had implemented some stuff
>> on this subject and had a "a continuous performance testbed"  project in my
>> first STF projects draft. But we needed to drop it due to other pressing
>> issues and insufficient budget. We can bring it up again if need (and
>> budget?) arises. Let me know if you and/or your employer would be
>> interested in funding such an effort.
>> 

Re: [log4j] Performance figures

Posted by Gary Gregory <ga...@gmail.com>.
We should IMO keep this information available _somewhere_, maybe in a new
stable historical-archival section of the site. I'm not a fan of using the
wiki because that's yet another place to look for information and it feels
transitory, unstable (as far as information permanance), and more like
something we should use (if at all) as a scratch pad.

Gary

On Mon, Jan 15, 2024, 7:34 AM Volkan Yazıcı <vo...@yazi.ci> wrote:

> *TLDR:* I want to remove performance figures from the Log4j website,
> because they don't serve any practical value anymore.
>
> Log4j website shares performance figures in several pages; Performance
> <https://logging.apache.org/log4j/2.x/performance.html#benchmarks>,
> Asynchronous
> Logging
> <
> https://logging.apache.org/log4j/2.x/manual/async.html#asynchronous-logging-performance
> >,
> Garbage-free Logging
> <https://logging.apache.org/log4j/2.x/manual/garbagefree.html> are among
> the well-known ones. I want to remove all performance figures and only keep
> pragmatic recommendations due to following reasons:
>
>    - *Insufficient relevancy* – Shared figures were mostly produced using
>    Log4j version `2.5` and `2.6`. These releases date back from late 2016
> and
>    *a lot* has changed since then.
>    - *Insufficient reliability* – There were many occasions PMC members
>    stated they weren't able to reproduce these figures.
>    - *Error prone* – As tipped in the website, measuring performance
>    correctly is very difficult
>    <https://www.infoq.com/presentations/latency-response-time>.
>    - *Context dependent* – Performance is an extremely subjective term. It
>    requires context. What kind of an application? What is the application
>    load? What is the logging load? What is the logging setup? etc. Sharing
> a
>    meaningful figure here that users can benefit in any way is, IMHO,
>    impossible.
>    - *2.x vs 3.x* – We are ramping down for the `3.0.0` release. I doubt if
>    any of the existing performance figures (produced using ~8 year old
> Log4j)
>    are applicable to Log4j 3.
>
> Will we have no performance figures at all? AFAIC, we need to have a
> continuous performance testbed that would not only give users an indication
> about performance of Log4j over time (in a reproducible way!), but also,
> maybe more importantly, guide maintainers on making changes affecting
> performance. Some of you might recall: I already had implemented some stuff
> on this subject and had a "a continuous performance testbed"  project in my
> first STF projects draft. But we needed to drop it due to other pressing
> issues and insufficient budget. We can bring it up again if need (and
> budget?) arises. Let me know if you and/or your employer would be
> interested in funding such an effort.
>