You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Volkan Yazıcı <vo...@yazi.ci.INVALID> on 2024/10/14 19:30:29 UTC
[log4j] Using `DateTimeFormatter` by default
*Abstract:* Log4j contains two custom date & time formatting classes. All
our competitors (Logback
<https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>,
Tinylog
<https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>,
etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
their performance issues. Shall we switch to `DateTimeFormatter` as the
default in the next minor release and make the custom implementations
opt-in?
Log4j ships two custom date & time formatting classes:
- `FixedDateFormat`
- Supports a hardcoded list of formats
- Bug: Conflates `n` directive with `S`
- Bug: It cannot calculate DST correctly for all time zones
<https://github.com/apache/logging-log4j2/issues/2943>
- `FastDateFormat`
- Copied from Commons Lang in 2015
I recently ran a test comparing the output of these with
`DateTimeFormatter` – the difference is *big*! I can spend months fixing
these issues, but... Do we really need them?
1. No other competitors have such an optimization (both Logback
<https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>
and Tinylog
<https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>
use
`DateTimeFormatter`)
2. Date & time formatting is very fragile (DST is more voodoo than
science)
3. If date & time formatting during logging is found to be the
bottleneck of an application, shouldn't they instead encode it differently,
e.g., encoding epoch numbers?
4. `DateTimeFormatter` performance will only get better over time
Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
Note that the caching optimization we have for the last formatted timestamp
will stay untouched.
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Volkan Yazıcı <vo...@yazi.ci.INVALID>.
Please take into account that date & time formatting constitutes a small
part of the entire encoding need of a layout. That is, if we switch from
`FixedDateFormat` to `DateTimeFormatter`, it does *NOT* mean that the
overall slowdown will be 20 fold. Below are some figures:
*Pattern*
%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%t] %p %-40.40c{1.} %notEmpty{%x }- %m%n
*Using full log events (incl. exception, MDC & NDC)*
FastDateFormat 0.039 ± 0.001 ops/ms
DateTimeFormatter 0.039 ± 0.001 ops/ms
FixedFormat 0.040 ± 0.001 ops/ms
Above figures show that when an exception rendering is involved, date &
time formatting efficiency becomes irrelevant.
*Using lite log events*
DateTimeFormatter 1.888 ± 0.106 ops/ms
FastDateFormat 2.567 ± 0.032 ops/ms
FixedDateFormat 4.793 ± 0.019 ops/ms
Above figures show that for message-only (i.e., lite) log events, there is
a ~37% performance degradation for those who want to encode more than ~1900
events per second per core. Such users can still fallback to legacy
formatters using the `log4j.time.legacyFormatterEnabled` property.
On Mon, Oct 14, 2024 at 10:57 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> Wow. I didn’t expect them to be that much faster. That makes me very
> reluctant to be ok with switching.
>
> Ralph
>
>
>
> > On Oct 14, 2024, at 1:51 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
> wrote:
> >
> > Some results using Java 17:
> >
> >
> > *yyyy-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter 2.844 ± 0.310 ops/ms
> > FastDateFormat 6.847 ± 1.302 ops/ms
> > FixedDateFormat 39.497 ± 6.423 ops/ms
> >
> > *HH:mm:ss.SSS*
> > DateTimeFormatter 3.881 ± 0.447 ops/ms
> > FastDateFormat 10.862 ± 1.915 ops/ms
> > FixedDateFormat 39.618 ± 9.318 ops/ms
> >
> >
> > I used distinct instants within the same day to avoid cache misses in
> > `FixedDateFormat`'s cache on the `yyyy-MM-dd` part, which is updated
> daily
> > at midnight. That is why `FixedDF` figures are more or less the same for
> > both patterns.
> >
> > On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> >> Before I can weigh in one way or another I would need to know what the
> >> performance difference is between DateTimeFormatter and the Fixed and
> Fast
> >> classes, at least where they are working properly. Any such test should
> >> include the caching otherwise you would be comparing worst case
> performance.
> >>
> >> Ralph
> >>
> >>> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
> >> wrote:
> >>>
> >>> *Abstract:* Log4j contains two custom date & time formatting classes.
> All
> >>> our competitors (Logback
> >>> <
> >>
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >>> ,
> >>> Tinylog
> >>> <
> >>
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >>> ,
> >>> etc.) use Java's `DateTimeFormatter` and I know of no user complaints
> >> about
> >>> their performance issues. Shall we switch to `DateTimeFormatter` as the
> >>> default in the next minor release and make the custom implementations
> >>> opt-in?
> >>>
> >>> Log4j ships two custom date & time formatting classes:
> >>>
> >>> - `FixedDateFormat`
> >>> - Supports a hardcoded list of formats
> >>> - Bug: Conflates `n` directive with `S`
> >>> - Bug: It cannot calculate DST correctly for all time zones
> >>> <https://github.com/apache/logging-log4j2/issues/2943>
> >>> - `FastDateFormat`
> >>> - Copied from Commons Lang in 2015
> >>>
> >>> I recently ran a test comparing the output of these with
> >>> `DateTimeFormatter` – the difference is *big*! I can spend months
> fixing
> >>> these issues, but... Do we really need them?
> >>>
> >>> 1. No other competitors have such an optimization (both Logback
> >>> <
> >>
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >>>
> >>> and Tinylog
> >>> <
> >>
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >>>
> >>> use
> >>> `DateTimeFormatter`)
> >>> 2. Date & time formatting is very fragile (DST is more voodoo than
> >>> science)
> >>> 3. If date & time formatting during logging is found to be the
> >>> bottleneck of an application, shouldn't they instead encode it
> >> differently,
> >>> e.g., encoding epoch numbers?
> >>> 4. `DateTimeFormatter` performance will only get better over time
> >>>
> >>> Hence, I propose switching all layouts to use `DateTimeFormatter`,
> unless
> >>> the `log4j.time.legacyFormatterEnabled` property is provided.
> Objections?
> >>>
> >>> Note that the caching optimization we have for the last formatted
> >> timestamp
> >>> will stay untouched.
> >>
> >>
>
>
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Gary Gregory <ga...@gmail.com>.
That's quite a difference! Why would we make things deliberately slower?
Gary
On Mon, Oct 14, 2024, 4:57 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> Wow. I didn’t expect them to be that much faster. That makes me very
> reluctant to be ok with switching.
>
> Ralph
>
>
>
> > On Oct 14, 2024, at 1:51 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
> wrote:
> >
> > Some results using Java 17:
> >
> >
> > *yyyy-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter 2.844 ± 0.310 ops/ms
> > FastDateFormat 6.847 ± 1.302 ops/ms
> > FixedDateFormat 39.497 ± 6.423 ops/ms
> >
> > *HH:mm:ss.SSS*
> > DateTimeFormatter 3.881 ± 0.447 ops/ms
> > FastDateFormat 10.862 ± 1.915 ops/ms
> > FixedDateFormat 39.618 ± 9.318 ops/ms
> >
> >
> > I used distinct instants within the same day to avoid cache misses in
> > `FixedDateFormat`'s cache on the `yyyy-MM-dd` part, which is updated
> daily
> > at midnight. That is why `FixedDF` figures are more or less the same for
> > both patterns.
> >
> > On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> >> Before I can weigh in one way or another I would need to know what the
> >> performance difference is between DateTimeFormatter and the Fixed and
> Fast
> >> classes, at least where they are working properly. Any such test should
> >> include the caching otherwise you would be comparing worst case
> performance.
> >>
> >> Ralph
> >>
> >>> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
> >> wrote:
> >>>
> >>> *Abstract:* Log4j contains two custom date & time formatting classes.
> All
> >>> our competitors (Logback
> >>> <
> >>
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >>> ,
> >>> Tinylog
> >>> <
> >>
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >>> ,
> >>> etc.) use Java's `DateTimeFormatter` and I know of no user complaints
> >> about
> >>> their performance issues. Shall we switch to `DateTimeFormatter` as the
> >>> default in the next minor release and make the custom implementations
> >>> opt-in?
> >>>
> >>> Log4j ships two custom date & time formatting classes:
> >>>
> >>> - `FixedDateFormat`
> >>> - Supports a hardcoded list of formats
> >>> - Bug: Conflates `n` directive with `S`
> >>> - Bug: It cannot calculate DST correctly for all time zones
> >>> <https://github.com/apache/logging-log4j2/issues/2943>
> >>> - `FastDateFormat`
> >>> - Copied from Commons Lang in 2015
> >>>
> >>> I recently ran a test comparing the output of these with
> >>> `DateTimeFormatter` – the difference is *big*! I can spend months
> fixing
> >>> these issues, but... Do we really need them?
> >>>
> >>> 1. No other competitors have such an optimization (both Logback
> >>> <
> >>
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >>>
> >>> and Tinylog
> >>> <
> >>
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >>>
> >>> use
> >>> `DateTimeFormatter`)
> >>> 2. Date & time formatting is very fragile (DST is more voodoo than
> >>> science)
> >>> 3. If date & time formatting during logging is found to be the
> >>> bottleneck of an application, shouldn't they instead encode it
> >> differently,
> >>> e.g., encoding epoch numbers?
> >>> 4. `DateTimeFormatter` performance will only get better over time
> >>>
> >>> Hence, I propose switching all layouts to use `DateTimeFormatter`,
> unless
> >>> the `log4j.time.legacyFormatterEnabled` property is provided.
> Objections?
> >>>
> >>> Note that the caching optimization we have for the last formatted
> >> timestamp
> >>> will stay untouched.
> >>
> >>
>
>
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Ralph Goers <ra...@dslextreme.com>.
Wow. I didn’t expect them to be that much faster. That makes me very reluctant to be ok with switching.
Ralph
> On Oct 14, 2024, at 1:51 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID> wrote:
>
> Some results using Java 17:
>
>
> *yyyy-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter 2.844 ± 0.310 ops/ms
> FastDateFormat 6.847 ± 1.302 ops/ms
> FixedDateFormat 39.497 ± 6.423 ops/ms
>
> *HH:mm:ss.SSS*
> DateTimeFormatter 3.881 ± 0.447 ops/ms
> FastDateFormat 10.862 ± 1.915 ops/ms
> FixedDateFormat 39.618 ± 9.318 ops/ms
>
>
> I used distinct instants within the same day to avoid cache misses in
> `FixedDateFormat`'s cache on the `yyyy-MM-dd` part, which is updated daily
> at midnight. That is why `FixedDF` figures are more or less the same for
> both patterns.
>
> On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> Before I can weigh in one way or another I would need to know what the
>> performance difference is between DateTimeFormatter and the Fixed and Fast
>> classes, at least where they are working properly. Any such test should
>> include the caching otherwise you would be comparing worst case performance.
>>
>> Ralph
>>
>>> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
>> wrote:
>>>
>>> *Abstract:* Log4j contains two custom date & time formatting classes. All
>>> our competitors (Logback
>>> <
>> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
>>> ,
>>> Tinylog
>>> <
>> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
>>> ,
>>> etc.) use Java's `DateTimeFormatter` and I know of no user complaints
>> about
>>> their performance issues. Shall we switch to `DateTimeFormatter` as the
>>> default in the next minor release and make the custom implementations
>>> opt-in?
>>>
>>> Log4j ships two custom date & time formatting classes:
>>>
>>> - `FixedDateFormat`
>>> - Supports a hardcoded list of formats
>>> - Bug: Conflates `n` directive with `S`
>>> - Bug: It cannot calculate DST correctly for all time zones
>>> <https://github.com/apache/logging-log4j2/issues/2943>
>>> - `FastDateFormat`
>>> - Copied from Commons Lang in 2015
>>>
>>> I recently ran a test comparing the output of these with
>>> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
>>> these issues, but... Do we really need them?
>>>
>>> 1. No other competitors have such an optimization (both Logback
>>> <
>> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
>>>
>>> and Tinylog
>>> <
>> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
>>>
>>> use
>>> `DateTimeFormatter`)
>>> 2. Date & time formatting is very fragile (DST is more voodoo than
>>> science)
>>> 3. If date & time formatting during logging is found to be the
>>> bottleneck of an application, shouldn't they instead encode it
>> differently,
>>> e.g., encoding epoch numbers?
>>> 4. `DateTimeFormatter` performance will only get better over time
>>>
>>> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
>>> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
>>>
>>> Note that the caching optimization we have for the last formatted
>> timestamp
>>> will stay untouched.
>>
>>
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Volkan Yazıcı <vo...@yazi.ci.INVALID>.
Some results using Java 17:
*yyyy-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter 2.844 ± 0.310 ops/ms
FastDateFormat 6.847 ± 1.302 ops/ms
FixedDateFormat 39.497 ± 6.423 ops/ms
*HH:mm:ss.SSS*
DateTimeFormatter 3.881 ± 0.447 ops/ms
FastDateFormat 10.862 ± 1.915 ops/ms
FixedDateFormat 39.618 ± 9.318 ops/ms
I used distinct instants within the same day to avoid cache misses in
`FixedDateFormat`'s cache on the `yyyy-MM-dd` part, which is updated daily
at midnight. That is why `FixedDF` figures are more or less the same for
both patterns.
On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> Before I can weigh in one way or another I would need to know what the
> performance difference is between DateTimeFormatter and the Fixed and Fast
> classes, at least where they are working properly. Any such test should
> include the caching otherwise you would be comparing worst case performance.
>
> Ralph
>
> > On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID>
> wrote:
> >
> > *Abstract:* Log4j contains two custom date & time formatting classes. All
> > our competitors (Logback
> > <
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >,
> > Tinylog
> > <
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >,
> > etc.) use Java's `DateTimeFormatter` and I know of no user complaints
> about
> > their performance issues. Shall we switch to `DateTimeFormatter` as the
> > default in the next minor release and make the custom implementations
> > opt-in?
> >
> > Log4j ships two custom date & time formatting classes:
> >
> > - `FixedDateFormat`
> > - Supports a hardcoded list of formats
> > - Bug: Conflates `n` directive with `S`
> > - Bug: It cannot calculate DST correctly for all time zones
> > <https://github.com/apache/logging-log4j2/issues/2943>
> > - `FastDateFormat`
> > - Copied from Commons Lang in 2015
> >
> > I recently ran a test comparing the output of these with
> > `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> > these issues, but... Do we really need them?
> >
> > 1. No other competitors have such an optimization (both Logback
> > <
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >
> > and Tinylog
> > <
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >
> > use
> > `DateTimeFormatter`)
> > 2. Date & time formatting is very fragile (DST is more voodoo than
> > science)
> > 3. If date & time formatting during logging is found to be the
> > bottleneck of an application, shouldn't they instead encode it
> differently,
> > e.g., encoding epoch numbers?
> > 4. `DateTimeFormatter` performance will only get better over time
> >
> > Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> > the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
> >
> > Note that the caching optimization we have for the last formatted
> timestamp
> > will stay untouched.
>
>
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Ralph Goers <ra...@dslextreme.com>.
Before I can weigh in one way or another I would need to know what the performance difference is between DateTimeFormatter and the Fixed and Fast classes, at least where they are working properly. Any such test should include the caching otherwise you would be comparing worst case performance.
Ralph
> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı <vo...@yazi.ci.INVALID> wrote:
>
> *Abstract:* Log4j contains two custom date & time formatting classes. All
> our competitors (Logback
> <https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>,
> Tinylog
> <https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>,
> etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
> their performance issues. Shall we switch to `DateTimeFormatter` as the
> default in the next minor release and make the custom implementations
> opt-in?
>
> Log4j ships two custom date & time formatting classes:
>
> - `FixedDateFormat`
> - Supports a hardcoded list of formats
> - Bug: Conflates `n` directive with `S`
> - Bug: It cannot calculate DST correctly for all time zones
> <https://github.com/apache/logging-log4j2/issues/2943>
> - `FastDateFormat`
> - Copied from Commons Lang in 2015
>
> I recently ran a test comparing the output of these with
> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> these issues, but... Do we really need them?
>
> 1. No other competitors have such an optimization (both Logback
> <https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>
> and Tinylog
> <https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>
> use
> `DateTimeFormatter`)
> 2. Date & time formatting is very fragile (DST is more voodoo than
> science)
> 3. If date & time formatting during logging is found to be the
> bottleneck of an application, shouldn't they instead encode it differently,
> e.g., encoding epoch numbers?
> 4. `DateTimeFormatter` performance will only get better over time
>
> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
>
> Note that the caching optimization we have for the last formatted timestamp
> will stay untouched.
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Volkan Yazıcı <vo...@yazi.ci>.
I've implemented the changes in the way you and Piotr requested:
https://github.com/apache/logging-log4j2/pull/3121 That is, implemented
minute-precision caching and use `DateTimeFormatter` for sub-minute
precision parts (e.g., `ss.SSS`). *Would you mind reviewing the changes,
please?*
(Change implications and performance figures are in the ticket description.)
On Tue, Oct 15, 2024 at 3:14 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> Volkan,
>
> Piotr’s proposal sounds reasonable to me. Does it to you?
>
> Ralph
>
> > On Oct 15, 2024, at 4:36 AM, Piotr P. Karwasz <pi...@gmail.com>
> wrote:
> >
> > Hi Volkan,
> >
> > On Mon, 14 Oct 2024 at 21:30, Volkan Yazıcı <vo...@yazi.ci.invalid>
> wrote:
> >> Hence, I propose switching all layouts to use `DateTimeFormatter`,
> unless
> >> the `log4j.time.legacyFormatterEnabled` property is provided.
> Objections?
> >
> > Can you upgrade `FixedDateFormat` to use DTF as date formatter and use
> > it whenever it is applicable?
> > The bug #2943 can IMHO opinion be fixed by changing the way
> > `FixedDateFormat` caches the date part of the timestamp:
> >
> > * currently the cache consists of the "date string" and up to 25
> > offsets that associate each hour with the offset to UTC. The latter
> > part is very prone to errors and will fail if a country decides to
> > switch to DST at 2:30 am.
> > * you can limit the cache to the "date string" and update the cache at
> > the end of the day or the start of DST, whichever occurs first. Twice
> > a year we will update the cache twice a day (at midnight and 2:00 am).
> >
> > If `FixedDateFormat` is fixed, we can use:
> >
> > * `FixedDateFormat` and `DateTimeFormatter` by default,
> > * `FastDateFormat` if users enable the
> > `log4j2.timeLegacyFormatterEnabled` (please use the post-2.10
> > normalized form).
> >
> > Piotr
> >
> > [1] https://github.com/apache/logging-log4j2/issues/2943
>
>
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Ralph Goers <ra...@dslextreme.com>.
Volkan,
Piotr’s proposal sounds reasonable to me. Does it to you?
Ralph
> On Oct 15, 2024, at 4:36 AM, Piotr P. Karwasz <pi...@gmail.com> wrote:
>
> Hi Volkan,
>
> On Mon, 14 Oct 2024 at 21:30, Volkan Yazıcı <vo...@yazi.ci.invalid> wrote:
>> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
>> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
>
> Can you upgrade `FixedDateFormat` to use DTF as date formatter and use
> it whenever it is applicable?
> The bug #2943 can IMHO opinion be fixed by changing the way
> `FixedDateFormat` caches the date part of the timestamp:
>
> * currently the cache consists of the "date string" and up to 25
> offsets that associate each hour with the offset to UTC. The latter
> part is very prone to errors and will fail if a country decides to
> switch to DST at 2:30 am.
> * you can limit the cache to the "date string" and update the cache at
> the end of the day or the start of DST, whichever occurs first. Twice
> a year we will update the cache twice a day (at midnight and 2:00 am).
>
> If `FixedDateFormat` is fixed, we can use:
>
> * `FixedDateFormat` and `DateTimeFormatter` by default,
> * `FastDateFormat` if users enable the
> `log4j2.timeLegacyFormatterEnabled` (please use the post-2.10
> normalized form).
>
> Piotr
>
> [1] https://github.com/apache/logging-log4j2/issues/2943
Re: [log4j] Using `DateTimeFormatter` by default
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi Volkan,
On Mon, 14 Oct 2024 at 21:30, Volkan Yazıcı <vo...@yazi.ci.invalid> wrote:
> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
Can you upgrade `FixedDateFormat` to use DTF as date formatter and use
it whenever it is applicable?
The bug #2943 can IMHO opinion be fixed by changing the way
`FixedDateFormat` caches the date part of the timestamp:
* currently the cache consists of the "date string" and up to 25
offsets that associate each hour with the offset to UTC. The latter
part is very prone to errors and will fail if a country decides to
switch to DST at 2:30 am.
* you can limit the cache to the "date string" and update the cache at
the end of the day or the start of DST, whichever occurs first. Twice
a year we will update the cache twice a day (at midnight and 2:00 am).
If `FixedDateFormat` is fixed, we can use:
* `FixedDateFormat` and `DateTimeFormatter` by default,
* `FastDateFormat` if users enable the
`log4j2.timeLegacyFormatterEnabled` (please use the post-2.10
normalized form).
Piotr
[1] https://github.com/apache/logging-log4j2/issues/2943
Re: [log4j] Using `DateTimeFormatter` by default
Posted by Matt Sicker <ma...@musigma.org>.
I think it would make sense to switch to the standard Java one. The custom classes mostly predated Java 8, and I’ve seen enough test failures whenever daylight saving time starts or ends to know that they’re buggy.
> On Oct 14, 2024, at 14:30, Volkan Yazıcı <vo...@yazi.ci.INVALID> wrote:
>
> *Abstract:* Log4j contains two custom date & time formatting classes. All
> our competitors (Logback
> <https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>,
> Tinylog
> <https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>,
> etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
> their performance issues. Shall we switch to `DateTimeFormatter` as the
> default in the next minor release and make the custom implementations
> opt-in?
>
> Log4j ships two custom date & time formatting classes:
>
> - `FixedDateFormat`
> - Supports a hardcoded list of formats
> - Bug: Conflates `n` directive with `S`
> - Bug: It cannot calculate DST correctly for all time zones
> <https://github.com/apache/logging-log4j2/issues/2943>
> - `FastDateFormat`
> - Copied from Commons Lang in 2015
>
> I recently ran a test comparing the output of these with
> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> these issues, but... Do we really need them?
>
> 1. No other competitors have such an optimization (both Logback
> <https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java>
> and Tinylog
> <https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java>
> use
> `DateTimeFormatter`)
> 2. Date & time formatting is very fragile (DST is more voodoo than
> science)
> 3. If date & time formatting during logging is found to be the
> bottleneck of an application, shouldn't they instead encode it differently,
> e.g., encoding epoch numbers?
> 4. `DateTimeFormatter` performance will only get better over time
>
> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
>
> Note that the caching optimization we have for the last formatted timestamp
> will stay untouched.