Re: GC Free Layouts

[Volkan Yazıcı <vo...@gmail.com>]

I have pushed the garbage-free version of log4j2-logstash-layout to the
branch: https://github.com/vy/log4j2-logstash-layout/tree/json-generator
There you can also see the JMH results in a table.


On Tue, Oct 23, 2018 at 2:28 AM Remko Popma <re...@gmail.com> wrote:

> Looks very nice!
>
> The non garbage-free implementation is in the toSerializable(LogEvent)
> method. As you point out, this often ends in a call to
> StringBuilder.toString().
>
> The garbage-free implementation is in the encode(LogEvent,
> ByteBufferDestination) method. This reuses a ThreadLocal StringBuilder.
>
> Be aware of edge cases like occasional very large messages that result in
> the reusable StringBuilder taking up a lot of memory and never releasing
> it. (We trim if some configurable max size is exceeded.)
> You get some of this for free by extending AbstractStringLayout.
>
> I took a brief look at the BENCHMARK.txt file but didn’t have time to look
> in detail. A graph comparison would be very cool!
>
> Good luck, keep us posted!
>
> (Shameless plug) Every java main() method deserves http://picocli.info
>
> > On Oct 23, 2018, at 6:40, Volkan Yazıcı <vo...@gmail.com> wrote:
> >
> > Hello,
> >
> > I am working on the next release of log4j2-logstash-layout
> > <https://github.com/vy/log4j2-logstash-layout/tree/json-generator>,
> which
> > is an alternative to the default JsonLayout and allows custom JSON
> schemas.
> > There I achieved to get up to 5x speed up compared to JsonLayout[1] and I
> > want to stretch this further by reducing the GC load. Though it is really
> > difficult given Layout interface (practically) requires an output of type
> > String. I examined the internals of other gc-free layouts, e.g., Pattern,
> > Gelf, etc. Though each ends up making a call to StringBuilder#toString(),
> > which is not gc-free, to the best of my knowledge. Would anyone mind
> > shedding some light onto the issue, please?
> >
> > Best.
> >
> > [1] Please feel free to check JMH benchmarks
> > <
> https://github.com/vy/log4j2-logstash-layout/blob/json-generator/layout/src/test/perf/com/vlkan/log4j2/logstash/layout/LogstashLayoutBenchmarkState.java
> >
> > and their results
> > <
> https://github.com/vy/log4j2-logstash-layout/blob/json-generator/BENCHMARK.txt
> >.
> > Maybe there I am misconfiguring JsonLayout to cause it under-perform.
>