Removing GelfLayout

[Volkan Yazıcı <vo...@gmail.com>]

As the title suggests, in accordance with the aim of reducing the
maintenance burden, I want to propose removal of the GelfLayout from
master, not release-2.x! Currently, JsonTemplateLayout already ships a
template producing exactly the same output as GelfLayout; see
GelfLayout.json template and GelfLayoutTest class. The only missing piece
of the puzzle is compression support, which I am gonna address in
LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I have
a go for this operation? What do you think?

On the other hand, I wish we would be able to implement compression as some
sort of filter applicable to layouts. Is such a thing possible?

Re: Removing GelfLayout

[Gary Gregory <ga...@gmail.com>]

FYI, at my day job, one major hurdle to updating one of our projects to
Log4j 2 are configuration files. This task is in progress. The less pain,
the better. In our case, our tooling is responsible for programmatically
updating these files. I do NOT want to have to sell this task AGAIN for
updating from Log4j 2 to 3.

Gary

On Thu, Feb 18, 2021, 11:49 Ralph Goers <ra...@dslextreme.com> wrote:

> Sorry, the one thing I have learned in the slow migration we have seen of
> people moving from Log4j 1 to Log4j 2 is that changing logging
> configuration files is difficult for many users while changing a jar
> version is not. We have had many customers complain that upgrading was very
> hard because they had hundreds of log4j.xml files to convert. Granted, this
> may be a somewhat simpler task but it is a task I am hesitant to inflict on
> users for no particularly good reason. After all, we could simply keep the
> old layouts around to ease their burden.
>
> Ralph
>
> > On Feb 18, 2021, at 8:59 AM, Volkan Yazıcı <vo...@gmail.com>
> wrote:
> >
> > I don't have the benchmark report anymore in
> json-template-layout-adoc.vm,
> > but the benchmark is still there: JsonTemplateLayoutBenchmark. The last
> > time I have benchmarked it, JsonTemplateLayout was on par with GelfLayout
> > for "lite" (i.e., no stack trace, mdc, ndc) log events and significantly
> > faster for "full" log events – JsonTemplateLayout is almost garbage-free
> > while rendering stack traces; whereas ThrowablePatternConverter, which is
> > internally used by GelfLayout through PatternLayout, performs extra
> > allocations each time. Yes, GelfLayout manually constructs JSON, but
> > JsonTemplateLayout too. It generates functions for rendering after
> parsing
> > the template. It doesn't interpret the template each time. But anyway...
> I
> > can share a benchmark report either today or tomorrow for a more factual
> > argument.
> >
> > I see and share your concerns regarding backward compatibility. 3.0.0
> > release is a good opportunity to remove functionality either deprecated
> or
> > superseded by alternatives without getting concerned too much about the
> > backward compatibility. Yes, indeed people might end up touching their
> > logging configuration files during migration, but that is not the end of
> > the world. Log4j already has a stunning track record for backward
> > compatibility and a major release is one of those rare chances we as
> > developers get to lose some extra maintenance burden. It won't be the
> first
> > time we have removed a functionality from Log4j without an alternative,
> > e.g., SerializedLayout. Also how long do we expect to maintain this
> > backward compatibility, if not we can opt out in a major release? In
> > conclusion, I think we can remove these modules and provide a migration
> > guide in the manual. I kindly ask you to reconsider your thoughts on
> this.
> > I by no means want to ruin the rock solid image of Log4j built by you and
> > others, which I highly respect.
> >
> > On Thu, Feb 18, 2021 at 4:29 PM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> >> You would have to prove to me that GelfLayout is slower than JsonLayout.
> >> That would be hard for me to believe since the GelfLayout is manually
> >> constructing the JSON.
> >>
> >> Also, You cannot remove any of the GelfLayout or any of the JsonLayouts.
> >> You will need to replace them with logic that invokes the
> >> JsonTemplateLayout in a backwards compatible way as users should not
> have
> >> to change their configurations.
> >>
> >> Ralph
> >>
> >>> On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com>
> >> wrote:
> >>>
> >>> As the title suggests, in accordance with the aim of reducing the
> >>> maintenance burden, I want to propose removal of the GelfLayout from
> >>> master, not release-2.x! Currently, JsonTemplateLayout already ships a
> >>> template producing exactly the same output as GelfLayout; see
> >>> GelfLayout.json template and GelfLayoutTest class. The only missing
> piece
> >>> of the puzzle is compression support, which I am gonna address in
> >>> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
> >> have
> >>> a go for this operation? What do you think?
> >>>
> >>> On the other hand, I wish we would be able to implement compression as
> >> some
> >>> sort of filter applicable to layouts. Is such a thing possible?
> >>
> >>
> >>
>
>
>

Re: Removing GelfLayout

[Matt Sicker <bo...@gmail.com>]

I believe we also have a backlog item about creating a sort of "fast
by default" or similar mode that allows users to opt-in to a sort of
"updated best practices" options for defaults. This would allow
configuration files to continue working while allowing new config
files to take advantage of simplified defaults.

On Thu, 18 Feb 2021 at 11:10, Carter Kozak <ck...@ckozak.net> wrote:
>
> I agree with Ralph's comments here. As a developer shipping an application, it's not clear how all my consumers have configured logging. I can upgrade jars, but I wouldn't want to make a change that breaks some percentage of my customers logging configurations. The cost of continuing to provide the json layout is relatively low, however I agree that we should deprecate it in favor of the much better json template layout and suggest users migrate if they report bugs on the json layout.
>
> On Thu, Feb 18, 2021, at 11:49, Ralph Goers wrote:
> > Sorry, the one thing I have learned in the slow migration we have seen of people moving from Log4j 1 to Log4j 2 is that changing logging configuration files is difficult for many users while changing a jar version is not. We have had many customers complain that upgrading was very hard because they had hundreds of log4j.xml files to convert. Granted, this may be a somewhat simpler task but it is a task I am hesitant to inflict on users for no particularly good reason. After all, we could simply keep the old layouts around to ease their burden.
> >
> > Ralph
> >
> > > On Feb 18, 2021, at 8:59 AM, Volkan Yazıcı <vo...@gmail.com> wrote:
> > >
> > > I don't have the benchmark report anymore in json-template-layout-adoc.vm,
> > > but the benchmark is still there: JsonTemplateLayoutBenchmark. The last
> > > time I have benchmarked it, JsonTemplateLayout was on par with GelfLayout
> > > for "lite" (i.e., no stack trace, mdc, ndc) log events and significantly
> > > faster for "full" log events – JsonTemplateLayout is almost garbage-free
> > > while rendering stack traces; whereas ThrowablePatternConverter, which is
> > > internally used by GelfLayout through PatternLayout, performs extra
> > > allocations each time. Yes, GelfLayout manually constructs JSON, but
> > > JsonTemplateLayout too. It generates functions for rendering after parsing
> > > the template. It doesn't interpret the template each time. But anyway... I
> > > can share a benchmark report either today or tomorrow for a more factual
> > > argument.
> > >
> > > I see and share your concerns regarding backward compatibility. 3.0.0
> > > release is a good opportunity to remove functionality either deprecated or
> > > superseded by alternatives without getting concerned too much about the
> > > backward compatibility. Yes, indeed people might end up touching their
> > > logging configuration files during migration, but that is not the end of
> > > the world. Log4j already has a stunning track record for backward
> > > compatibility and a major release is one of those rare chances we as
> > > developers get to lose some extra maintenance burden. It won't be the first
> > > time we have removed a functionality from Log4j without an alternative,
> > > e.g., SerializedLayout. Also how long do we expect to maintain this
> > > backward compatibility, if not we can opt out in a major release? In
> > > conclusion, I think we can remove these modules and provide a migration
> > > guide in the manual. I kindly ask you to reconsider your thoughts on this.
> > > I by no means want to ruin the rock solid image of Log4j built by you and
> > > others, which I highly respect.
> > >
> > > On Thu, Feb 18, 2021 at 4:29 PM Ralph Goers <ra...@dslextreme.com>
> > > wrote:
> > >
> > >> You would have to prove to me that GelfLayout is slower than JsonLayout.
> > >> That would be hard for me to believe since the GelfLayout is manually
> > >> constructing the JSON.
> > >>
> > >> Also, You cannot remove any of the GelfLayout or any of the JsonLayouts.
> > >> You will need to replace them with logic that invokes the
> > >> JsonTemplateLayout in a backwards compatible way as users should not have
> > >> to change their configurations.
> > >>
> > >> Ralph
> > >>
> > >>> On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com>
> > >> wrote:
> > >>>
> > >>> As the title suggests, in accordance with the aim of reducing the
> > >>> maintenance burden, I want to propose removal of the GelfLayout from
> > >>> master, not release-2.x! Currently, JsonTemplateLayout already ships a
> > >>> template producing exactly the same output as GelfLayout; see
> > >>> GelfLayout.json template and GelfLayoutTest class. The only missing piece
> > >>> of the puzzle is compression support, which I am gonna address in
> > >>> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
> > >> have
> > >>> a go for this operation? What do you think?
> > >>>
> > >>> On the other hand, I wish we would be able to implement compression as
> > >> some
> > >>> sort of filter applicable to layouts. Is such a thing possible?
> > >>
> > >>
> > >>
> >
> >
> >

Re: Removing GelfLayout

[Carter Kozak <ck...@ckozak.net>]

I agree with Ralph's comments here. As a developer shipping an application, it's not clear how all my consumers have configured logging. I can upgrade jars, but I wouldn't want to make a change that breaks some percentage of my customers logging configurations. The cost of continuing to provide the json layout is relatively low, however I agree that we should deprecate it in favor of the much better json template layout and suggest users migrate if they report bugs on the json layout.

On Thu, Feb 18, 2021, at 11:49, Ralph Goers wrote:
> Sorry, the one thing I have learned in the slow migration we have seen of people moving from Log4j 1 to Log4j 2 is that changing logging configuration files is difficult for many users while changing a jar version is not. We have had many customers complain that upgrading was very hard because they had hundreds of log4j.xml files to convert. Granted, this may be a somewhat simpler task but it is a task I am hesitant to inflict on users for no particularly good reason. After all, we could simply keep the old layouts around to ease their burden.
> 
> Ralph
> 
> > On Feb 18, 2021, at 8:59 AM, Volkan Yazıcı <vo...@gmail.com> wrote:
> > 
> > I don't have the benchmark report anymore in json-template-layout-adoc.vm,
> > but the benchmark is still there: JsonTemplateLayoutBenchmark. The last
> > time I have benchmarked it, JsonTemplateLayout was on par with GelfLayout
> > for "lite" (i.e., no stack trace, mdc, ndc) log events and significantly
> > faster for "full" log events – JsonTemplateLayout is almost garbage-free
> > while rendering stack traces; whereas ThrowablePatternConverter, which is
> > internally used by GelfLayout through PatternLayout, performs extra
> > allocations each time. Yes, GelfLayout manually constructs JSON, but
> > JsonTemplateLayout too. It generates functions for rendering after parsing
> > the template. It doesn't interpret the template each time. But anyway... I
> > can share a benchmark report either today or tomorrow for a more factual
> > argument.
> > 
> > I see and share your concerns regarding backward compatibility. 3.0.0
> > release is a good opportunity to remove functionality either deprecated or
> > superseded by alternatives without getting concerned too much about the
> > backward compatibility. Yes, indeed people might end up touching their
> > logging configuration files during migration, but that is not the end of
> > the world. Log4j already has a stunning track record for backward
> > compatibility and a major release is one of those rare chances we as
> > developers get to lose some extra maintenance burden. It won't be the first
> > time we have removed a functionality from Log4j without an alternative,
> > e.g., SerializedLayout. Also how long do we expect to maintain this
> > backward compatibility, if not we can opt out in a major release? In
> > conclusion, I think we can remove these modules and provide a migration
> > guide in the manual. I kindly ask you to reconsider your thoughts on this.
> > I by no means want to ruin the rock solid image of Log4j built by you and
> > others, which I highly respect.
> > 
> > On Thu, Feb 18, 2021 at 4:29 PM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> > 
> >> You would have to prove to me that GelfLayout is slower than JsonLayout.
> >> That would be hard for me to believe since the GelfLayout is manually
> >> constructing the JSON.
> >> 
> >> Also, You cannot remove any of the GelfLayout or any of the JsonLayouts.
> >> You will need to replace them with logic that invokes the
> >> JsonTemplateLayout in a backwards compatible way as users should not have
> >> to change their configurations.
> >> 
> >> Ralph
> >> 
> >>> On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com>
> >> wrote:
> >>> 
> >>> As the title suggests, in accordance with the aim of reducing the
> >>> maintenance burden, I want to propose removal of the GelfLayout from
> >>> master, not release-2.x! Currently, JsonTemplateLayout already ships a
> >>> template producing exactly the same output as GelfLayout; see
> >>> GelfLayout.json template and GelfLayoutTest class. The only missing piece
> >>> of the puzzle is compression support, which I am gonna address in
> >>> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
> >> have
> >>> a go for this operation? What do you think?
> >>> 
> >>> On the other hand, I wish we would be able to implement compression as
> >> some
> >>> sort of filter applicable to layouts. Is such a thing possible?
> >> 
> >> 
> >> 
> 
> 
> 

Re: Removing GelfLayout

[Ralph Goers <ra...@dslextreme.com>]

Sorry, the one thing I have learned in the slow migration we have seen of people moving from Log4j 1 to Log4j 2 is that changing logging configuration files is difficult for many users while changing a jar version is not. We have had many customers complain that upgrading was very hard because they had hundreds of log4j.xml files to convert. Granted, this may be a somewhat simpler task but it is a task I am hesitant to inflict on users for no particularly good reason. After all, we could simply keep the old layouts around to ease their burden.

Ralph

> On Feb 18, 2021, at 8:59 AM, Volkan Yazıcı <vo...@gmail.com> wrote:
> 
> I don't have the benchmark report anymore in json-template-layout-adoc.vm,
> but the benchmark is still there: JsonTemplateLayoutBenchmark. The last
> time I have benchmarked it, JsonTemplateLayout was on par with GelfLayout
> for "lite" (i.e., no stack trace, mdc, ndc) log events and significantly
> faster for "full" log events – JsonTemplateLayout is almost garbage-free
> while rendering stack traces; whereas ThrowablePatternConverter, which is
> internally used by GelfLayout through PatternLayout, performs extra
> allocations each time. Yes, GelfLayout manually constructs JSON, but
> JsonTemplateLayout too. It generates functions for rendering after parsing
> the template. It doesn't interpret the template each time. But anyway... I
> can share a benchmark report either today or tomorrow for a more factual
> argument.
> 
> I see and share your concerns regarding backward compatibility. 3.0.0
> release is a good opportunity to remove functionality either deprecated or
> superseded by alternatives without getting concerned too much about the
> backward compatibility. Yes, indeed people might end up touching their
> logging configuration files during migration, but that is not the end of
> the world. Log4j already has a stunning track record for backward
> compatibility and a major release is one of those rare chances we as
> developers get to lose some extra maintenance burden. It won't be the first
> time we have removed a functionality from Log4j without an alternative,
> e.g., SerializedLayout. Also how long do we expect to maintain this
> backward compatibility, if not we can opt out in a major release? In
> conclusion, I think we can remove these modules and provide a migration
> guide in the manual. I kindly ask you to reconsider your thoughts on this.
> I by no means want to ruin the rock solid image of Log4j built by you and
> others, which I highly respect.
> 
> On Thu, Feb 18, 2021 at 4:29 PM Ralph Goers <ra...@dslextreme.com>
> wrote:
> 
>> You would have to prove to me that GelfLayout is slower than JsonLayout.
>> That would be hard for me to believe since the GelfLayout is manually
>> constructing the JSON.
>> 
>> Also, You cannot remove any of the GelfLayout or any of the JsonLayouts.
>> You will need to replace them with logic that invokes the
>> JsonTemplateLayout in a backwards compatible way as users should not have
>> to change their configurations.
>> 
>> Ralph
>> 
>>> On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com>
>> wrote:
>>> 
>>> As the title suggests, in accordance with the aim of reducing the
>>> maintenance burden, I want to propose removal of the GelfLayout from
>>> master, not release-2.x! Currently, JsonTemplateLayout already ships a
>>> template producing exactly the same output as GelfLayout; see
>>> GelfLayout.json template and GelfLayoutTest class. The only missing piece
>>> of the puzzle is compression support, which I am gonna address in
>>> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
>> have
>>> a go for this operation? What do you think?
>>> 
>>> On the other hand, I wish we would be able to implement compression as
>> some
>>> sort of filter applicable to layouts. Is such a thing possible?
>> 
>> 
>> 

Re: Removing GelfLayout

[Volkan Yazıcı <vo...@gmail.com>]

I don't have the benchmark report anymore in json-template-layout-adoc.vm,
but the benchmark is still there: JsonTemplateLayoutBenchmark. The last
time I have benchmarked it, JsonTemplateLayout was on par with GelfLayout
for "lite" (i.e., no stack trace, mdc, ndc) log events and significantly
faster for "full" log events – JsonTemplateLayout is almost garbage-free
while rendering stack traces; whereas ThrowablePatternConverter, which is
internally used by GelfLayout through PatternLayout, performs extra
allocations each time. Yes, GelfLayout manually constructs JSON, but
JsonTemplateLayout too. It generates functions for rendering after parsing
the template. It doesn't interpret the template each time. But anyway... I
can share a benchmark report either today or tomorrow for a more factual
argument.

I see and share your concerns regarding backward compatibility. 3.0.0
release is a good opportunity to remove functionality either deprecated or
superseded by alternatives without getting concerned too much about the
backward compatibility. Yes, indeed people might end up touching their
logging configuration files during migration, but that is not the end of
the world. Log4j already has a stunning track record for backward
compatibility and a major release is one of those rare chances we as
developers get to lose some extra maintenance burden. It won't be the first
time we have removed a functionality from Log4j without an alternative,
e.g., SerializedLayout. Also how long do we expect to maintain this
backward compatibility, if not we can opt out in a major release? In
conclusion, I think we can remove these modules and provide a migration
guide in the manual. I kindly ask you to reconsider your thoughts on this.
I by no means want to ruin the rock solid image of Log4j built by you and
others, which I highly respect.

On Thu, Feb 18, 2021 at 4:29 PM Ralph Goers <ra...@dslextreme.com>
wrote:

> You would have to prove to me that GelfLayout is slower than JsonLayout.
> That would be hard for me to believe since the GelfLayout is manually
> constructing the JSON.
>
> Also, You cannot remove any of the GelfLayout or any of the JsonLayouts.
> You will need to replace them with logic that invokes the
> JsonTemplateLayout in a backwards compatible way as users should not have
> to change their configurations.
>
> Ralph
>
> > On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com>
> wrote:
> >
> > As the title suggests, in accordance with the aim of reducing the
> > maintenance burden, I want to propose removal of the GelfLayout from
> > master, not release-2.x! Currently, JsonTemplateLayout already ships a
> > template producing exactly the same output as GelfLayout; see
> > GelfLayout.json template and GelfLayoutTest class. The only missing piece
> > of the puzzle is compression support, which I am gonna address in
> > LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
> have
> > a go for this operation? What do you think?
> >
> > On the other hand, I wish we would be able to implement compression as
> some
> > sort of filter applicable to layouts. Is such a thing possible?
>
>
>

Re: Removing GelfLayout

[Ralph Goers <ra...@dslextreme.com>]

You would have to prove to me that GelfLayout is slower than JsonLayout. That would be hard for me to believe since the GelfLayout is manually constructing the JSON. 

Also, You cannot remove any of the GelfLayout or any of the JsonLayouts. You will need to replace them with logic that invokes the JsonTemplateLayout in a backwards compatible way as users should not have to change their configurations.

Ralph

> On Feb 18, 2021, at 8:10 AM, Volkan Yazıcı <vo...@gmail.com> wrote:
> 
> As the title suggests, in accordance with the aim of reducing the
> maintenance burden, I want to propose removal of the GelfLayout from
> master, not release-2.x! Currently, JsonTemplateLayout already ships a
> template producing exactly the same output as GelfLayout; see
> GelfLayout.json template and GelfLayoutTest class. The only missing piece
> of the puzzle is compression support, which I am gonna address in
> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I have
> a go for this operation? What do you think?
> 
> On the other hand, I wish we would be able to implement compression as some
> sort of filter applicable to layouts. Is such a thing possible?

Re: Removing GelfLayout

[Ralph Goers <ra...@dslextreme.com>]

Thanks for your understanding. As I said, I am not against routing them all to JsonTemplateLayout but I can understand that that might be difficult. 

We do need to make sure that the Layout documentation on the web site makes it clear that JsonTemplateLayout is the preferred solution.

Ralph

> On Feb 23, 2021, at 2:03 PM, Volkan Yazıcı <vo...@gmail.com> wrote:
> 
> I am dropping the case due to objections from other committers.
> I have closed the GitHub PR and JIRA ticket for the deprecation of
> log4j-layout-jackson* modules.
> 
> On Thu, Feb 18, 2021 at 4:10 PM Volkan Yazıcı <vo...@gmail.com>
> wrote:
> 
>> As the title suggests, in accordance with the aim of reducing the
>> maintenance burden, I want to propose removal of the GelfLayout from
>> master, not release-2.x! Currently, JsonTemplateLayout already ships a
>> template producing exactly the same output as GelfLayout; see
>> GelfLayout.json template and GelfLayoutTest class. The only missing piece
>> of the puzzle is compression support, which I am gonna address in
>> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
>> have a go for this operation? What do you think?
>> 
>> On the other hand, I wish we would be able to implement compression as
>> some sort of filter applicable to layouts. Is such a thing possible?
>> 

Re: Removing GelfLayout

[Volkan Yazıcı <vo...@gmail.com>]

I am dropping the case due to objections from other committers.
I have closed the GitHub PR and JIRA ticket for the deprecation of
log4j-layout-jackson* modules.

On Thu, Feb 18, 2021 at 4:10 PM Volkan Yazıcı <vo...@gmail.com>
wrote:

> As the title suggests, in accordance with the aim of reducing the
> maintenance burden, I want to propose removal of the GelfLayout from
> master, not release-2.x! Currently, JsonTemplateLayout already ships a
> template producing exactly the same output as GelfLayout; see
> GelfLayout.json template and GelfLayoutTest class. The only missing piece
> of the puzzle is compression support, which I am gonna address in
> LOG4J2-3023 <https://issues.apache.org/jira/browse/LOG4J2-3023>. Do I
> have a go for this operation? What do you think?
>
> On the other hand, I wish we would be able to implement compression as
> some sort of filter applicable to layouts. Is such a thing possible?
>