You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Ayush Kumar <ay...@finixpayments.com> on 2020/06/30 18:47:36 UTC
JsonLayout Configuration
Not sure where else to ask this question (stackoverflow maybe?), so
apologies if this isn't the right forum.
We've recently migrated to log4j2 from logback and are using the jsonlayout
for a file appender. These logs are then parsed by scalyr (awesome tool
btw). We've noticed that extended stacktraces in jsonlayout are
ridiculously long. Reaching >20k characters, even with string mode enabled.
This is a problem because there's a max log line size that we can parse
(something around 8k I think).
I was wondering if there was any way to get the default jsonlayout to cut
down the size of stacktraces. PatternLayout has %ex{n}, and I was hoping
for something that wasn't creating my own layout, rewriting every log
message, or manually making pattern layout be a json format with %enc{json}
and a pattern like {"message": "%enc(%m){json}", ...}.
I've created a ticket for it as well
https://issues.apache.org/jira/browse/LOG4J2-2879.
Thanks.
Re: JsonLayout Configuration
Posted by Ayush Kumar <ay...@finixpayments.com>.
Thank you Volkan and Ralph for your ridiculously fast responses. I think we
can put in the, what seems to be pretty minimal, effort to get the logstash
layout plugin working and then migrate to JsonTemplateLayout when that
releases. The ability to order fields, while less critical, is also a nice
boon as well.
On Tue, Jun 30, 2020 at 1:06 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> If you can live with the GelfLayout you could also try that. It will let
> you format the message attribute using a Pattern String equivalent to the
> PatternLayout, so you use its expression to limit the length.
>
> Ralph
>
> > On Jun 30, 2020, at 12:28 PM, Volkan Yazıcı <vo...@gmail.com>
> wrote:
> >
> > Hello Ayush,
> >
> > As of this moment, there doesn't exist such a configuration knob in
> > JsonLayout, AFAIK. Though the future is bright! There is an external
> > plugin, log4j2-logstash-layout[1], which delivers what you're looking
> > for (see its maxStringLength parameter) and more! The good news is, we
> > are in the process of merging this external plugin into the Log4j core
> > with a major overhaul and rebranding, JsonTemplateLayout.
> > Implementation is completed, but it is waiting for a final review on a
> > particular aspect by one of the core members, Ralph Goers.
> >
> > My personal advice would be to go with LogstashLayout and switch to
> > JsonTemplateLayout once it is there.
> >
> > Kind regards.
> >
> > [1] https://github.com/vy/log4j2-logstash-layout
> >
> > On Tue, Jun 30, 2020 at 8:47 PM Ayush Kumar <ay...@finixpayments.com>
> wrote:
> >>
> >> Not sure where else to ask this question (stackoverflow maybe?), so
> >> apologies if this isn't the right forum.
> >>
> >> We've recently migrated to log4j2 from logback and are using the
> jsonlayout
> >> for a file appender. These logs are then parsed by scalyr (awesome tool
> >> btw). We've noticed that extended stacktraces in jsonlayout are
> >> ridiculously long. Reaching >20k characters, even with string mode
> enabled.
> >> This is a problem because there's a max log line size that we can parse
> >> (something around 8k I think).
> >>
> >> I was wondering if there was any way to get the default jsonlayout to
> cut
> >> down the size of stacktraces. PatternLayout has %ex{n}, and I was hoping
> >> for something that wasn't creating my own layout, rewriting every log
> >> message, or manually making pattern layout be a json format with
> %enc{json}
> >> and a pattern like {"message": "%enc(%m){json}", ...}.
> >>
> >> I've created a ticket for it as well
> >> https://issues.apache.org/jira/browse/LOG4J2-2879.
> >>
> >> Thanks.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
Re: JsonLayout Configuration
Posted by Ralph Goers <ra...@dslextreme.com>.
If you can live with the GelfLayout you could also try that. It will let you format the message attribute using a Pattern String equivalent to the PatternLayout, so you use its expression to limit the length.
Ralph
> On Jun 30, 2020, at 12:28 PM, Volkan Yazıcı <vo...@gmail.com> wrote:
>
> Hello Ayush,
>
> As of this moment, there doesn't exist such a configuration knob in
> JsonLayout, AFAIK. Though the future is bright! There is an external
> plugin, log4j2-logstash-layout[1], which delivers what you're looking
> for (see its maxStringLength parameter) and more! The good news is, we
> are in the process of merging this external plugin into the Log4j core
> with a major overhaul and rebranding, JsonTemplateLayout.
> Implementation is completed, but it is waiting for a final review on a
> particular aspect by one of the core members, Ralph Goers.
>
> My personal advice would be to go with LogstashLayout and switch to
> JsonTemplateLayout once it is there.
>
> Kind regards.
>
> [1] https://github.com/vy/log4j2-logstash-layout
>
> On Tue, Jun 30, 2020 at 8:47 PM Ayush Kumar <ay...@finixpayments.com> wrote:
>>
>> Not sure where else to ask this question (stackoverflow maybe?), so
>> apologies if this isn't the right forum.
>>
>> We've recently migrated to log4j2 from logback and are using the jsonlayout
>> for a file appender. These logs are then parsed by scalyr (awesome tool
>> btw). We've noticed that extended stacktraces in jsonlayout are
>> ridiculously long. Reaching >20k characters, even with string mode enabled.
>> This is a problem because there's a max log line size that we can parse
>> (something around 8k I think).
>>
>> I was wondering if there was any way to get the default jsonlayout to cut
>> down the size of stacktraces. PatternLayout has %ex{n}, and I was hoping
>> for something that wasn't creating my own layout, rewriting every log
>> message, or manually making pattern layout be a json format with %enc{json}
>> and a pattern like {"message": "%enc(%m){json}", ...}.
>>
>> I've created a ticket for it as well
>> https://issues.apache.org/jira/browse/LOG4J2-2879.
>>
>> Thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: JsonLayout Configuration
Posted by Volkan Yazıcı <vo...@gmail.com>.
Hello Ayush,
As of this moment, there doesn't exist such a configuration knob in
JsonLayout, AFAIK. Though the future is bright! There is an external
plugin, log4j2-logstash-layout[1], which delivers what you're looking
for (see its maxStringLength parameter) and more! The good news is, we
are in the process of merging this external plugin into the Log4j core
with a major overhaul and rebranding, JsonTemplateLayout.
Implementation is completed, but it is waiting for a final review on a
particular aspect by one of the core members, Ralph Goers.
My personal advice would be to go with LogstashLayout and switch to
JsonTemplateLayout once it is there.
Kind regards.
[1] https://github.com/vy/log4j2-logstash-layout
On Tue, Jun 30, 2020 at 8:47 PM Ayush Kumar <ay...@finixpayments.com> wrote:
>
> Not sure where else to ask this question (stackoverflow maybe?), so
> apologies if this isn't the right forum.
>
> We've recently migrated to log4j2 from logback and are using the jsonlayout
> for a file appender. These logs are then parsed by scalyr (awesome tool
> btw). We've noticed that extended stacktraces in jsonlayout are
> ridiculously long. Reaching >20k characters, even with string mode enabled.
> This is a problem because there's a max log line size that we can parse
> (something around 8k I think).
>
> I was wondering if there was any way to get the default jsonlayout to cut
> down the size of stacktraces. PatternLayout has %ex{n}, and I was hoping
> for something that wasn't creating my own layout, rewriting every log
> message, or manually making pattern layout be a json format with %enc{json}
> and a pattern like {"message": "%enc(%m){json}", ...}.
>
> I've created a ticket for it as well
> https://issues.apache.org/jira/browse/LOG4J2-2879.
>
> Thanks.
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org