[jira] [Comment Edited] (LOG4J2-2936) Add message parameters in JsonLayout

["Diego Pettisani (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LOG4J2-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201964#comment-17201964 ] 

Diego Pettisani edited comment on LOG4J2-2936 at 9/25/20, 7:26 AM:
-------------------------------------------------------------------

Thanks for info [~rgoers], I gave a look at that ([here|https://vlkan.com/log4j/json-template-layout.html]) and I did not find a _resolver_ able to isolate the message parameters in some other (dedicated) Json section.

I try to explain the reasons for implementing this kind of feature.

Most of all programmers use this kind of APIs (through Slf4j or Log4j2):

{code:java}
// case 1
LOGGER.info("Found purchase order {} in {} milliseconds", po, duration);
// case 2
LOGGER.warn("Product {} out of stock: requested {} units, but only {} units are stocked", sku, request, stocked);
// case 3
LOGGER.error("HTTP request sent to shipper service failed with code {}", httpErrorCode);
{code}
 
If I would like to make some kind of dashboard in Kibana like:

* *case 1*: The average response time for founding a purchase order during the time.
* *case 2*: The most frequent products out of stock.
* *case 3*: The most frequent HTTP error codes when I try to call the shipper service.

I should need to do some special extraction parameters *for each case* (the cases could be hundreds) in the Fluentd/Logstash tools by playing with pattern/regex/grok for having a custom field to read in Kibana and build the dashboards.

With the proposal described in this Jira issue, there is no need to create any special extraction rule _for each case_ in order to build the desired dashboard in Kibana, just:

* *case 1*: Get the field {{params.p1}} for the messages having the format {{"Found purchase order * in * milliseconds"}}
* *case 2*: Get the field {{params.p0}} for the messages having the format {{"Product {} out of stock: requested {} units, but only {} units are stocked"}}
* *case 3*: Get the field {{params.p0}} for the messages having the format {{"HTTP request sent to shipper service failed with code {}"}}

And implementing a new configuration option for *JsonLayout* or a new resolver for *JsonTemplateLayout* should be great for obtaining that.



was (Author: diepet):
Thanks for info [~rgoers], I gave a look at that ([here|https://vlkan.com/log4j/json-template-layout.html]) and I did not find a _resolver_ able to isolate the message parameters in some other (dedicated) Json section.

I try to explain the reasons for implementing this kind of feature.

Most of all programmers use this kind of APIs (through Slf4j or Log4j2):

{code:java}
// case 1
LOGGER.info("Found purchase order {} in {} milliseconds", po, duration);
// case 2
LOGGER.warn("Product {} out of stock: requested {} units, but only {} units are stocked", sku, request, stocked);
// case 3
LOGGER.error("HTTP request sent to shipper service failed with code {}", httpErrorCode);
{code}
 
If I would like to make some kind of dashboard in Kibana like:

* *case 1*: The average response time for founding a purchase order during the time.
* *case 2*: The most frequent products out of stock.
* *case 3*: The most frequent HTTP error codes when I try to call the shipper service.

I should need to do some special extraction parameters *for each case* (the cases could be hundreds) in the Fluentd/Logstash tools by playing with pattern/regex/grok for having a custom field to get in Kibana and build the dashboards.

With the proposal described in this Jira issue, there is no need to create any special extraction rule _for each case_ in order to build the desired dashboard in Kibana, just:

* *case 1*: Get the field {{params.p1}} for the messages having the format {{"Found purchase order * in * milliseconds"}}
* *case 2*: Get the field {{params.p0}} for the messages having the format {{"Product {} out of stock: requested {} units, but only {} units are stocked"}}
* *case 3*: Get the field {{params.p0}} for the messages having the format {{"HTTP request sent to shipper service failed with code {}"}}

And implementing a new configuration option for *JsonLayout* or a new resolver for *JsonTemplateLayout* should be great for obtaining that.


> Add message parameters in JsonLayout
> ------------------------------------
>
>                 Key: LOG4J2-2936
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-2936
>             Project: Log4j 2
>          Issue Type: New Feature
>          Components: Layouts
>    Affects Versions: 2.13.3
>            Reporter: Diego Pettisani
>            Priority: Major
>              Labels: JsonLayout
>
> By using the JsonLayout the following call:
>   
> {code:java}
> String invoiceNo = "A12345";
> long duration = 3999l;
> LOGGER.info("Invoice {} elaborated in {} milliseconds", invoceNo, duration);{code}
> will produce something like this:
>   
> {code:json}
> {   // ...
>    "message" : "Invoice A12345 elaborated in 3999 milliseconds"
>    // ...
> }{code}
> And for extracting the {{invoceNo}} and {{duration}} values from a log collector tool (e.g. Elastic) it is needed to play with patterns and regex hard to maintain in the log shipper (e.g. Fluentd).
> It could be very helpful to add a new boolean configuration property in the JsonLayout element in order to enable the printing of the message parameters as Json fields.
> After enabled this new property (e.g. {{parameters=true}}), the output JSON should be something like this:
> {code:json}
> {   // ...
>    "message" : "Invoice A12345 elaborated in 3999 milliseconds"
>    "params" : {
>           "p0" : "A12345"
>           "p1" : "3999"
>     }
>    // ...
> }{code}
> In this way, it will be possible to extract that values very easily from the log collector tool, without implementing ad-hoc pattern/regex in the log shipper.
> This feature could be especially useful for legacy code hard to change that uses SLF4J APIs for logging: extracting values from that kind of messages will be a piece of cake.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)