You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@logging.apache.org by "ngocnhan-tran1996 (via GitHub)" <gi...@apache.org> on 2024/06/14 15:13:10 UTC
[I] Log4j2 TemplateLayout will support Property Substitution or Lookups? (logging-log4j2)
ngocnhan-tran1996 opened a new issue, #2666:
URL: https://github.com/apache/logging-log4j2/issues/2666
I have an application use log4j2 template layout with below config
File `EcsLayout.json`
```json
{
"message1": {
"$resolver": "pattern",
"pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}}"
},
"message2": {
"$resolver": "pattern",
"pattern": "${env:TZ:-${TIMEZONE}}"
}
}
```
File `log4j2.xml`
```xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration name="Mask Library Test" xmlns:xi="http://www.w3.org/2001/XInclude">
<Properties>
<Property name="TIMEZONE">%d{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
disableAnsi="false"
pattern="%style{${TIMEZONE}}{black} %highlight{%-5level}[%style{%t}{bright,blue}] %style{%c{1.}}{bright,yellow}: %m%n%throwable" />
</Console>
<RollingFile name="RollingFile"
filePattern="logs/app-%d{yyyy-MM-dd-HH-mm}.log">
<JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json" />
<TimeBasedTriggeringPolicy />
</RollingFile>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console" />
<AppenderRef ref="RollingFile" />
</Root>
</Loggers>
</Configuration>
```
And file log
```text
{"message1":"${env:TZ:-2024-06-14 15:00:33.561}","message2":"%d{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}"}
```
**Expected**
**message1**
If `env:TZ` do not exist, it will be replace by `{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}`
or
**message2**
log will be recorded date time instead `"%d{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}"`
I have a small demo, just run and go to file `logs/app-{yyyy-MM-dd-HH-mm}.log` and you will see result. Please tell me if I miss something.
https://github.com/ngocnhan-tran1996/log4j2-demo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `PatternResolver` does not support substitution (logging-log4j2)
Posted by "vy (via GitHub)" <gi...@apache.org>.
vy commented on issue #2666:
URL: https://github.com/apache/logging-log4j2/issues/2666#issuecomment-2172733263
@ngocnhan-tran1996, thanks for the report, I've submitted #2667 fixing the issue. I would really appreciate it if you can
1. Checkout the `fix/2.x/JTL-prop-subs` branch
2. Quickly build it using `./mvnw install -DskipTests` – see [`BUILDING.adoc`](/apache/logging-log4j2/blob/2.x/BUILDING.adoc) for details
3. And test the installed `2.24.0-SNAPSHOT` artifacts in your project
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `PatternResolver` does not support substitution (logging-log4j2)
Posted by "vy (via GitHub)" <gi...@apache.org>.
vy commented on issue #2666:
URL: https://github.com/apache/logging-log4j2/issues/2666#issuecomment-2176884248
@ngocnhan-tran1996, I collaborated on this issue with @ppkarwasz, and I decided to not update JTL (JSON Template Layout) to perform property substitution on resolver configuration objects. Yet the issue you reported about the trailing `}` is indeed a `StrSubstitutor` bug, which I will address in another ticket.
### JTL and property substitution
JTL supports property substitution on string members of a JSON object, granted they are not part of a resolver configuration. Let me make this clear with an example:
```json
{
"x": "${bar}",
"y": {
"$resolver": "...",
"z": "${bar}"
}
}
```
`${bar}` for `x` will be substituted at *configuration-time* (i.e., while compiling JTL templates), whereas `${bar}` of `z` will not be substituted, since it is part of a *template configuration object*. In #2667, I tried various approaches to improve the situation, but it always caused other issues.
It is important to understand property substitution gets performed at three levels in your example:
* For literals in `log4j2.xml` (at configuration-time)
* For literals in a JTL template (at configuration-time)
* By Pattern Layout wired by `pattern` resolver (at runtime)
When you use `${env:TZ:-${TIMEZONE}}`, you actually want this to happen at configuration-time, since
1. For performance reasons, it better be executed once instead of for every log event
2. It accesses to a *configuration* property: `TIMEZONE`
Hence, it doesn't make sense to access to a Log4j configuration property (`TIMEZONE`) from an external JSON file, i.e., event template provided by the `eventTemplateUri` attribute. That said, as I tried to explain, if you would have inlined your template into `log4j2.xml` using the `eventTemplate` attribute, `${env:TZ:-${TIMEZONE}}` would have just worked fine.
I advise you to inline your _"JTL template fields accessing to configuration properties"_ using [event template additional fields](https://logging.staged.apache.org/log4j/2.x/manual/json-template-layout.html#plugin-element-EventTemplateAdditionalField):
```json
<JsonTemplateLayout eventTemplateUri="classpath:MyLayout.json">
<EventTemplateAdditionalField
key="instant"
format="JSON"
value='{"$resolver": "pattern", "pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}}"}'/>
```
All `${foo}` in the above example will be subject to property substitution at configuration-time.
I will repurpose #2667 to improve JTL docs on this subject.
### Notes on time zone
It is also peculiar that you need to change the time zone in your application depending on an environment variable. Would you mind sharing a little bit more detail about your use case, please? Where do you eventually persist/forward these logs? Why not sticking to UTC doesn't work for you?
### Notes on trailing `}`
You stated
> I dont know why `message1` and `message3` have brace `}` at the end
It is a `StrSubstitutor` bug, you can also reproduce it with `PatternLayout` too:
```xml
<PatternLayout pattern="${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}{GMT+00}}"/>
```
I will create a separate issue for this.
### Conclusion
In short,
* We don't think JTL should be fixed
* I will update JTL docs on this subject
* You can/should use `EventTemplateAdditionalField`s
* Trailing `}` is a bug, which I will address in another ticket/PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `PatternResolver` does not support substitution (logging-log4j2)
Posted by "ppkarwasz (via GitHub)" <gi...@apache.org>.
ppkarwasz commented on issue #2666:
URL: https://github.com/apache/logging-log4j2/issues/2666#issuecomment-2217034113
**Remark**: You can easily workaround the problem with the evaluation of `${env:TZ:-%d{yyyy-MM-dd HH:mm:ss.SSS}{UTC+07}}` by:
* either defining a `TZ` configuration property instead of using a default value in the `${...}` expression:
```xml
<Properties>
<Property name="TZ" value="%d{yyyy-MM-dd HH:mm:ss.SSS}{Asia/Bangkok}"/>
</Properties>
```
Then you can use `${env:TZ}`.
* or definining a configuration property to escape the closing curly brace `}`. E.g.
```xml
<Properties>
<Property name="CBRACE" value="}"/>
</Properties>
```
Then you can use `${end:TZ:-%d{yyyy-MM-dd HH:mm:ss.SSS${CBRACE}{Asia/Bangkok${CBRACE}}`.
[Java does not seem to support the `Asia/Hanoi` timezone]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `PatternResolver` does not support substitution (logging-log4j2)
Posted by "ngocnhan-tran1996 (via GitHub)" <gi...@apache.org>.
ngocnhan-tran1996 commented on issue #2666:
URL: https://github.com/apache/logging-log4j2/issues/2666#issuecomment-2173652696
> @ngocnhan-tran1996, thanks for the report, I've submitted #2667 fixing the issue. I would really appreciate it if you can
>
> 1. Checkout the `fix/2.x/JTL-prop-subs` branch
> 2. Quickly build it using `./mvnw install -DskipTests` – see [`BUILDING.adoc`](/apache/logging-log4j2/blob/2.x/BUILDING.adoc) for details
> 3. And test the installed `2.24.0-SNAPSHOT` artifacts in your project
@vy
Thanks for the answer, I have test on cases with below `EcsLayout.json` and `log4j2.xml`. I type wrong timezone value, GMT instead UTC.
File `EcsLayout.json`
```json
{
"message1": {
"$resolver": "pattern",
"pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}{GMT+00}}"
},
"message2": {
"$resolver": "pattern",
"pattern": "${env:TZ:-${TIMEZONE}}"
},
"message3": {
"$resolver": "pattern",
"pattern": "${env:TZ:-%d{yyyy-MM-dd HH:mm:ss}}"
}
}
```
File `log4j2.xml`
```xml
<Properties>
<Property name="TIMEZONE">%d{yyyy-MM-dd HH:mm:ss}{GMT+03}</Property>
</Properties>
```
**1. Environment `TZ` does not exists**
The result
```json
{"message1":"2024-06-17 21:49:18,263","message2":"2024-06-17 17:49:18","message3":"2024-06-17 21:49:18"}
```
**message2** and **message3** work as my expected
**2. Environment `TZ` do exists**
The result
```json
{"message1":"2024-06-17 14:53}","message2":"2024-06-17 21:53","message3":"2024-06-17 21:53}"}
```
**message2** works as my expected
I dont know why **message1** and **message3** have brace `}` at the end
**Note:** GMT in my local is setting to +07:00
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org