You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Stamatis Zampetakis <za...@gmail.com> on 2022/04/02 16:43:44 UTC
Configure composite filter for specific logger in Log4j2 via properties file
Hi all,
After many unsuccessful attempts to configure a composite filter for a
specified logger [1] using the property syntax, I posted the following
question on stackoverflow [2].
In [1], I tried to create a unit test simulating what I would like to
achieve but didn't manage to make it work yet. I searched in the code to
find examples of how this can be done but couldn't find any.
Any feedback would be much appreciated!
Best,
Stamatis
[1]
https://github.com/zabetak/logging-log4j2/commit/a9944885100db42f7e3ba1b3ff81d59b743d0ab7
[2]
https://stackoverflow.com/questions/71719354/configure-composite-filter-for-specific-logger-in-log4j2-via-properties-file
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by Stamatis Zampetakis <za...@apache.org>.
Thanks a lot for the feedback everyone!
I also find the property syntax a bit hard to use compared to the other available options. I rarely use properties when I am starting a new project but in projects that it is used already I don't have many options apart from learning it and improving it if necessary. Migrating from property syntax to another is always an option but this can have quite a big impact on various aspects so I wouldn't push hard into this direction.
Few comments on the main points raised so far:
1. Wrap individual filters in a "Filters" component
I think it makes sense since that would make it inline with how the XML syntax works and it would solve the problem of multiple filters defined together. Alternatively, raising an error or warning when a filter is defined without an explicit "filters" step would be more helpful for end-users to understand they are doing something wrong. I have a few concerns around this but I guess they could be discussed under the JIRA case if we decide to move this forward.
2. Deterministic order on composite filters
It could be a nice feature since users could order their filters and possibly have performance gains as well. I am not sure though if it is a good idea to make this dependent on the order the properties/elements are defined in the configuration. Looking back into this it was a bad idea from my side to rely on the order that filters are defined. Order of properties, or elements in XML, JSON, etc., shouldn't matter in general so people (including myself) shouldn't rely on it. If we really want to enforce an order on the evaluation of filters then maybe it makes sense to introduce a dedicated attribute.
3. Documentation for property syntax & composite filters
I agree with others, it would be very helpful if they were some more examples in the documentation and/or code base.
The third point can be adressed rather easily since with Piotr's help there is a working example so I can raise a PR for adding the unit test and/or a respective example somewhere in the docs.
If there is consensus about points 1, 2, I can create the respective JIRAs and start working on them when I find some time.
Best,
Stamatis
On 2022/04/03 21:51:31 "Piotr P. Karwasz" wrote:
> On Sun, 3 Apr 2022 at 21:53, Ralph Goers <ra...@dslextreme.com> wrote:
> > Bugs?
> >
> > When it was first implemented it followed a standard format (which should
> > still work). Almost immediately folks started “simplifying” it since the standard
> > format is cumbersome. But being cumbersome is not a bug.
> >
> > The biggest issue is that we simply don’t document it very well and don’t
> > have a lot of examples.
> >
> > But I wouldn’t call having to declare the Filters element a bug.
>
> If not bugs, then discrepancies between the XML format and the
> properties format. One might expect the dot-separated fragments of a
> property to correspond to the element hierarchy in XML. For example:
>
> appender.$1.layout.pattern
>
> might correspond to a
> `/Configuration/Appenders/Console/PatternLayout/@pattern` XPath.
> Discrepancies start to appear in the `<Properties>` and `<Loggers>`
> sub-trees. The former has rules of its own, while the latter:
> 1. does not allow an equivalent of `<Loggers><Root ...` (I would
> expect `logger.$1.type = Root`),
> 2. does not allow a strict equivalent of `<Loggers><Logger
> ...><AppenderRef>` (I would expect `logger.$1.$2.type = AppenderRef`,
> while it is `logger.$1.appenderRef.$2`),
>
> While `logger.$1.filter` follows the same rule as
> `logger.$1.appenderRef` (i.e. introduces a "virtual" XML element for
> the filters), there is a difference between the two: there can be only
> one filter in a `<Logger>` element. That is why it seems natural for
> me to interpret `logger.$1.filter` as the equivalent of
> `<Loggers><Logger ...><Filters>`.
>
> Piotr
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
On Sun, 3 Apr 2022 at 21:53, Ralph Goers <ra...@dslextreme.com> wrote:
> Bugs?
>
> When it was first implemented it followed a standard format (which should
> still work). Almost immediately folks started “simplifying” it since the standard
> format is cumbersome. But being cumbersome is not a bug.
>
> The biggest issue is that we simply don’t document it very well and don’t
> have a lot of examples.
>
> But I wouldn’t call having to declare the Filters element a bug.
If not bugs, then discrepancies between the XML format and the
properties format. One might expect the dot-separated fragments of a
property to correspond to the element hierarchy in XML. For example:
appender.$1.layout.pattern
might correspond to a
`/Configuration/Appenders/Console/PatternLayout/@pattern` XPath.
Discrepancies start to appear in the `<Properties>` and `<Loggers>`
sub-trees. The former has rules of its own, while the latter:
1. does not allow an equivalent of `<Loggers><Root ...` (I would
expect `logger.$1.type = Root`),
2. does not allow a strict equivalent of `<Loggers><Logger
...><AppenderRef>` (I would expect `logger.$1.$2.type = AppenderRef`,
while it is `logger.$1.appenderRef.$2`),
While `logger.$1.filter` follows the same rule as
`logger.$1.appenderRef` (i.e. introduces a "virtual" XML element for
the filters), there is a difference between the two: there can be only
one filter in a `<Logger>` element. That is why it seems natural for
me to interpret `logger.$1.filter` as the equivalent of
`<Loggers><Logger ...><Filters>`.
Piotr
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Apr 3, 2022, at 11:59 AM, Piotr P. Karwasz <pi...@gmail.com> wrote:.
>
> I would go even further, regarding the properties format: no Log4j2
> developer uses it and that explains why there are still many bugs in
> the `PropertiesConfiguration`.
Bugs?
When it was first implemented it followed a standard format (which should
still work). Almost immediately folks started “simplifying” it since the standard
format is cumbersome. But being cumbersome is not a bug.
The biggest issue is that we simply don’t document it very well and don’t
have a lot of examples.
But I wouldn’t call having to declare the Filters element a bug.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hi Ralph,
On Sun, 3 Apr 2022 at 19:00, Ralph Goers <ra...@dslextreme.com> wrote:
> The enhancement you suggest would make the properties configuration different than
> the other configuration formats as they all require the Filters element. This is yet another
> example of why most (or all) of the log4j 2 developers really do not like the properties
> format and wonder why so many people prefer it.
Yes, you are right, no need to introduce exceptions. Even if there is
only one filter, a property
logger.logger-id.filter.filter-id.type = ThresholdFilter
should be equivalent to:
<Loggers>
<Logger ...>
<Filters>
<ThresholdFilter .../>
Right now the `PropertiesConfigurationBuilder` does not add the
"Filters" component, so the property above gives the equivalent of:
<Loggers>
<Logger ...>
<ThresholdFilter .../>
This works, but fails if the user configures more than one filter (as
it would in every configuration format). BTW, I think that a property:
logger.logger-id.filter-id.type = ThresholdFilter
should also work to define a single filter. Right now it is ignored by
the `PropertiesConfigurationBuilder`.
I would go even further, regarding the properties format: no Log4j2
developer uses it and that explains why there are still many bugs in
the `PropertiesConfiguration`.
Piotr
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by Ralph Goers <ra...@dslextreme.com>.
I would expect the Properties issue would affect all the implementations.
The enhancement you suggest would make the properties configuration different than
the other configuration formats as they all require the Filters element. This is yet another
example of why most (or all) of the log4j 2 developers really do not like the properties
format and wonder why so many people prefer it.
FWIW, I am not against making the proposed change. We already do other stuff to make
the properties configuration more usable.
Ralph
> On Apr 2, 2022, at 11:08 PM, Piotr P. Karwasz <pi...@gmail.com> wrote:
>
> Hello Stamatis,
>
> I posted a working solution on SO:
>
> logger.t1.filter.M.type = Filters
> logger.t1.filter.M.f1.type = RegexFilter
> logger.t1.filter.M.f1.regex = .*SQL.*
> logger.t1.filter.M.f1.onMatch = NEUTRAL
> logger.t1.filter.M.f1.onMismatch = DENY
> logger.t1.filter.M.f2.type = RegexFilter
> logger.t1.filter.M.f2.regex = .*JPQL.*
> logger.t1.filter.M.f2.onMatch = NEUTRAL
> logger.t1.filter.M.f2.onMismatch = DENY
>
> The problem is caused by two facts:
> 1. The `LoggerConfig` only accepts a single filter, so you have to
> wrap them in a "Filters" element,
> 2. A `Properties` class is a `Hashtable`, so the order of the keys is random.
>
> I believe both of them to be bugs of `PropertiesConfigurationBuilder`:
> 1. I interpret the "filter" prefix in the properties configuration as
> an implicit "Filters" component. `addFiltersToComponent` should wrap
> the filters in a "Filters" component if more than one is defined.
> 2. `PropertiesUtil#partitionOnCommonPrefixes` should probably return a
> `TreeMap` instead of a `HashMap` to impose a deterministic order on
> the filters (and other components that use this function).
>
> You should open a JIRA ticket for this problem and since you
> apparently have a PR in progress, submit your test case and a solution
> as PR.
>
> Best wishes,
> Piotr
>
> On Sat, 2 Apr 2022 at 18:44, Stamatis Zampetakis <za...@gmail.com> wrote:
>>
>> Hi all,
>>
>> After many unsuccessful attempts to configure a composite filter for a
>> specified logger [1] using the property syntax, I posted the following
>> question on stackoverflow [2].
>>
>> In [1], I tried to create a unit test simulating what I would like to
>> achieve but didn't manage to make it work yet. I searched in the code to
>> find examples of how this can be done but couldn't find any.
>>
>> Any feedback would be much appreciated!
>>
>> Best,
>> Stamatis
>>
>> [1]
>> https://github.com/zabetak/logging-log4j2/commit/a9944885100db42f7e3ba1b3ff81d59b743d0ab7
>> [2]
>> https://stackoverflow.com/questions/71719354/configure-composite-filter-for-specific-logger-in-log4j2-via-properties-file
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: Configure composite filter for specific logger in Log4j2 via properties file
Posted by "Piotr P. Karwasz" <pi...@gmail.com>.
Hello Stamatis,
I posted a working solution on SO:
logger.t1.filter.M.type = Filters
logger.t1.filter.M.f1.type = RegexFilter
logger.t1.filter.M.f1.regex = .*SQL.*
logger.t1.filter.M.f1.onMatch = NEUTRAL
logger.t1.filter.M.f1.onMismatch = DENY
logger.t1.filter.M.f2.type = RegexFilter
logger.t1.filter.M.f2.regex = .*JPQL.*
logger.t1.filter.M.f2.onMatch = NEUTRAL
logger.t1.filter.M.f2.onMismatch = DENY
The problem is caused by two facts:
1. The `LoggerConfig` only accepts a single filter, so you have to
wrap them in a "Filters" element,
2. A `Properties` class is a `Hashtable`, so the order of the keys is random.
I believe both of them to be bugs of `PropertiesConfigurationBuilder`:
1. I interpret the "filter" prefix in the properties configuration as
an implicit "Filters" component. `addFiltersToComponent` should wrap
the filters in a "Filters" component if more than one is defined.
2. `PropertiesUtil#partitionOnCommonPrefixes` should probably return a
`TreeMap` instead of a `HashMap` to impose a deterministic order on
the filters (and other components that use this function).
You should open a JIRA ticket for this problem and since you
apparently have a PR in progress, submit your test case and a solution
as PR.
Best wishes,
Piotr
On Sat, 2 Apr 2022 at 18:44, Stamatis Zampetakis <za...@gmail.com> wrote:
>
> Hi all,
>
> After many unsuccessful attempts to configure a composite filter for a
> specified logger [1] using the property syntax, I posted the following
> question on stackoverflow [2].
>
> In [1], I tried to create a unit test simulating what I would like to
> achieve but didn't manage to make it work yet. I searched in the code to
> find examples of how this can be done but couldn't find any.
>
> Any feedback would be much appreciated!
>
> Best,
> Stamatis
>
> [1]
> https://github.com/zabetak/logging-log4j2/commit/a9944885100db42f7e3ba1b3ff81d59b743d0ab7
> [2]
> https://stackoverflow.com/questions/71719354/configure-composite-filter-for-specific-logger-in-log4j2-via-properties-file
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org