You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Adwait Kumar Singh <th...@gmail.com> on 2021/10/21 13:07:24 UTC
How to inject a dependency into a custom filter.
I am creating a filter which requires the user to give a Supplier<Rule> to
fetch some custom rules. I am wondering how to go about getting this
Supplier from the configuration xml.
One way I have thought of is using some sort of Registry and then just
asking the user to provide a key to lookup into the Registry to fetch the
Supplier. The user then registers his Supplier to this Registry in Java
code.
What are your thoughts on this and do you guys know of a better way to do
this?
Thanks,
Adwait.
Re: How to inject a dependency into a custom filter.
Posted by Ralph Goers <ra...@dslextreme.com>.
As Gary already mentioned you really should consider using Log4j’s dynamic reloading of the configuration along with support for composite configurations. I am using this at storing our log4j configuration files in a Spring Cloud Configuration Server backed by Git. We have a common log4j2.xml that is shared by all our services and each service can have its own override file. Log4j’s Spring Cloud Config support can hook into Spring’s notification bus and be notified when changes are made or it can simply periodically poll to see if anything has changed.
There are issues with what you are proposing.
1. You are trying to add new filter levels for a logger as a filter. Log4j optimizes for checking logging levels. The check you are proposing would be quite a bit slower.
2. Is allowedHosts evaluating a value from the ThreadContext?
3. It isn't clear how the rules interrelate. I am guessing they are all logically anded together?
What is clear is that you would like sections to the configuration to be bounded by a time interval. That seems like a reasonable request but could be difficult to accomplish in the existing components. That is where your syntax makes more sense but I think that might be accomplished by wrapping a set of filters in a time filter.
Ralph
> On Oct 21, 2021, at 11:18 PM, Adwait Kumar Singh <th...@gmail.com> wrote:
>
> This is a sample Rule I had in mind,
>
> {
>> "loggerThresholds": {
>> "a.b.c.Class1": "WARN",
>> "a.b.c.Class2": "TRACE",
>> "a.b.c.Class3": "OFF"
>> },
>> "allowlistedHosts": [
>> "a.b.c.us-west-1.com",
>> "a.b.c.us-west-2.com"
>> ],
>> "ruleValidityTime": "2022-01-24T00:03:31.593"
>> }
>>
>
> This basically would filter logging from the given loggers in
> loggerThresholds only on the allowlistedHosts until ruleValidityTime.
>
> The advantage of this over new filters is, new filters require code
> changes, this wouldn't as my intention is to store the rules in some
> configuration store and it would be fetched via the Supplier.
>
> On Fri, Oct 22, 2021 at 7:01 AM Apache <ra...@dslextreme.com> wrote:
>
>> After thinking about this more I can’t quite figure out what the user
>> would be specifying in the configuration. A Java lambda expression? I.e
>> what is the rule syntax? Or are they expected to just configure predefined
>> rules with perhaps some variables. But if that is the case how is this
>> really any different than just creating new filters? The only real
>> difference I see is that the Supplier presumably returns a Boolean instead
>> of a filter result.
>>
>> Ralph
>>
>>> On Oct 21, 2021, at 4:51 PM, Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>>
>>> This sounds interesting. I might give it a whirl but firstI am trying
>> to finish up some work on
>>> Apache Flume so it can get a new release and I need to do some other
>> work here as we
>>> are overdue for a release.
>>>
>>> Ralph
>>>
>>>> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <
>> theadvaitkumarsingh@gmail.com> wrote:
>>>>
>>>> The problem I am stuck on is the best way to get the user to provide the
>>>> ruleSupplier via configuration.
>>>>
>>>>> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>
>>>>> Yes I mean filtering on those loggers.
>>>>>
>>>>> I don't expect the Supplier to be dynamically changed, only the rules
>> they
>>>>> are fetching. Something roughly like this
>>>>>
>>>>> public class RuleBaseFilter extends AbstractFilter {
>>>>>
>>>>> //This will be initialized only once
>>>>> Supplier<List<Rules>> rulesSupplier;
>>>>>
>>>>>
>>>>> @Override
>>>>> public Result filter(final Logger logger, final Level level, final
>>>>> Marker marker, final Message msg,
>>>>> final Throwable t) {
>>>>> List<Rules> rules = rulesSupplier.get();
>>>>> //evaluate rules and filter
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <
>> ralph.goers@dslextreme.com>
>>>>> wrote:
>>>>>
>>>>>> Sorry, you have lost me a little bit.
>>>>>>
>>>>>> When you say dynamically turn on/off loggers I am assuming you really
>>>>>> mean
>>>>>> enable/disable filtering on those Loggers?
>>>>>>
>>>>>> Are you thinking that the rules can be added or removed dynamically?
>> Or
>>>>>> just the
>>>>>> parameters to the rules? I ask because you can’t really add a Supplier
>>>>>> dynamically
>>>>>> if it is part of the configuration. Log4j will process the
>> configuration
>>>>>> only when it is
>>>>>> changed. However, the data they rely on can come from anywhere - more
>> or
>>>>>> less
>>>>>> how we use Lookups to evaluate conditions on every log event.
>>>>>>
>>>>>> Ralph
>>>>>>
>>>>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>>
>>>>>>> This is still in the proof of concept phase, but what I am trying to
>> do
>>>>>> is
>>>>>>> create a filter which allows developers to dynamically turn on/off
>>>>>> loggers
>>>>>>> based on some parameters without restarting their system. The
>> parameters
>>>>>>> can be varied, like:
>>>>>>> 1. Turn on WARN log levels of a particular class.
>>>>>>> 2. Turn on TRACE logging only on a particular host.
>>>>>>> 3. Turn on DEBUG logs only for requests by a particular client for a
>>>>>> short
>>>>>>> duration,
>>>>>>> ... and any combination of such parameters.
>>>>>>>
>>>>>>> The way I was thinking of doing this was creating a filter which
>> takes
>>>>>> in a
>>>>>>> Supplier<Rule> and then evaluates a log event against that rule.
>>>>>>>
>>>>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> It sounds to me like you want to create a new type of plugin
>> similar to
>>>>>>>> PatternConerters. I am curious as to what your filter does. Is it
>>>>>> something
>>>>>>>> that would be useful to others so that we would be interested in
>>>>>> including
>>>>>>>> it?
>>>>>>>>
>>>>>>>> Ralph
>>>>>>>>
>>>>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>>>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> I am creating a filter which requires the user to give a
>>>>>> Supplier<Rule>
>>>>>>>> to
>>>>>>>>> fetch some custom rules. I am wondering how to go about getting
>> this
>>>>>>>>> Supplier from the configuration xml.
>>>>>>>>>
>>>>>>>>> One way I have thought of is using some sort of Registry and then
>> just
>>>>>>>>> asking the user to provide a key to lookup into the Registry to
>> fetch
>>>>>> the
>>>>>>>>> Supplier. The user then registers his Supplier to this Registry in
>>>>>> Java
>>>>>>>>> code.
>>>>>>>>> What are your thoughts on this and do you guys know of a better way
>>>>>> to do
>>>>>>>>> this?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Adwait.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
>>
>>
>>
Re: How to inject a dependency into a custom filter.
Posted by Matt Sicker <bo...@gmail.com>.
The script filters are already dynamic for script engines. Have you looked at that?
Matt Sicker
> On Oct 22, 2021, at 03:22, Gary Gregory <ga...@gmail.com> wrote:
>
> Why not reuse the feature that lets log4j automatically reload a
> configuration file if it detects the file has changed?
>
> Then we only have to consider what kind of configuration is needed.
>
> Gary
>
>> On Fri, Oct 22, 2021, 02:18 Adwait Kumar Singh <
>> theadvaitkumarsingh@gmail.com> wrote:
>>
>> This is a sample Rule I had in mind,
>>
>> {
>>> "loggerThresholds": {
>>> "a.b.c.Class1": "WARN",
>>> "a.b.c.Class2": "TRACE",
>>> "a.b.c.Class3": "OFF"
>>> },
>>> "allowlistedHosts": [
>>> "a.b.c.us-west-1.com",
>>> "a.b.c.us-west-2.com"
>>> ],
>>> "ruleValidityTime": "2022-01-24T00:03:31.593"
>>> }
>>>
>>
>> This basically would filter logging from the given loggers in
>> loggerThresholds only on the allowlistedHosts until ruleValidityTime.
>>
>> The advantage of this over new filters is, new filters require code
>> changes, this wouldn't as my intention is to store the rules in some
>> configuration store and it would be fetched via the Supplier.
>>
>>> On Fri, Oct 22, 2021 at 7:01 AM Apache <ra...@dslextreme.com> wrote:
>>>
>>> After thinking about this more I can’t quite figure out what the user
>>> would be specifying in the configuration. A Java lambda expression? I.e
>>> what is the rule syntax? Or are they expected to just configure
>> predefined
>>> rules with perhaps some variables. But if that is the case how is this
>>> really any different than just creating new filters? The only real
>>> difference I see is that the Supplier presumably returns a Boolean
>> instead
>>> of a filter result.
>>>
>>> Ralph
>>>
>>>> On Oct 21, 2021, at 4:51 PM, Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>>
>>>> This sounds interesting. I might give it a whirl but firstI am trying
>>> to finish up some work on
>>>> Apache Flume so it can get a new release and I need to do some other
>>> work here as we
>>>> are overdue for a release.
>>>>
>>>> Ralph
>>>>
>>>>> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <
>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>
>>>>> The problem I am stuck on is the best way to get the user to provide
>> the
>>>>> ruleSupplier via configuration.
>>>>>
>>>>>> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>
>>>>>> Yes I mean filtering on those loggers.
>>>>>>
>>>>>> I don't expect the Supplier to be dynamically changed, only the rules
>>> they
>>>>>> are fetching. Something roughly like this
>>>>>>
>>>>>> public class RuleBaseFilter extends AbstractFilter {
>>>>>>
>>>>>> //This will be initialized only once
>>>>>> Supplier<List<Rules>> rulesSupplier;
>>>>>>
>>>>>>
>>>>>> @Override
>>>>>> public Result filter(final Logger logger, final Level level, final
>>>>>> Marker marker, final Message msg,
>>>>>> final Throwable t) {
>>>>>> List<Rules> rules = rulesSupplier.get();
>>>>>> //evaluate rules and filter
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <
>>> ralph.goers@dslextreme.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Sorry, you have lost me a little bit.
>>>>>>>
>>>>>>> When you say dynamically turn on/off loggers I am assuming you
>> really
>>>>>>> mean
>>>>>>> enable/disable filtering on those Loggers?
>>>>>>>
>>>>>>> Are you thinking that the rules can be added or removed dynamically?
>>> Or
>>>>>>> just the
>>>>>>> parameters to the rules? I ask because you can’t really add a
>> Supplier
>>>>>>> dynamically
>>>>>>> if it is part of the configuration. Log4j will process the
>>> configuration
>>>>>>> only when it is
>>>>>>> changed. However, the data they rely on can come from anywhere -
>> more
>>> or
>>>>>>> less
>>>>>>> how we use Lookups to evaluate conditions on every log event.
>>>>>>>
>>>>>>> Ralph
>>>>>>>
>>>>>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
>>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>>>
>>>>>>>> This is still in the proof of concept phase, but what I am trying
>> to
>>> do
>>>>>>> is
>>>>>>>> create a filter which allows developers to dynamically turn on/off
>>>>>>> loggers
>>>>>>>> based on some parameters without restarting their system. The
>>> parameters
>>>>>>>> can be varied, like:
>>>>>>>> 1. Turn on WARN log levels of a particular class.
>>>>>>>> 2. Turn on TRACE logging only on a particular host.
>>>>>>>> 3. Turn on DEBUG logs only for requests by a particular client for
>> a
>>>>>>> short
>>>>>>>> duration,
>>>>>>>> ... and any combination of such parameters.
>>>>>>>>
>>>>>>>> The way I was thinking of doing this was creating a filter which
>>> takes
>>>>>>> in a
>>>>>>>> Supplier<Rule> and then evaluates a log event against that rule.
>>>>>>>>
>>>>>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ralph.goers@dslextreme.com
>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> It sounds to me like you want to create a new type of plugin
>>> similar to
>>>>>>>>> PatternConerters. I am curious as to what your filter does. Is it
>>>>>>> something
>>>>>>>>> that would be useful to others so that we would be interested in
>>>>>>> including
>>>>>>>>> it?
>>>>>>>>>
>>>>>>>>> Ralph
>>>>>>>>>
>>>>>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>>>>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> I am creating a filter which requires the user to give a
>>>>>>> Supplier<Rule>
>>>>>>>>> to
>>>>>>>>>> fetch some custom rules. I am wondering how to go about getting
>>> this
>>>>>>>>>> Supplier from the configuration xml.
>>>>>>>>>>
>>>>>>>>>> One way I have thought of is using some sort of Registry and then
>>> just
>>>>>>>>>> asking the user to provide a key to lookup into the Registry to
>>> fetch
>>>>>>> the
>>>>>>>>>> Supplier. The user then registers his Supplier to this Registry
>> in
>>>>>>> Java
>>>>>>>>>> code.
>>>>>>>>>> What are your thoughts on this and do you guys know of a better
>> way
>>>>>>> to do
>>>>>>>>>> this?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Adwait.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>
>>>
>>>
>>
Re: How to inject a dependency into a custom filter.
Posted by Gary Gregory <ga...@gmail.com>.
Why not reuse the feature that lets log4j automatically reload a
configuration file if it detects the file has changed?
Then we only have to consider what kind of configuration is needed.
Gary
On Fri, Oct 22, 2021, 02:18 Adwait Kumar Singh <
theadvaitkumarsingh@gmail.com> wrote:
> This is a sample Rule I had in mind,
>
> {
> > "loggerThresholds": {
> > "a.b.c.Class1": "WARN",
> > "a.b.c.Class2": "TRACE",
> > "a.b.c.Class3": "OFF"
> > },
> > "allowlistedHosts": [
> > "a.b.c.us-west-1.com",
> > "a.b.c.us-west-2.com"
> > ],
> > "ruleValidityTime": "2022-01-24T00:03:31.593"
> > }
> >
>
> This basically would filter logging from the given loggers in
> loggerThresholds only on the allowlistedHosts until ruleValidityTime.
>
> The advantage of this over new filters is, new filters require code
> changes, this wouldn't as my intention is to store the rules in some
> configuration store and it would be fetched via the Supplier.
>
> On Fri, Oct 22, 2021 at 7:01 AM Apache <ra...@dslextreme.com> wrote:
>
> > After thinking about this more I can’t quite figure out what the user
> > would be specifying in the configuration. A Java lambda expression? I.e
> > what is the rule syntax? Or are they expected to just configure
> predefined
> > rules with perhaps some variables. But if that is the case how is this
> > really any different than just creating new filters? The only real
> > difference I see is that the Supplier presumably returns a Boolean
> instead
> > of a filter result.
> >
> > Ralph
> >
> > > On Oct 21, 2021, at 4:51 PM, Ralph Goers <ra...@dslextreme.com>
> > wrote:
> > >
> > > This sounds interesting. I might give it a whirl but firstI am trying
> > to finish up some work on
> > > Apache Flume so it can get a new release and I need to do some other
> > work here as we
> > > are overdue for a release.
> > >
> > > Ralph
> > >
> > >> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <
> > theadvaitkumarsingh@gmail.com> wrote:
> > >>
> > >> The problem I am stuck on is the best way to get the user to provide
> the
> > >> ruleSupplier via configuration.
> > >>
> > >>> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
> > >>> theadvaitkumarsingh@gmail.com> wrote:
> > >>>
> > >>> Yes I mean filtering on those loggers.
> > >>>
> > >>> I don't expect the Supplier to be dynamically changed, only the rules
> > they
> > >>> are fetching. Something roughly like this
> > >>>
> > >>> public class RuleBaseFilter extends AbstractFilter {
> > >>>
> > >>> //This will be initialized only once
> > >>> Supplier<List<Rules>> rulesSupplier;
> > >>>
> > >>>
> > >>> @Override
> > >>> public Result filter(final Logger logger, final Level level, final
> > >>> Marker marker, final Message msg,
> > >>> final Throwable t) {
> > >>> List<Rules> rules = rulesSupplier.get();
> > >>> //evaluate rules and filter
> > >>> }
> > >>>
> > >>> }
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <
> > ralph.goers@dslextreme.com>
> > >>> wrote:
> > >>>
> > >>>> Sorry, you have lost me a little bit.
> > >>>>
> > >>>> When you say dynamically turn on/off loggers I am assuming you
> really
> > >>>> mean
> > >>>> enable/disable filtering on those Loggers?
> > >>>>
> > >>>> Are you thinking that the rules can be added or removed dynamically?
> > Or
> > >>>> just the
> > >>>> parameters to the rules? I ask because you can’t really add a
> Supplier
> > >>>> dynamically
> > >>>> if it is part of the configuration. Log4j will process the
> > configuration
> > >>>> only when it is
> > >>>> changed. However, the data they rely on can come from anywhere -
> more
> > or
> > >>>> less
> > >>>> how we use Lookups to evaluate conditions on every log event.
> > >>>>
> > >>>> Ralph
> > >>>>
> > >>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
> > >>>> theadvaitkumarsingh@gmail.com> wrote:
> > >>>>>
> > >>>>> This is still in the proof of concept phase, but what I am trying
> to
> > do
> > >>>> is
> > >>>>> create a filter which allows developers to dynamically turn on/off
> > >>>> loggers
> > >>>>> based on some parameters without restarting their system. The
> > parameters
> > >>>>> can be varied, like:
> > >>>>> 1. Turn on WARN log levels of a particular class.
> > >>>>> 2. Turn on TRACE logging only on a particular host.
> > >>>>> 3. Turn on DEBUG logs only for requests by a particular client for
> a
> > >>>> short
> > >>>>> duration,
> > >>>>> ... and any combination of such parameters.
> > >>>>>
> > >>>>> The way I was thinking of doing this was creating a filter which
> > takes
> > >>>> in a
> > >>>>> Supplier<Rule> and then evaluates a log event against that rule.
> > >>>>>
> > >>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ralph.goers@dslextreme.com
> >
> > >>>> wrote:
> > >>>>>
> > >>>>>> It sounds to me like you want to create a new type of plugin
> > similar to
> > >>>>>> PatternConerters. I am curious as to what your filter does. Is it
> > >>>> something
> > >>>>>> that would be useful to others so that we would be interested in
> > >>>> including
> > >>>>>> it?
> > >>>>>>
> > >>>>>> Ralph
> > >>>>>>
> > >>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
> > >>>>>> theadvaitkumarsingh@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>> I am creating a filter which requires the user to give a
> > >>>> Supplier<Rule>
> > >>>>>> to
> > >>>>>>> fetch some custom rules. I am wondering how to go about getting
> > this
> > >>>>>>> Supplier from the configuration xml.
> > >>>>>>>
> > >>>>>>> One way I have thought of is using some sort of Registry and then
> > just
> > >>>>>>> asking the user to provide a key to lookup into the Registry to
> > fetch
> > >>>> the
> > >>>>>>> Supplier. The user then registers his Supplier to this Registry
> in
> > >>>> Java
> > >>>>>>> code.
> > >>>>>>> What are your thoughts on this and do you guys know of a better
> way
> > >>>> to do
> > >>>>>>> this?
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Adwait.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>>>
> > >
> >
> >
> >
>
Re: How to inject a dependency into a custom filter.
Posted by Adwait Kumar Singh <th...@gmail.com>.
This is a sample Rule I had in mind,
{
> "loggerThresholds": {
> "a.b.c.Class1": "WARN",
> "a.b.c.Class2": "TRACE",
> "a.b.c.Class3": "OFF"
> },
> "allowlistedHosts": [
> "a.b.c.us-west-1.com",
> "a.b.c.us-west-2.com"
> ],
> "ruleValidityTime": "2022-01-24T00:03:31.593"
> }
>
This basically would filter logging from the given loggers in
loggerThresholds only on the allowlistedHosts until ruleValidityTime.
The advantage of this over new filters is, new filters require code
changes, this wouldn't as my intention is to store the rules in some
configuration store and it would be fetched via the Supplier.
On Fri, Oct 22, 2021 at 7:01 AM Apache <ra...@dslextreme.com> wrote:
> After thinking about this more I can’t quite figure out what the user
> would be specifying in the configuration. A Java lambda expression? I.e
> what is the rule syntax? Or are they expected to just configure predefined
> rules with perhaps some variables. But if that is the case how is this
> really any different than just creating new filters? The only real
> difference I see is that the Supplier presumably returns a Boolean instead
> of a filter result.
>
> Ralph
>
> > On Oct 21, 2021, at 4:51 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
> >
> > This sounds interesting. I might give it a whirl but firstI am trying
> to finish up some work on
> > Apache Flume so it can get a new release and I need to do some other
> work here as we
> > are overdue for a release.
> >
> > Ralph
> >
> >> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <
> theadvaitkumarsingh@gmail.com> wrote:
> >>
> >> The problem I am stuck on is the best way to get the user to provide the
> >> ruleSupplier via configuration.
> >>
> >>> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
> >>> theadvaitkumarsingh@gmail.com> wrote:
> >>>
> >>> Yes I mean filtering on those loggers.
> >>>
> >>> I don't expect the Supplier to be dynamically changed, only the rules
> they
> >>> are fetching. Something roughly like this
> >>>
> >>> public class RuleBaseFilter extends AbstractFilter {
> >>>
> >>> //This will be initialized only once
> >>> Supplier<List<Rules>> rulesSupplier;
> >>>
> >>>
> >>> @Override
> >>> public Result filter(final Logger logger, final Level level, final
> >>> Marker marker, final Message msg,
> >>> final Throwable t) {
> >>> List<Rules> rules = rulesSupplier.get();
> >>> //evaluate rules and filter
> >>> }
> >>>
> >>> }
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <
> ralph.goers@dslextreme.com>
> >>> wrote:
> >>>
> >>>> Sorry, you have lost me a little bit.
> >>>>
> >>>> When you say dynamically turn on/off loggers I am assuming you really
> >>>> mean
> >>>> enable/disable filtering on those Loggers?
> >>>>
> >>>> Are you thinking that the rules can be added or removed dynamically?
> Or
> >>>> just the
> >>>> parameters to the rules? I ask because you can’t really add a Supplier
> >>>> dynamically
> >>>> if it is part of the configuration. Log4j will process the
> configuration
> >>>> only when it is
> >>>> changed. However, the data they rely on can come from anywhere - more
> or
> >>>> less
> >>>> how we use Lookups to evaluate conditions on every log event.
> >>>>
> >>>> Ralph
> >>>>
> >>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
> >>>> theadvaitkumarsingh@gmail.com> wrote:
> >>>>>
> >>>>> This is still in the proof of concept phase, but what I am trying to
> do
> >>>> is
> >>>>> create a filter which allows developers to dynamically turn on/off
> >>>> loggers
> >>>>> based on some parameters without restarting their system. The
> parameters
> >>>>> can be varied, like:
> >>>>> 1. Turn on WARN log levels of a particular class.
> >>>>> 2. Turn on TRACE logging only on a particular host.
> >>>>> 3. Turn on DEBUG logs only for requests by a particular client for a
> >>>> short
> >>>>> duration,
> >>>>> ... and any combination of such parameters.
> >>>>>
> >>>>> The way I was thinking of doing this was creating a filter which
> takes
> >>>> in a
> >>>>> Supplier<Rule> and then evaluates a log event against that rule.
> >>>>>
> >>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
> >>>> wrote:
> >>>>>
> >>>>>> It sounds to me like you want to create a new type of plugin
> similar to
> >>>>>> PatternConerters. I am curious as to what your filter does. Is it
> >>>> something
> >>>>>> that would be useful to others so that we would be interested in
> >>>> including
> >>>>>> it?
> >>>>>>
> >>>>>> Ralph
> >>>>>>
> >>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
> >>>>>> theadvaitkumarsingh@gmail.com> wrote:
> >>>>>>>
> >>>>>>> I am creating a filter which requires the user to give a
> >>>> Supplier<Rule>
> >>>>>> to
> >>>>>>> fetch some custom rules. I am wondering how to go about getting
> this
> >>>>>>> Supplier from the configuration xml.
> >>>>>>>
> >>>>>>> One way I have thought of is using some sort of Registry and then
> just
> >>>>>>> asking the user to provide a key to lookup into the Registry to
> fetch
> >>>> the
> >>>>>>> Supplier. The user then registers his Supplier to this Registry in
> >>>> Java
> >>>>>>> code.
> >>>>>>> What are your thoughts on this and do you guys know of a better way
> >>>> to do
> >>>>>>> this?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Adwait.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >
>
>
>
Re: How to inject a dependency into a custom filter.
Posted by Apache <ra...@dslextreme.com>.
After thinking about this more I can’t quite figure out what the user would be specifying in the configuration. A Java lambda expression? I.e what is the rule syntax? Or are they expected to just configure predefined rules with perhaps some variables. But if that is the case how is this really any different than just creating new filters? The only real difference I see is that the Supplier presumably returns a Boolean instead of a filter result.
Ralph
> On Oct 21, 2021, at 4:51 PM, Ralph Goers <ra...@dslextreme.com> wrote:
>
> This sounds interesting. I might give it a whirl but firstI am trying to finish up some work on
> Apache Flume so it can get a new release and I need to do some other work here as we
> are overdue for a release.
>
> Ralph
>
>> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <th...@gmail.com> wrote:
>>
>> The problem I am stuck on is the best way to get the user to provide the
>> ruleSupplier via configuration.
>>
>>> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
>>> theadvaitkumarsingh@gmail.com> wrote:
>>>
>>> Yes I mean filtering on those loggers.
>>>
>>> I don't expect the Supplier to be dynamically changed, only the rules they
>>> are fetching. Something roughly like this
>>>
>>> public class RuleBaseFilter extends AbstractFilter {
>>>
>>> //This will be initialized only once
>>> Supplier<List<Rules>> rulesSupplier;
>>>
>>>
>>> @Override
>>> public Result filter(final Logger logger, final Level level, final
>>> Marker marker, final Message msg,
>>> final Throwable t) {
>>> List<Rules> rules = rulesSupplier.get();
>>> //evaluate rules and filter
>>> }
>>>
>>> }
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>>> Sorry, you have lost me a little bit.
>>>>
>>>> When you say dynamically turn on/off loggers I am assuming you really
>>>> mean
>>>> enable/disable filtering on those Loggers?
>>>>
>>>> Are you thinking that the rules can be added or removed dynamically? Or
>>>> just the
>>>> parameters to the rules? I ask because you can’t really add a Supplier
>>>> dynamically
>>>> if it is part of the configuration. Log4j will process the configuration
>>>> only when it is
>>>> changed. However, the data they rely on can come from anywhere - more or
>>>> less
>>>> how we use Lookups to evaluate conditions on every log event.
>>>>
>>>> Ralph
>>>>
>>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>
>>>>> This is still in the proof of concept phase, but what I am trying to do
>>>> is
>>>>> create a filter which allows developers to dynamically turn on/off
>>>> loggers
>>>>> based on some parameters without restarting their system. The parameters
>>>>> can be varied, like:
>>>>> 1. Turn on WARN log levels of a particular class.
>>>>> 2. Turn on TRACE logging only on a particular host.
>>>>> 3. Turn on DEBUG logs only for requests by a particular client for a
>>>> short
>>>>> duration,
>>>>> ... and any combination of such parameters.
>>>>>
>>>>> The way I was thinking of doing this was creating a filter which takes
>>>> in a
>>>>> Supplier<Rule> and then evaluates a log event against that rule.
>>>>>
>>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
>>>> wrote:
>>>>>
>>>>>> It sounds to me like you want to create a new type of plugin similar to
>>>>>> PatternConerters. I am curious as to what your filter does. Is it
>>>> something
>>>>>> that would be useful to others so that we would be interested in
>>>> including
>>>>>> it?
>>>>>>
>>>>>> Ralph
>>>>>>
>>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>>
>>>>>>> I am creating a filter which requires the user to give a
>>>> Supplier<Rule>
>>>>>> to
>>>>>>> fetch some custom rules. I am wondering how to go about getting this
>>>>>>> Supplier from the configuration xml.
>>>>>>>
>>>>>>> One way I have thought of is using some sort of Registry and then just
>>>>>>> asking the user to provide a key to lookup into the Registry to fetch
>>>> the
>>>>>>> Supplier. The user then registers his Supplier to this Registry in
>>>> Java
>>>>>>> code.
>>>>>>> What are your thoughts on this and do you guys know of a better way
>>>> to do
>>>>>>> this?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Adwait.
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>
Re: How to inject a dependency into a custom filter.
Posted by Ralph Goers <ra...@dslextreme.com>.
This sounds interesting. I might give it a whirl but firstI am trying to finish up some work on
Apache Flume so it can get a new release and I need to do some other work here as we
are overdue for a release.
Ralph
> On Oct 21, 2021, at 11:28 AM, Adwait Kumar Singh <th...@gmail.com> wrote:
>
> The problem I am stuck on is the best way to get the user to provide the
> ruleSupplier via configuration.
>
> On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
> theadvaitkumarsingh@gmail.com> wrote:
>
>> Yes I mean filtering on those loggers.
>>
>> I don't expect the Supplier to be dynamically changed, only the rules they
>> are fetching. Something roughly like this
>>
>> public class RuleBaseFilter extends AbstractFilter {
>>
>> //This will be initialized only once
>> Supplier<List<Rules>> rulesSupplier;
>>
>>
>> @Override
>> public Result filter(final Logger logger, final Level level, final
>> Marker marker, final Message msg,
>> final Throwable t) {
>> List<Rules> rules = rulesSupplier.get();
>> //evaluate rules and filter
>> }
>>
>> }
>>
>>
>>
>>
>>
>> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <ra...@dslextreme.com>
>> wrote:
>>
>>> Sorry, you have lost me a little bit.
>>>
>>> When you say dynamically turn on/off loggers I am assuming you really
>>> mean
>>> enable/disable filtering on those Loggers?
>>>
>>> Are you thinking that the rules can be added or removed dynamically? Or
>>> just the
>>> parameters to the rules? I ask because you can’t really add a Supplier
>>> dynamically
>>> if it is part of the configuration. Log4j will process the configuration
>>> only when it is
>>> changed. However, the data they rely on can come from anywhere - more or
>>> less
>>> how we use Lookups to evaluate conditions on every log event.
>>>
>>> Ralph
>>>
>>>> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>
>>>> This is still in the proof of concept phase, but what I am trying to do
>>> is
>>>> create a filter which allows developers to dynamically turn on/off
>>> loggers
>>>> based on some parameters without restarting their system. The parameters
>>>> can be varied, like:
>>>> 1. Turn on WARN log levels of a particular class.
>>>> 2. Turn on TRACE logging only on a particular host.
>>>> 3. Turn on DEBUG logs only for requests by a particular client for a
>>> short
>>>> duration,
>>>> ... and any combination of such parameters.
>>>>
>>>> The way I was thinking of doing this was creating a filter which takes
>>> in a
>>>> Supplier<Rule> and then evaluates a log event against that rule.
>>>>
>>>> On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
>>> wrote:
>>>>
>>>>> It sounds to me like you want to create a new type of plugin similar to
>>>>> PatternConerters. I am curious as to what your filter does. Is it
>>> something
>>>>> that would be useful to others so that we would be interested in
>>> including
>>>>> it?
>>>>>
>>>>> Ralph
>>>>>
>>>>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>>>>> theadvaitkumarsingh@gmail.com> wrote:
>>>>>>
>>>>>> I am creating a filter which requires the user to give a
>>> Supplier<Rule>
>>>>> to
>>>>>> fetch some custom rules. I am wondering how to go about getting this
>>>>>> Supplier from the configuration xml.
>>>>>>
>>>>>> One way I have thought of is using some sort of Registry and then just
>>>>>> asking the user to provide a key to lookup into the Registry to fetch
>>> the
>>>>>> Supplier. The user then registers his Supplier to this Registry in
>>> Java
>>>>>> code.
>>>>>> What are your thoughts on this and do you guys know of a better way
>>> to do
>>>>>> this?
>>>>>>
>>>>>> Thanks,
>>>>>> Adwait.
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
Re: How to inject a dependency into a custom filter.
Posted by Adwait Kumar Singh <th...@gmail.com>.
The problem I am stuck on is the best way to get the user to provide the
ruleSupplier via configuration.
On Thu, Oct 21, 2021 at 11:55 PM Adwait Kumar Singh <
theadvaitkumarsingh@gmail.com> wrote:
> Yes I mean filtering on those loggers.
>
> I don't expect the Supplier to be dynamically changed, only the rules they
> are fetching. Something roughly like this
>
> public class RuleBaseFilter extends AbstractFilter {
>
> //This will be initialized only once
> Supplier<List<Rules>> rulesSupplier;
>
>
> @Override
> public Result filter(final Logger logger, final Level level, final
> Marker marker, final Message msg,
> final Throwable t) {
> List<Rules> rules = rulesSupplier.get();
> //evaluate rules and filter
> }
>
> }
>
>
>
>
>
> On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> Sorry, you have lost me a little bit.
>>
>> When you say dynamically turn on/off loggers I am assuming you really
>> mean
>> enable/disable filtering on those Loggers?
>>
>> Are you thinking that the rules can be added or removed dynamically? Or
>> just the
>> parameters to the rules? I ask because you can’t really add a Supplier
>> dynamically
>> if it is part of the configuration. Log4j will process the configuration
>> only when it is
>> changed. However, the data they rely on can come from anywhere - more or
>> less
>> how we use Lookups to evaluate conditions on every log event.
>>
>> Ralph
>>
>> > On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
>> theadvaitkumarsingh@gmail.com> wrote:
>> >
>> > This is still in the proof of concept phase, but what I am trying to do
>> is
>> > create a filter which allows developers to dynamically turn on/off
>> loggers
>> > based on some parameters without restarting their system. The parameters
>> > can be varied, like:
>> > 1. Turn on WARN log levels of a particular class.
>> > 2. Turn on TRACE logging only on a particular host.
>> > 3. Turn on DEBUG logs only for requests by a particular client for a
>> short
>> > duration,
>> > ... and any combination of such parameters.
>> >
>> > The way I was thinking of doing this was creating a filter which takes
>> in a
>> > Supplier<Rule> and then evaluates a log event against that rule.
>> >
>> > On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
>> wrote:
>> >
>> >> It sounds to me like you want to create a new type of plugin similar to
>> >> PatternConerters. I am curious as to what your filter does. Is it
>> something
>> >> that would be useful to others so that we would be interested in
>> including
>> >> it?
>> >>
>> >> Ralph
>> >>
>> >>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>> >> theadvaitkumarsingh@gmail.com> wrote:
>> >>>
>> >>> I am creating a filter which requires the user to give a
>> Supplier<Rule>
>> >> to
>> >>> fetch some custom rules. I am wondering how to go about getting this
>> >>> Supplier from the configuration xml.
>> >>>
>> >>> One way I have thought of is using some sort of Registry and then just
>> >>> asking the user to provide a key to lookup into the Registry to fetch
>> the
>> >>> Supplier. The user then registers his Supplier to this Registry in
>> Java
>> >>> code.
>> >>> What are your thoughts on this and do you guys know of a better way
>> to do
>> >>> this?
>> >>>
>> >>> Thanks,
>> >>> Adwait.
>> >>
>> >>
>> >>
>>
>>
>>
Re: How to inject a dependency into a custom filter.
Posted by Adwait Kumar Singh <th...@gmail.com>.
Yes I mean filtering on those loggers.
I don't expect the Supplier to be dynamically changed, only the rules they
are fetching. Something roughly like this
public class RuleBaseFilter extends AbstractFilter {
//This will be initialized only once
Supplier<List<Rules>> rulesSupplier;
@Override
public Result filter(final Logger logger, final Level level, final Marker
marker, final Message msg,
final Throwable t) {
List<Rules> rules = rulesSupplier.get();
//evaluate rules and filter
}
}
On Thu, Oct 21, 2021 at 11:19 PM Ralph Goers <ra...@dslextreme.com>
wrote:
> Sorry, you have lost me a little bit.
>
> When you say dynamically turn on/off loggers I am assuming you really mean
> enable/disable filtering on those Loggers?
>
> Are you thinking that the rules can be added or removed dynamically? Or
> just the
> parameters to the rules? I ask because you can’t really add a Supplier
> dynamically
> if it is part of the configuration. Log4j will process the configuration
> only when it is
> changed. However, the data they rely on can come from anywhere - more or
> less
> how we use Lookups to evaluate conditions on every log event.
>
> Ralph
>
> > On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <
> theadvaitkumarsingh@gmail.com> wrote:
> >
> > This is still in the proof of concept phase, but what I am trying to do
> is
> > create a filter which allows developers to dynamically turn on/off
> loggers
> > based on some parameters without restarting their system. The parameters
> > can be varied, like:
> > 1. Turn on WARN log levels of a particular class.
> > 2. Turn on TRACE logging only on a particular host.
> > 3. Turn on DEBUG logs only for requests by a particular client for a
> short
> > duration,
> > ... and any combination of such parameters.
> >
> > The way I was thinking of doing this was creating a filter which takes
> in a
> > Supplier<Rule> and then evaluates a log event against that rule.
> >
> > On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com>
> wrote:
> >
> >> It sounds to me like you want to create a new type of plugin similar to
> >> PatternConerters. I am curious as to what your filter does. Is it
> something
> >> that would be useful to others so that we would be interested in
> including
> >> it?
> >>
> >> Ralph
> >>
> >>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
> >> theadvaitkumarsingh@gmail.com> wrote:
> >>>
> >>> I am creating a filter which requires the user to give a
> Supplier<Rule>
> >> to
> >>> fetch some custom rules. I am wondering how to go about getting this
> >>> Supplier from the configuration xml.
> >>>
> >>> One way I have thought of is using some sort of Registry and then just
> >>> asking the user to provide a key to lookup into the Registry to fetch
> the
> >>> Supplier. The user then registers his Supplier to this Registry in Java
> >>> code.
> >>> What are your thoughts on this and do you guys know of a better way to
> do
> >>> this?
> >>>
> >>> Thanks,
> >>> Adwait.
> >>
> >>
> >>
>
>
>
Re: How to inject a dependency into a custom filter.
Posted by Ralph Goers <ra...@dslextreme.com>.
Sorry, you have lost me a little bit.
When you say dynamically turn on/off loggers I am assuming you really mean
enable/disable filtering on those Loggers?
Are you thinking that the rules can be added or removed dynamically? Or just the
parameters to the rules? I ask because you can’t really add a Supplier dynamically
if it is part of the configuration. Log4j will process the configuration only when it is
changed. However, the data they rely on can come from anywhere - more or less
how we use Lookups to evaluate conditions on every log event.
Ralph
> On Oct 21, 2021, at 8:44 AM, Adwait Kumar Singh <th...@gmail.com> wrote:
>
> This is still in the proof of concept phase, but what I am trying to do is
> create a filter which allows developers to dynamically turn on/off loggers
> based on some parameters without restarting their system. The parameters
> can be varied, like:
> 1. Turn on WARN log levels of a particular class.
> 2. Turn on TRACE logging only on a particular host.
> 3. Turn on DEBUG logs only for requests by a particular client for a short
> duration,
> ... and any combination of such parameters.
>
> The way I was thinking of doing this was creating a filter which takes in a
> Supplier<Rule> and then evaluates a log event against that rule.
>
> On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com> wrote:
>
>> It sounds to me like you want to create a new type of plugin similar to
>> PatternConerters. I am curious as to what your filter does. Is it something
>> that would be useful to others so that we would be interested in including
>> it?
>>
>> Ralph
>>
>>> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
>> theadvaitkumarsingh@gmail.com> wrote:
>>>
>>> I am creating a filter which requires the user to give a Supplier<Rule>
>> to
>>> fetch some custom rules. I am wondering how to go about getting this
>>> Supplier from the configuration xml.
>>>
>>> One way I have thought of is using some sort of Registry and then just
>>> asking the user to provide a key to lookup into the Registry to fetch the
>>> Supplier. The user then registers his Supplier to this Registry in Java
>>> code.
>>> What are your thoughts on this and do you guys know of a better way to do
>>> this?
>>>
>>> Thanks,
>>> Adwait.
>>
>>
>>
Re: How to inject a dependency into a custom filter.
Posted by Adwait Kumar Singh <th...@gmail.com>.
This is still in the proof of concept phase, but what I am trying to do is
create a filter which allows developers to dynamically turn on/off loggers
based on some parameters without restarting their system. The parameters
can be varied, like:
1. Turn on WARN log levels of a particular class.
2. Turn on TRACE logging only on a particular host.
3. Turn on DEBUG logs only for requests by a particular client for a short
duration,
... and any combination of such parameters.
The way I was thinking of doing this was creating a filter which takes in a
Supplier<Rule> and then evaluates a log event against that rule.
On Thu, Oct 21, 2021 at 7:37 PM Apache <ra...@dslextreme.com> wrote:
> It sounds to me like you want to create a new type of plugin similar to
> PatternConerters. I am curious as to what your filter does. Is it something
> that would be useful to others so that we would be interested in including
> it?
>
> Ralph
>
> > On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <
> theadvaitkumarsingh@gmail.com> wrote:
> >
> > I am creating a filter which requires the user to give a Supplier<Rule>
> to
> > fetch some custom rules. I am wondering how to go about getting this
> > Supplier from the configuration xml.
> >
> > One way I have thought of is using some sort of Registry and then just
> > asking the user to provide a key to lookup into the Registry to fetch the
> > Supplier. The user then registers his Supplier to this Registry in Java
> > code.
> > What are your thoughts on this and do you guys know of a better way to do
> > this?
> >
> > Thanks,
> > Adwait.
>
>
>
Re: How to inject a dependency into a custom filter.
Posted by Apache <ra...@dslextreme.com>.
It sounds to me like you want to create a new type of plugin similar to PatternConerters. I am curious as to what your filter does. Is it something that would be useful to others so that we would be interested in including it?
Ralph
> On Oct 21, 2021, at 6:07 AM, Adwait Kumar Singh <th...@gmail.com> wrote:
>
> I am creating a filter which requires the user to give a Supplier<Rule> to
> fetch some custom rules. I am wondering how to go about getting this
> Supplier from the configuration xml.
>
> One way I have thought of is using some sort of Registry and then just
> asking the user to provide a key to lookup into the Registry to fetch the
> Supplier. The user then registers his Supplier to this Registry in Java
> code.
> What are your thoughts on this and do you guys know of a better way to do
> this?
>
> Thanks,
> Adwait.