You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Vi...@nokia.com on 2006/07/19 13:15:35 UTC
High performance filters
Hi all,
I'm building some kind of server and I use log4j for logging.
For troubleshooting issue I whant to be able to debug some few requests
only (based on matching criterias) so that it is possible to debug live
servers without killing performances.
I managed to use NDC and filters to do that and it does well. However,
filters are actually applied *after* log messages. Then using
NDC+filters solves only half the issue since the debug messages are
still created (even if not logged to disk). And because this is quite
CPU and RAM intensive it would drop the performances below acceptable
production-level.
Do you know if there is any way to use a filter-like mechanis at
log.isDebugEnabled() level for example? This way it would be possible to
not even build debug messages when the NDC does not match criterias,
thus saving a lot CPU and RAM usage and allowing debugging live servers.
Thanks all,
Vincent Bourdaraud.
Re: High performance filters
Posted by James Stauffer <st...@gmail.com>.
Correct. I thought you were concerned about the layout generating the
final message.
The only two options that come to mind do require changing the code
(changing the logger name to include the IP address or creating your
own isDebugEnabled that checks the IP in the NDC).
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> I believe this is not necessary. Using a NDC and a Filter impl (the log4j_sandbox NDCMatchFilter e.g.) would do the same if I'm correct.
> But still it does not prevent the application (that is the buildDebugMessage() call) from building the debug message, which is quite CPU intensive.
>
> >-----Original Message-----
> >From: ext James Stauffer [mailto:stauffer.james@gmail.com]
> >Sent: Wednesday, July 19, 2006 5:26 PM
> >To: Log4J Users List
> >Subject: Re: High performance filters
> >
> >Another option would be to write your own appender which
> >filters before creating the message and if it passes then
> >sends it to a normal appender.
> >
> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> ><Vi...@nokia.com> wrote:
> >> This is the kind of answer I was afraid of ;)
> >>
> >> But that is a very powerfull feature some of our (large)
> >customers are asking for very loudly: it gives both debug
> >messages and good-enough performance for live systems.
> >>
> >> Sure I could do it myself, but then that would not apply to already
> >> existing log4j-compliant code :(
> >>
> >> I'll ask to the dev mailing list, just in case...
> >>
> >> Thanks for your help,
> >> Vincent.
> >>
> >> >-----Original Message-----
> >> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
> >> >Sent: Wednesday, July 19, 2006 5:08 PM
> >> >To: Log4J Users List
> >> >Subject: Re: High performance filters
> >> >
> >> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> >> ><Vi...@nokia.com> wrote:
> >> >> I hope this is clear enough :)
> >> >
> >> >It is, and your problem looks very non-trivial :)
> >> >
> >> >To me, it looks like you need to implement your own
> >filtering before
> >> >invoking buildDebugMessage(), since to access the log4j
> >framework you
> >> >need to pass an object (whose construction is what you want to
> >> >avoid).
> >> >
> >> >--
> >> >Javier González Nicolini
> >> >
> >>
> >>---------------------------------------------------------------------
> >> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >> >
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >> For additional commands, e-mail: log4j-user-help@logging.apache.org
> >>
> >>
> >
> >
> >--
> >James Stauffer
> >Are you good? Take the test at http://www.livingwaters.com/good/
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
--
James Stauffer
Are you good? Take the test at http://www.livingwaters.com/good/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
I believe this is not necessary. Using a NDC and a Filter impl (the log4j_sandbox NDCMatchFilter e.g.) would do the same if I'm correct.
But still it does not prevent the application (that is the buildDebugMessage() call) from building the debug message, which is quite CPU intensive.
>-----Original Message-----
>From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>Sent: Wednesday, July 19, 2006 5:26 PM
>To: Log4J Users List
>Subject: Re: High performance filters
>
>Another option would be to write your own appender which
>filters before creating the message and if it passes then
>sends it to a normal appender.
>
>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com> wrote:
>> This is the kind of answer I was afraid of ;)
>>
>> But that is a very powerfull feature some of our (large)
>customers are asking for very loudly: it gives both debug
>messages and good-enough performance for live systems.
>>
>> Sure I could do it myself, but then that would not apply to already
>> existing log4j-compliant code :(
>>
>> I'll ask to the dev mailing list, just in case...
>>
>> Thanks for your help,
>> Vincent.
>>
>> >-----Original Message-----
>> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>> >Sent: Wednesday, July 19, 2006 5:08 PM
>> >To: Log4J Users List
>> >Subject: Re: High performance filters
>> >
>> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>> ><Vi...@nokia.com> wrote:
>> >> I hope this is clear enough :)
>> >
>> >It is, and your problem looks very non-trivial :)
>> >
>> >To me, it looks like you need to implement your own
>filtering before
>> >invoking buildDebugMessage(), since to access the log4j
>framework you
>> >need to pass an object (whose construction is what you want to
>> >avoid).
>> >
>> >--
>> >Javier González Nicolini
>> >
>>
>>---------------------------------------------------------------------
>> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> >For additional commands, e-mail: log4j-user-help@logging.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
>>
>
>
>--
>James Stauffer
>Are you good? Take the test at http://www.livingwaters.com/good/
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: High performance filters
Posted by James Stauffer <st...@gmail.com>.
Another option would be to write your own appender which filters
before creating the message and if it passes then sends it to a normal
appender.
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> This is the kind of answer I was afraid of ;)
>
> But that is a very powerfull feature some of our (large) customers are asking for very loudly: it gives both debug messages and good-enough performance for live systems.
>
> Sure I could do it myself, but then that would not apply to already existing log4j-compliant code :(
>
> I'll ask to the dev mailing list, just in case...
>
> Thanks for your help,
> Vincent.
>
> >-----Original Message-----
> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
> >Sent: Wednesday, July 19, 2006 5:08 PM
> >To: Log4J Users List
> >Subject: Re: High performance filters
> >
> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> ><Vi...@nokia.com> wrote:
> >> I hope this is clear enough :)
> >
> >It is, and your problem looks very non-trivial :)
> >
> >To me, it looks like you need to implement your own filtering
> >before invoking buildDebugMessage(), since to access the log4j
> >framework you need to pass an object (whose construction is
> >what you want to avoid).
> >
> >--
> >Javier González Nicolini
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
--
James Stauffer
Are you good? Take the test at http://www.livingwaters.com/good/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
This is the kind of answer I was afraid of ;)
But that is a very powerfull feature some of our (large) customers are asking for very loudly: it gives both debug messages and good-enough performance for live systems.
Sure I could do it myself, but then that would not apply to already existing log4j-compliant code :(
I'll ask to the dev mailing list, just in case...
Thanks for your help,
Vincent.
>-----Original Message-----
>From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>Sent: Wednesday, July 19, 2006 5:08 PM
>To: Log4J Users List
>Subject: Re: High performance filters
>
>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com> wrote:
>> I hope this is clear enough :)
>
>It is, and your problem looks very non-trivial :)
>
>To me, it looks like you need to implement your own filtering
>before invoking buildDebugMessage(), since to access the log4j
>framework you need to pass an object (whose construction is
>what you want to avoid).
>
>--
>Javier González Nicolini
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: High performance filters
Posted by Javier Gonzalez <ja...@gmail.com>.
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> I hope this is clear enough :)
It is, and your problem looks very non-trivial :)
To me, it looks like you need to implement your own filtering before
invoking buildDebugMessage(), since to access the log4j framework you
need to pass an object (whose construction is what you want to avoid).
--
Javier González Nicolini
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
I tried and this solution works like a charm!
Log4j designers are really clever guys :)
>-----Original Message-----
>From: ext
>log4j-user-return-18513-Vincent.Bourdaraud=nokia.com@logging.ap
>ache.org
>[mailto:log4j-user-return-18513-Vincent.Bourdaraud=nokia.com@lo
>gging.apache.org]
>Sent: Wednesday, July 19, 2006 6:26 PM
>To: log4j-user@logging.apache.org
>Subject: RE: High performance filters
>
>After an extra look at log4j source I wonder if I've found
>some kind of solution:
>Logger contains the following signature:
>log.debug(Object message)
>
>Here this is an "Ojbject" and not a "String".
>
>If I create some class in charge of building the debug message
>only when its toString() method is called and I call
>log.debug() with some instance of that class, then message
>creation is payed only when log4j actually builds the message.
>And I believe this to append after filtering if filtering
>allowed the message to be logged. If I'm correct, this would
>be a neat solution.
>
>What do you guys think about it?
>
>>-----Original Message-----
>>From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>>Sent: Wednesday, July 19, 2006 5:44 PM
>>To: Log4J Users List
>>Subject: Re: High performance filters
>>
>>It would really only work with 1 extra criteria.
>>I don't know the impact of many loggers.
>>
>>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com>
>>wrote:
>>> That's an interresting work arround.
>>> Still that is tedious when several criterias are needed.
>>>
>>> Since we are talking about a server, that would create a
>>huge number of loggers. Do you have any idea about performance impact?
>>>
>>> >-----Original Message-----
>>> >From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>>> >Sent: Wednesday, July 19, 2006 5:22 PM
>>> >To: Log4J Users List
>>> >Subject: Re: High performance filters
>>> >
>>> >One option would be to include to the IP address in the
>logger name.
>>> >Logger log = Logger.getLogger(getClass().getName() + "." +
>>> >request.getSourceAddress());
>>> >
>>> >That would create many more loggers but would allow
>>> >log.isDebugEnabled() to work correctly.
>>> >
>>> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>>> ><Vi...@nokia.com> wrote:
>>> >> I'm not sure your proposal solves my problem.
>>> >> I'm sorry if my question was not clear enough. Let me try
>>> >another way:
>>> >>
>>> >> Considering the following code snippet
>>> >>
>>> >> // the following method is called-back hundred of time //
>>per second.
>>> >> public void process(Request request, Response response) {
>>> >> NDC.push("source-ip=" + request.getSourceAddress());
>>> >> if (log.isDebugEnabled())
>>> >> {
>>> >> log.debug(buildDebugMessage(request));
>>> >> }
>>> >> // process request and send response ...
>>> >> NDCP.pop();
>>> >> }
>>> >>
>>> >> I need that log.debug(buildDebugMessage(request)) is called
>>> >only if the logger is in debug mode AND the source-ip
>>within the NDC
>>> >is 192.168.0.12 (this source cause some trouble to the
>>server in this
>>> >example), so that it would be possible to turn live servers
>>in debug
>>> >mode for some (problematic) context and investigate directly in
>>> >production without killing performances.
>>> >>
>>> >> The NDC+Filtering method does not help that much here. It
>>> >does prevent messages from beeing wrote if they don't match the
>>> >expected context (source IP 192.168.0.12 in my example),
>>but it does
>>> >not prevent the debug message from being built (the
>>> >buildDebugMessage(request) call) which is still quite
>>expensive (and
>>> >useless in this context).
>>> >>
>>> >> I hope this is clear enough :)
>>> >>
>>> >> Thanks a lot for your time!
>>> >> Vincent.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> >-----Original Message-----
>>> >> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>>> >> >Sent: Wednesday, July 19, 2006 3:44 PM
>>> >> >To: Log4J Users List
>>> >> >Subject: Re: High performance filters
>>> >> >
>>> >> >Create separate Loggers for the debug requests, and leave
>>> >only those
>>> >> >loggers at debug level. Then prepend log.debug() calls with
>>> >> >if(log.isDebugEnabled()), so that the debug messages
>>will only be
>>> >> >created if the particular logger is debug enabled.
>>> >> >
>>> >> >(I'm not sure I understood your question correctly, though)
>>> >> >
>>> >> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>>> >> ><Vi...@nokia.com> wrote:
>>> >> >> Hi all,
>>> >> >>
>>> >> >> I'm building some kind of server and I use log4j for logging.
>>> >> >> For troubleshooting issue I whant to be able to debug
>some few
>>> >> >> requests only (based on matching criterias) so that it is
>>> >> >possible to
>>> >> >> debug live servers without killing performances.
>>> >> >>
>>> >> >> I managed to use NDC and filters to do that and it does
>>> >> >well. However,
>>> >> >> filters are actually applied *after* log messages. Then using
>>> >> >> NDC+filters solves only half the issue since the debug
>>> >messages are
>>> >> >> still created (even if not logged to disk). And because this
>>> >> >is quite
>>> >> >> CPU and RAM intensive it would drop the performances below
>>> >> >acceptable
>>> >> >> production-level.
>>> >> >>
>>> >> >> Do you know if there is any way to use a filter-like
>>mechanis at
>>> >> >> log.isDebugEnabled() level for example? This way it would be
>>> >> >possible
>>> >> >> to not even build debug messages when the NDC does not match
>>> >> >> criterias, thus saving a lot CPU and RAM usage and allowing
>>> >> >debugging live servers.
>>> >> >>
>>> >> >> Thanks all,
>>> >> >> Vincent Bourdaraud.
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> >--
>>> >> >Javier González Nicolini
>>> >> >
>>> >>
>>>
>>>>--------------------------------------------------------------------
>>> >>-
>>> >> >To unsubscribe, e-mail:
>log4j-user-unsubscribe@logging.apache.org
>>> >> >For additional commands, e-mail:
>>> >> >log4j-user-help@logging.apache.org
>>> >> >
>>> >> >
>>> >>
>>> >>
>>-------------------------------------------------------------------
>>> >> -- To unsubscribe, e-mail:
>>> >> log4j-user-unsubscribe@logging.apache.org
>>> >> For additional commands, e-mail:
>>log4j-user-help@logging.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> >--
>>> >James Stauffer
>>> >Are you good? Take the test at http://www.livingwaters.com/good/
>>> >
>>>
>>>---------------------------------------------------------------------
>>> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>> >For additional commands, e-mail: log4j-user-help@logging.apache.org
>>> >
>>> >
>>>
>>>
>---------------------------------------------------------------------
>>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>>
>>>
>>
>>
>>--
>>James Stauffer
>>Are you good? Take the test at http://www.livingwaters.com/good/
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>>For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
After an extra look at log4j source I wonder if I've found some kind of solution:
Logger contains the following signature:
log.debug(Object message)
Here this is an "Ojbject" and not a "String".
If I create some class in charge of building the debug message only when its toString() method is called and I call log.debug() with some instance of that class, then message creation is payed only when log4j actually builds the message. And I believe this to append after filtering if filtering allowed the message to be logged. If I'm correct, this would be a neat solution.
What do you guys think about it?
>-----Original Message-----
>From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>Sent: Wednesday, July 19, 2006 5:44 PM
>To: Log4J Users List
>Subject: Re: High performance filters
>
>It would really only work with 1 extra criteria.
>I don't know the impact of many loggers.
>
>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com> wrote:
>> That's an interresting work arround.
>> Still that is tedious when several criterias are needed.
>>
>> Since we are talking about a server, that would create a
>huge number of loggers. Do you have any idea about performance impact?
>>
>> >-----Original Message-----
>> >From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>> >Sent: Wednesday, July 19, 2006 5:22 PM
>> >To: Log4J Users List
>> >Subject: Re: High performance filters
>> >
>> >One option would be to include to the IP address in the logger name.
>> >Logger log = Logger.getLogger(getClass().getName() + "." +
>> >request.getSourceAddress());
>> >
>> >That would create many more loggers but would allow
>> >log.isDebugEnabled() to work correctly.
>> >
>> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>> ><Vi...@nokia.com> wrote:
>> >> I'm not sure your proposal solves my problem.
>> >> I'm sorry if my question was not clear enough. Let me try
>> >another way:
>> >>
>> >> Considering the following code snippet
>> >>
>> >> // the following method is called-back hundred of time //
>per second.
>> >> public void process(Request request, Response response) {
>> >> NDC.push("source-ip=" + request.getSourceAddress());
>> >> if (log.isDebugEnabled())
>> >> {
>> >> log.debug(buildDebugMessage(request));
>> >> }
>> >> // process request and send response ...
>> >> NDCP.pop();
>> >> }
>> >>
>> >> I need that log.debug(buildDebugMessage(request)) is called
>> >only if the logger is in debug mode AND the source-ip
>within the NDC
>> >is 192.168.0.12 (this source cause some trouble to the
>server in this
>> >example), so that it would be possible to turn live servers
>in debug
>> >mode for some (problematic) context and investigate directly in
>> >production without killing performances.
>> >>
>> >> The NDC+Filtering method does not help that much here. It
>> >does prevent messages from beeing wrote if they don't match the
>> >expected context (source IP 192.168.0.12 in my example),
>but it does
>> >not prevent the debug message from being built (the
>> >buildDebugMessage(request) call) which is still quite
>expensive (and
>> >useless in this context).
>> >>
>> >> I hope this is clear enough :)
>> >>
>> >> Thanks a lot for your time!
>> >> Vincent.
>> >>
>> >>
>> >>
>> >>
>> >> >-----Original Message-----
>> >> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>> >> >Sent: Wednesday, July 19, 2006 3:44 PM
>> >> >To: Log4J Users List
>> >> >Subject: Re: High performance filters
>> >> >
>> >> >Create separate Loggers for the debug requests, and leave
>> >only those
>> >> >loggers at debug level. Then prepend log.debug() calls with
>> >> >if(log.isDebugEnabled()), so that the debug messages
>will only be
>> >> >created if the particular logger is debug enabled.
>> >> >
>> >> >(I'm not sure I understood your question correctly, though)
>> >> >
>> >> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>> >> ><Vi...@nokia.com> wrote:
>> >> >> Hi all,
>> >> >>
>> >> >> I'm building some kind of server and I use log4j for logging.
>> >> >> For troubleshooting issue I whant to be able to debug some few
>> >> >> requests only (based on matching criterias) so that it is
>> >> >possible to
>> >> >> debug live servers without killing performances.
>> >> >>
>> >> >> I managed to use NDC and filters to do that and it does
>> >> >well. However,
>> >> >> filters are actually applied *after* log messages. Then using
>> >> >> NDC+filters solves only half the issue since the debug
>> >messages are
>> >> >> still created (even if not logged to disk). And because this
>> >> >is quite
>> >> >> CPU and RAM intensive it would drop the performances below
>> >> >acceptable
>> >> >> production-level.
>> >> >>
>> >> >> Do you know if there is any way to use a filter-like
>mechanis at
>> >> >> log.isDebugEnabled() level for example? This way it would be
>> >> >possible
>> >> >> to not even build debug messages when the NDC does not match
>> >> >> criterias, thus saving a lot CPU and RAM usage and allowing
>> >> >debugging live servers.
>> >> >>
>> >> >> Thanks all,
>> >> >> Vincent Bourdaraud.
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >--
>> >> >Javier González Nicolini
>> >> >
>> >>
>>
>>>--------------------------------------------------------------------
>> >>-
>> >> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> >> >For additional commands, e-mail:
>> >> >log4j-user-help@logging.apache.org
>> >> >
>> >> >
>> >>
>> >>
>-------------------------------------------------------------------
>> >> -- To unsubscribe, e-mail:
>> >> log4j-user-unsubscribe@logging.apache.org
>> >> For additional commands, e-mail:
>log4j-user-help@logging.apache.org
>> >>
>> >>
>> >
>> >
>> >--
>> >James Stauffer
>> >Are you good? Take the test at http://www.livingwaters.com/good/
>> >
>>
>>---------------------------------------------------------------------
>> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> >For additional commands, e-mail: log4j-user-help@logging.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
>>
>
>
>--
>James Stauffer
>Are you good? Take the test at http://www.livingwaters.com/good/
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: High performance filters
Posted by James Stauffer <st...@gmail.com>.
It would really only work with 1 extra criteria.
I don't know the impact of many loggers.
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> That's an interresting work arround.
> Still that is tedious when several criterias are needed.
>
> Since we are talking about a server, that would create a huge number of loggers. Do you have any idea about performance impact?
>
> >-----Original Message-----
> >From: ext James Stauffer [mailto:stauffer.james@gmail.com]
> >Sent: Wednesday, July 19, 2006 5:22 PM
> >To: Log4J Users List
> >Subject: Re: High performance filters
> >
> >One option would be to include to the IP address in the logger name.
> >Logger log = Logger.getLogger(getClass().getName() + "." +
> >request.getSourceAddress());
> >
> >That would create many more loggers but would allow
> >log.isDebugEnabled() to work correctly.
> >
> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> ><Vi...@nokia.com> wrote:
> >> I'm not sure your proposal solves my problem.
> >> I'm sorry if my question was not clear enough. Let me try
> >another way:
> >>
> >> Considering the following code snippet
> >>
> >> // the following method is called-back hundred of time // per second.
> >> public void process(Request request, Response response) {
> >> NDC.push("source-ip=" + request.getSourceAddress());
> >> if (log.isDebugEnabled())
> >> {
> >> log.debug(buildDebugMessage(request));
> >> }
> >> // process request and send response ...
> >> NDCP.pop();
> >> }
> >>
> >> I need that log.debug(buildDebugMessage(request)) is called
> >only if the logger is in debug mode AND the source-ip within
> >the NDC is 192.168.0.12 (this source cause some trouble to the
> >server in this example), so that it would be possible to turn
> >live servers in debug mode for some (problematic) context and
> >investigate directly in production without killing performances.
> >>
> >> The NDC+Filtering method does not help that much here. It
> >does prevent messages from beeing wrote if they don't match
> >the expected context (source IP 192.168.0.12 in my example),
> >but it does not prevent the debug message from being built
> >(the buildDebugMessage(request) call) which is still quite
> >expensive (and useless in this context).
> >>
> >> I hope this is clear enough :)
> >>
> >> Thanks a lot for your time!
> >> Vincent.
> >>
> >>
> >>
> >>
> >> >-----Original Message-----
> >> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
> >> >Sent: Wednesday, July 19, 2006 3:44 PM
> >> >To: Log4J Users List
> >> >Subject: Re: High performance filters
> >> >
> >> >Create separate Loggers for the debug requests, and leave
> >only those
> >> >loggers at debug level. Then prepend log.debug() calls with
> >> >if(log.isDebugEnabled()), so that the debug messages will only be
> >> >created if the particular logger is debug enabled.
> >> >
> >> >(I'm not sure I understood your question correctly, though)
> >> >
> >> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> >> ><Vi...@nokia.com> wrote:
> >> >> Hi all,
> >> >>
> >> >> I'm building some kind of server and I use log4j for logging.
> >> >> For troubleshooting issue I whant to be able to debug some few
> >> >> requests only (based on matching criterias) so that it is
> >> >possible to
> >> >> debug live servers without killing performances.
> >> >>
> >> >> I managed to use NDC and filters to do that and it does
> >> >well. However,
> >> >> filters are actually applied *after* log messages. Then using
> >> >> NDC+filters solves only half the issue since the debug
> >messages are
> >> >> still created (even if not logged to disk). And because this
> >> >is quite
> >> >> CPU and RAM intensive it would drop the performances below
> >> >acceptable
> >> >> production-level.
> >> >>
> >> >> Do you know if there is any way to use a filter-like mechanis at
> >> >> log.isDebugEnabled() level for example? This way it would be
> >> >possible
> >> >> to not even build debug messages when the NDC does not match
> >> >> criterias, thus saving a lot CPU and RAM usage and allowing
> >> >debugging live servers.
> >> >>
> >> >> Thanks all,
> >> >> Vincent Bourdaraud.
> >> >>
> >> >>
> >> >
> >> >
> >> >--
> >> >Javier González Nicolini
> >> >
> >>
> >>---------------------------------------------------------------------
> >> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >> >
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >> For additional commands, e-mail: log4j-user-help@logging.apache.org
> >>
> >>
> >
> >
> >--
> >James Stauffer
> >Are you good? Take the test at http://www.livingwaters.com/good/
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
--
James Stauffer
Are you good? Take the test at http://www.livingwaters.com/good/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
That's an interresting work arround.
Still that is tedious when several criterias are needed.
Since we are talking about a server, that would create a huge number of loggers. Do you have any idea about performance impact?
>-----Original Message-----
>From: ext James Stauffer [mailto:stauffer.james@gmail.com]
>Sent: Wednesday, July 19, 2006 5:22 PM
>To: Log4J Users List
>Subject: Re: High performance filters
>
>One option would be to include to the IP address in the logger name.
>Logger log = Logger.getLogger(getClass().getName() + "." +
>request.getSourceAddress());
>
>That would create many more loggers but would allow
>log.isDebugEnabled() to work correctly.
>
>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com> wrote:
>> I'm not sure your proposal solves my problem.
>> I'm sorry if my question was not clear enough. Let me try
>another way:
>>
>> Considering the following code snippet
>>
>> // the following method is called-back hundred of time // per second.
>> public void process(Request request, Response response) {
>> NDC.push("source-ip=" + request.getSourceAddress());
>> if (log.isDebugEnabled())
>> {
>> log.debug(buildDebugMessage(request));
>> }
>> // process request and send response ...
>> NDCP.pop();
>> }
>>
>> I need that log.debug(buildDebugMessage(request)) is called
>only if the logger is in debug mode AND the source-ip within
>the NDC is 192.168.0.12 (this source cause some trouble to the
>server in this example), so that it would be possible to turn
>live servers in debug mode for some (problematic) context and
>investigate directly in production without killing performances.
>>
>> The NDC+Filtering method does not help that much here. It
>does prevent messages from beeing wrote if they don't match
>the expected context (source IP 192.168.0.12 in my example),
>but it does not prevent the debug message from being built
>(the buildDebugMessage(request) call) which is still quite
>expensive (and useless in this context).
>>
>> I hope this is clear enough :)
>>
>> Thanks a lot for your time!
>> Vincent.
>>
>>
>>
>>
>> >-----Original Message-----
>> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>> >Sent: Wednesday, July 19, 2006 3:44 PM
>> >To: Log4J Users List
>> >Subject: Re: High performance filters
>> >
>> >Create separate Loggers for the debug requests, and leave
>only those
>> >loggers at debug level. Then prepend log.debug() calls with
>> >if(log.isDebugEnabled()), so that the debug messages will only be
>> >created if the particular logger is debug enabled.
>> >
>> >(I'm not sure I understood your question correctly, though)
>> >
>> >On 7/19/06, Vincent.Bourdaraud@nokia.com
>> ><Vi...@nokia.com> wrote:
>> >> Hi all,
>> >>
>> >> I'm building some kind of server and I use log4j for logging.
>> >> For troubleshooting issue I whant to be able to debug some few
>> >> requests only (based on matching criterias) so that it is
>> >possible to
>> >> debug live servers without killing performances.
>> >>
>> >> I managed to use NDC and filters to do that and it does
>> >well. However,
>> >> filters are actually applied *after* log messages. Then using
>> >> NDC+filters solves only half the issue since the debug
>messages are
>> >> still created (even if not logged to disk). And because this
>> >is quite
>> >> CPU and RAM intensive it would drop the performances below
>> >acceptable
>> >> production-level.
>> >>
>> >> Do you know if there is any way to use a filter-like mechanis at
>> >> log.isDebugEnabled() level for example? This way it would be
>> >possible
>> >> to not even build debug messages when the NDC does not match
>> >> criterias, thus saving a lot CPU and RAM usage and allowing
>> >debugging live servers.
>> >>
>> >> Thanks all,
>> >> Vincent Bourdaraud.
>> >>
>> >>
>> >
>> >
>> >--
>> >Javier González Nicolini
>> >
>>
>>---------------------------------------------------------------------
>> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> >For additional commands, e-mail: log4j-user-help@logging.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>> For additional commands, e-mail: log4j-user-help@logging.apache.org
>>
>>
>
>
>--
>James Stauffer
>Are you good? Take the test at http://www.livingwaters.com/good/
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: High performance filters
Posted by James Stauffer <st...@gmail.com>.
One option would be to include to the IP address in the logger name.
Logger log = Logger.getLogger(getClass().getName() + "." +
request.getSourceAddress());
That would create many more loggers but would allow
log.isDebugEnabled() to work correctly.
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> I'm not sure your proposal solves my problem.
> I'm sorry if my question was not clear enough. Let me try another way:
>
> Considering the following code snippet
>
> // the following method is called-back hundred of time
> // per second.
> public void process(Request request, Response response)
> {
> NDC.push("source-ip=" + request.getSourceAddress());
> if (log.isDebugEnabled())
> {
> log.debug(buildDebugMessage(request));
> }
> // process request and send response ...
> NDCP.pop();
> }
>
> I need that log.debug(buildDebugMessage(request)) is called only if the logger is in debug mode AND the source-ip within the NDC is 192.168.0.12 (this source cause some trouble to the server in this example), so that it would be possible to turn live servers in debug mode for some (problematic) context and investigate directly in production without killing performances.
>
> The NDC+Filtering method does not help that much here. It does prevent messages from beeing wrote if they don't match the expected context (source IP 192.168.0.12 in my example), but it does not prevent the debug message from being built (the buildDebugMessage(request) call) which is still quite expensive (and useless in this context).
>
> I hope this is clear enough :)
>
> Thanks a lot for your time!
> Vincent.
>
>
>
>
> >-----Original Message-----
> >From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
> >Sent: Wednesday, July 19, 2006 3:44 PM
> >To: Log4J Users List
> >Subject: Re: High performance filters
> >
> >Create separate Loggers for the debug requests, and leave only
> >those loggers at debug level. Then prepend log.debug() calls
> >with if(log.isDebugEnabled()), so that the debug messages will
> >only be created if the particular logger is debug enabled.
> >
> >(I'm not sure I understood your question correctly, though)
> >
> >On 7/19/06, Vincent.Bourdaraud@nokia.com
> ><Vi...@nokia.com> wrote:
> >> Hi all,
> >>
> >> I'm building some kind of server and I use log4j for logging.
> >> For troubleshooting issue I whant to be able to debug some few
> >> requests only (based on matching criterias) so that it is
> >possible to
> >> debug live servers without killing performances.
> >>
> >> I managed to use NDC and filters to do that and it does
> >well. However,
> >> filters are actually applied *after* log messages. Then using
> >> NDC+filters solves only half the issue since the debug messages are
> >> still created (even if not logged to disk). And because this
> >is quite
> >> CPU and RAM intensive it would drop the performances below
> >acceptable
> >> production-level.
> >>
> >> Do you know if there is any way to use a filter-like mechanis at
> >> log.isDebugEnabled() level for example? This way it would be
> >possible
> >> to not even build debug messages when the NDC does not match
> >> criterias, thus saving a lot CPU and RAM usage and allowing
> >debugging live servers.
> >>
> >> Thanks all,
> >> Vincent Bourdaraud.
> >>
> >>
> >
> >
> >--
> >Javier González Nicolini
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> >For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
--
James Stauffer
Are you good? Take the test at http://www.livingwaters.com/good/
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
RE: High performance filters
Posted by Vi...@nokia.com.
I'm not sure your proposal solves my problem.
I'm sorry if my question was not clear enough. Let me try another way:
Considering the following code snippet
// the following method is called-back hundred of time
// per second.
public void process(Request request, Response response)
{
NDC.push("source-ip=" + request.getSourceAddress());
if (log.isDebugEnabled())
{
log.debug(buildDebugMessage(request));
}
// process request and send response ...
NDCP.pop();
}
I need that log.debug(buildDebugMessage(request)) is called only if the logger is in debug mode AND the source-ip within the NDC is 192.168.0.12 (this source cause some trouble to the server in this example), so that it would be possible to turn live servers in debug mode for some (problematic) context and investigate directly in production without killing performances.
The NDC+Filtering method does not help that much here. It does prevent messages from beeing wrote if they don't match the expected context (source IP 192.168.0.12 in my example), but it does not prevent the debug message from being built (the buildDebugMessage(request) call) which is still quite expensive (and useless in this context).
I hope this is clear enough :)
Thanks a lot for your time!
Vincent.
>-----Original Message-----
>From: ext Javier Gonzalez [mailto:jagonzal@gmail.com]
>Sent: Wednesday, July 19, 2006 3:44 PM
>To: Log4J Users List
>Subject: Re: High performance filters
>
>Create separate Loggers for the debug requests, and leave only
>those loggers at debug level. Then prepend log.debug() calls
>with if(log.isDebugEnabled()), so that the debug messages will
>only be created if the particular logger is debug enabled.
>
>(I'm not sure I understood your question correctly, though)
>
>On 7/19/06, Vincent.Bourdaraud@nokia.com
><Vi...@nokia.com> wrote:
>> Hi all,
>>
>> I'm building some kind of server and I use log4j for logging.
>> For troubleshooting issue I whant to be able to debug some few
>> requests only (based on matching criterias) so that it is
>possible to
>> debug live servers without killing performances.
>>
>> I managed to use NDC and filters to do that and it does
>well. However,
>> filters are actually applied *after* log messages. Then using
>> NDC+filters solves only half the issue since the debug messages are
>> still created (even if not logged to disk). And because this
>is quite
>> CPU and RAM intensive it would drop the performances below
>acceptable
>> production-level.
>>
>> Do you know if there is any way to use a filter-like mechanis at
>> log.isDebugEnabled() level for example? This way it would be
>possible
>> to not even build debug messages when the NDC does not match
>> criterias, thus saving a lot CPU and RAM usage and allowing
>debugging live servers.
>>
>> Thanks all,
>> Vincent Bourdaraud.
>>
>>
>
>
>--
>Javier González Nicolini
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
>For additional commands, e-mail: log4j-user-help@logging.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org
Re: High performance filters
Posted by Javier Gonzalez <ja...@gmail.com>.
Create separate Loggers for the debug requests, and leave only those
loggers at debug level. Then prepend log.debug() calls with
if(log.isDebugEnabled()), so that the debug messages will only be
created if the particular logger is debug enabled.
(I'm not sure I understood your question correctly, though)
On 7/19/06, Vincent.Bourdaraud@nokia.com <Vi...@nokia.com> wrote:
> Hi all,
>
> I'm building some kind of server and I use log4j for logging.
> For troubleshooting issue I whant to be able to debug some few requests
> only (based on matching criterias) so that it is possible to debug live
> servers without killing performances.
>
> I managed to use NDC and filters to do that and it does well. However,
> filters are actually applied *after* log messages. Then using
> NDC+filters solves only half the issue since the debug messages are
> still created (even if not logged to disk). And because this is quite
> CPU and RAM intensive it would drop the performances below acceptable
> production-level.
>
> Do you know if there is any way to use a filter-like mechanis at
> log.isDebugEnabled() level for example? This way it would be possible to
> not even build debug messages when the NDC does not match criterias,
> thus saving a lot CPU and RAM usage and allowing debugging live servers.
>
> Thanks all,
> Vincent Bourdaraud.
>
>
--
Javier González Nicolini
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-user-help@logging.apache.org