You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Leo Donahue - PLANDEVX <Le...@mail.maricopa.gov> on 2011/03/29 18:57:50 UTC
Logging request parameters - Filter vs Servlet
Where do you initialize the Logger (Filter or Servlet)?
The servlet 2.5 spec says you can use filters for logging, but since I'm not modifying the request or response, is logging from within a filter the right approach to logging request parameters?
Using CATALINA_BASE: "C:\ApacheTomcat\apache-tomcat-6.0.32"
Using CATALINA_HOME: "C:\ApacheTomcat\apache-tomcat-6.0.32"
Using CATALINA_TMPDIR: "C:\ApacheTomcat\apache-tomcat-6.0.32\temp"
Using JRE_HOME: "C:\jdk1.6.0_24"
Using CLASSPATH: "C:\ApacheTomcat\apache-tomcat-6.0.32\bin\bootstrap.jar"
Server version: Apache Tomcat/6.0.32
Server built: February 2 2011 2003
Server number: 6.0.32.0
OS Name: Windows XP
OS Version: 5.1
Architecture: x86
JVM Version: 1.6.0_24-b07
JVM Vendor: Sun Microsystems Inc.
Leo
Re: Logging request parameters - Filter vs Servlet
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Leon,
On 3/30/2011 3:36 PM, Leo Donahue - PLANDEVX wrote:
> I'm going this route for now:
>
> FacesContext.getCurrentInstance().getExternalContext().log(the string buffer of the parcel listbox);
> FacesContext.getCurrentInstance().getExternalContext().log(the map URL);
Might I suggest that you try to cram all that information onto a single
line and replace any CR or LF characters with a space or something? If
you want to post-process the log, having each even on a single line will
be sooo much easier than writing a script just to piece everything back
together again.
Also, in high-volume situations, you may get logs from one request
interleaved with those of another, like this:
> Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log
> INFO: PARCEL = '125-27-089'
> Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log
> INFO: PARCEL = '456-67-890'
> Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log
> INFO:
http://planning.maricopa.gov/agsoutput/_ags_map08eb57df58224e1884e17a3e8a59b555.jpg
> Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log
> INFO:
http://planning.maricopa.gov/agsoutput/_ags_map738249178234b78b09213aba07f78923.jpg
Good luck,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2U2bcACgkQ9CaO5/Lv0PAvAwCfdiGr2HewlXGxzB05uvQwnIj6
8SAAn3IeKzFMBlwmPnDejpOs1buN87tS
=QUb7
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Posted by Leo Donahue - PLANDEVX <Le...@mail.maricopa.gov>.
Chris,
Thanks for the feedback.. very much appreciated!
>-----Original Message-----
>From: Christopher Schultz [mailto:chris@christopherschultz.net]
>Subject: Re: Logging request parameters - Filter vs Servlet
>
>So you need to log the "txtAPN" parameter, right?
Yes. Actually, the contents of the listbox... as you'll see below.
>> Those are the parameters I'm trying to capture, along with the map image url.
>> A. We'd like to know how many requests actually generated a map image.
>
>Can you tell that, just from a parameter value? I would think that the
>URL itself would more likely tell you if a map was generated.
No, I couldn't tell that from a parameter value, which is why I knew I had to log from within the webapp somewhere, but I didn't know whether logging should happen in a filter or at a specific place in the model. Or should I log in the "fascade" (pattern)?
I wanted the logs to capture the url so that I knew the model reached a point where it generated an image.
http://planning.maricopa.gov/agsoutput/_ags_mapdee1a8d3a28f49a48f44aaf4dc4cc316.jpg
Those images are only there for 20 min, so that url is already dead.
>> B. We'd like to know whether this app is searching for parcels
>> primarily in the unincorporated areas of the County, or parcels
>> located within a city jurisdiction. That part I can figure out once
>> I know which parcels people are searching.
>
>Simply logging the parcels used in searches would allow you to do that,
>as you've said. The RequestDumperValve logs a /ton/ of information, and
>probably wouldn't get you what you want.
>
>I'm not sure where you read it, but that sounds like a platitude applied
>as a blanket admonition not to log in your webapp. If it's appropriate
>for your situation, then feel free to do it.
>
>Back to the original "filter" question: logging using a filter is
>perfectly acceptable if it's the right solution (see below for questions
>that might lead you to other options). Writing the filter is trivial:
>just implement the javax.servlet.Filter interface and be sure to read
>the javadoc for it before you try: you'll thank yourself, later.
>
>You can even use ServletContext.log() to write to the context log if you
>want. Otherwise, feel free to use your webapp's log4j or other logging
>facility (you'll have to configure this yourself). Just don't use
>System.out :)
>
>One might argue that blindly logging request parameters is not
>particularly useful. For instance, a quick look at your interface
>indicates that you can add several plots of land before performing the
>search. Those initial "add" operations may be of little use to you.
You are right.
>Instead, you may wish to log them only at a certain point in your
>workflow. Since most requests go to "oppositioncase.faces", you may not
>be able to map your Filter to a URL pattern that is fine-grained enough.
>Instead, it might make more sense to log this data when you know there's
>a reason to log it. Since you're expecting to perform statistical
>analysis on the data, you might even consider writing it directly to a
>database instead of to a plain-old log file.
I'm going this route for now:
FacesContext.getCurrentInstance().getExternalContext().log(the string buffer of the parcel listbox);
FacesContext.getCurrentInstance().getExternalContext().log(the map URL);
Produces:
Mar 30, 2011 12:02:35 PM org.apache.catalina.core.ApplicationContext log
INFO: PARCEL = '125-27-089'
Mar 30, 2011 12:02:45 PM org.apache.catalina.core.ApplicationContext log
INFO: http://planning.maricopa.gov/agsoutput/_ags_map08eb57df58224e1884e17a3e8a59b555.jpg
Mar 30, 2011 12:03:16 PM org.apache.catalina.core.ApplicationContext log
INFO: PARCEL = '125-27-089' OR PARCEL = '125-27-090' OR PARCEL = '125-27-091'
Mar 30, 2011 12:03:27 PM org.apache.catalina.core.ApplicationContext log
INFO: http://planning.maricopa.gov/agsoutput/_ags_map1a6e940afbd1494794c8d22b36f3a11a.jpg
>Cool webapp, by the way!
Thanks!
>
>NB: The "ctrl" key isn't always the modifier key to use to de-select
>items from a multi-select list. I think that's a Microsoft Windows
>convention, but it's CMD-click on Mac and probably something like
>META-click on *NIX.
Sadly, my department wanted that blurb on there. They forget stuff.
Leo
Re: Logging request parameters - Filter vs Servlet
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Leo,
On 3/30/2011 12:28 PM, Leo Donahue - PLANDEVX wrote:
>> -----Original Message-----
>> From: Christopher Schultz [mailto:chris@christopherschultz.net]
>> Subject: Re: Logging request parameters - Filter vs Servlet
>>>
>>> I don't know what I don't know... Log4j I guess. That is what
>>> everyone recommends.
>>
>> Hmm...
>
> ServletContext logging then?
The only logging the ServletContext will do is if you explicitly call
ServletContext.log(...) which Tomcat will direct to the appropriate
logger (set up using logging.properties or, if you go through the
configuration, log4j).
>> There is an AccessLogValve that you can use if you know which request
>> parameters you want to log. There is also a RequestDumperValve (and
>> RequestDumperFilter in 7.0) that you can use to dump everything from the
>> request. See the docs for details.
>>
>> Would those work for you?
>
> The AccessLog pattern I'm using:
> pattern="%h %l %u %t %r %q %s %b"
>
> The results:
> 2.3.4.5 - - [29/Mar/2011:16:02:39 -0700] "POST /oppositioncase/oppositioncase.faces HTTP/1.1" 200 38621
Hmm... sorry to have steered you astray: looking at the documentation,
AccessLogValve only allows you to log request headers and attributes,
not parameters. That's a shame...
> My web app uses the JSF framework, forgot to mention that.
>
> http://planning.maricopa.gov/oppositioncase
>
> samples:
> 211-52-002A
> 211-74-016
> 211-53-005C
So you need to log the "txtAPN" parameter, right?
> Those are the parameters I'm trying to capture, along with the map image url.
> A. We'd like to know how many requests actually generated a map image.
Can you tell that, just from a parameter value? I would think that the
URL itself would more likely tell you if a map was generated.
> B. We'd like to know whether this app is searching for parcels
> primarily in the unincorporated areas of the County, or parcels
> located within a city jurisdiction. That part I can figure out once
> I know which parcels people are searching.
Simply logging the parcels used in searches would allow you to do that,
as you've said. The RequestDumperValve logs a /ton/ of information, and
probably wouldn't get you what you want.
>>> Note that reading request parameters in a Filter may trigger parsing of a POST request body
>>> which may not be something you want to happen on every request.
>
> I guess my only option then is to log them from the web app. But
> somewhere I've read that is the wrong/lazy way to do logging.
I'm not sure where you read it, but that sounds like a platitude applied
as a blanket admonition not to log in your webapp. If it's appropriate
for your situation, then feel free to do it.
Back to the original "filter" question: logging using a filter is
perfectly acceptable if it's the right solution (see below for questions
that might lead you to other options). Writing the filter is trivial:
just implement the javax.servlet.Filter interface and be sure to read
the javadoc for it before you try: you'll thank yourself, later.
You can even use ServletContext.log() to write to the context log if you
want. Otherwise, feel free to use your webapp's log4j or other logging
facility (you'll have to configure this yourself). Just don't use
System.out :)
One might argue that blindly logging request parameters is not
particularly useful. For instance, a quick look at your interface
indicates that you can add several plots of land before performing the
search. Those initial "add" operations may be of little use to you.
Instead, you may wish to log them only at a certain point in your
workflow. Since most requests go to "oppositioncase.faces", you may not
be able to map your Filter to a URL pattern that is fine-grained enough.
Instead, it might make more sense to log this data when you know there's
a reason to log it. Since you're expecting to perform statistical
analysis on the data, you might even consider writing it directly to a
database instead of to a plain-old log file.
Cool webapp, by the way!
NB: The "ctrl" key isn't always the modifier key to use to de-select
items from a multi-select list. I think that's a Microsoft Windows
convention, but it's CMD-click on Mac and probably something like
META-click on *NIX.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2TeiYACgkQ9CaO5/Lv0PC3BgCcDrR863az0BctJcOI3gGcjD0J
kpcAn3WMLt7MseeErBLAADS6Gh+UNTbS
=LZdC
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Posted by Leo Donahue - PLANDEVX <Le...@mail.maricopa.gov>.
Chris,
>-----Original Message-----
>From: Christopher Schultz [mailto:chris@christopherschultz.net]
>Subject: Re: Logging request parameters - Filter vs Servlet
>>
>> I don't know what I don't know... Log4j I guess. That is what
>> everyone recommends.
>
>Hmm...
ServletContext logging then?
>There is an AccessLogValve that you can use if you know which request
>parameters you want to log. There is also a RequestDumperValve (and
>RequestDumperFilter in 7.0) that you can use to dump everything from the
>request. See the docs for details.
>
>Would those work for you?
The AccessLog pattern I'm using:
pattern="%h %l %u %t %r %q %s %b"
The results:
2.3.4.5 - - [29/Mar/2011:16:02:39 -0700] "POST /oppositioncase/oppositioncase.faces HTTP/1.1" 200 38621
My web app uses the JSF framework, forgot to mention that.
http://planning.maricopa.gov/oppositioncase
samples:
211-52-002A
211-74-016
211-53-005C
Those are the parameters I'm trying to capture, along with the map image url.
A. We'd like to know how many requests actually generated a map image.
B. We'd like to know whether this app is searching for parcels primarily in the unincorporated areas of the County, or parcels located within a city jurisdiction. That part I can figure out once I know which parcels people are searching.
>>Note that reading request parameters in a Filter may trigger parsing of a POST request body
>>which may not be something you want to happen on every request.
I guess my only option then is to log them from the web app. But somewhere I've read that is the wrong/lazy way to do logging.
Re: Logging request parameters - Filter vs Servlet
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Leo,
On 3/29/2011 5:44 PM, Leo Donahue - PLANDEVX wrote:
>> -----Original Message-----
>> From: Christopher Schultz [mailto:chris@christopherschultz.net]
>> Subject: Re: Logging request parameters - Filter vs Servlet
>>
>> On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote:
>>> Where do you initialize the Logger (Filter or Servlet)?
>>
>> Which logger?
>
> I don't know what I don't know... Log4j I guess. That is what everyone recommends.
Hmm...
>>> The servlet 2.5 spec says you can use filters for logging, but since
>>> I'm not modifying the request or response, is logging from within a
>>> filter the right approach to logging request parameters?
>>
>> That depends upon your requirements. What are they?
>
> I just want to capture the request parameters on a certain web app
> and log them (time and what they were) in a separate log file from
> the standard logs, so that I don't have to hunt them down in the
> standard Tomcat logs. The security tool that our telecom office uses
> for auditing our sites makes quite a mess of my standard logs
> periodically.
There is an AccessLogValve that you can use if you know which request
parameters you want to log. There is also a RequestDumperValve (and
RequestDumperFilter in 7.0) that you can use to dump everything from the
request. See the docs for details.
Would those work for you?
>> Note that reading request parameters in a Filter may trigger parsing of
>> a POST request body which may not be something you want to happen on
>> every request.
>
> An example of things I don't know that I don't know...
Heh. I suppose you'll know if things stop working.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2TRmcACgkQ9CaO5/Lv0PAIBQCfeDY2N9qXHbuijqz5cIOSnkj8
POgAn0ghVWuAkkiEv3cTrOTWsyCRe4IB
=c7cf
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Logging request parameters - Filter vs Servlet
Posted by Leo Donahue - PLANDEVX <Le...@mail.maricopa.gov>.
Chris,
>-----Original Message-----
>From: Christopher Schultz [mailto:chris@christopherschultz.net]
>Subject: Re: Logging request parameters - Filter vs Servlet
>
>Leo,
>
>On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote:
>> Where do you initialize the Logger (Filter or Servlet)?
>
>Which logger?
I don't know what I don't know... Log4j I guess. That is what everyone recommends.
>
>> The servlet 2.5 spec says you can use filters for logging, but since
>> I'm not modifying the request or response, is logging from within a
>> filter the right approach to logging request parameters?
>
>That depends upon your requirements. What are they?
I just want to capture the request parameters on a certain web app and log them (time and what they were) in a separate log file from the standard logs, so that I don't have to hunt them down in the standard Tomcat logs. The security tool that our telecom office uses for auditing our sites makes quite a mess of my standard logs periodically.
>
>Note that reading request parameters in a Filter may trigger parsing of
>a POST request body which may not be something you want to happen on
>every request.
An example of things I don't know that I don't know...
Leo
Re: Logging request parameters - Filter vs Servlet
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Leon,
On 3/29/2011 12:57 PM, Leo Donahue - PLANDEVX wrote:
> Where do you initialize the Logger (Filter or Servlet)?
Which logger?
> The servlet 2.5 spec says you can use filters for logging, but since
> I'm not modifying the request or response, is logging from within a
> filter the right approach to logging request parameters?
That depends upon your requirements. What are they?
Note that reading request parameters in a Filter may trigger parsing of
a POST request body which may not be something you want to happen on
every request.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2SQ2QACgkQ9CaO5/Lv0PCkogCfcjmdq3Y2nEboa10pIoqx532R
Wv0An0lqu05E16lGvZNOYBUedUFHAAF3
=VaL5
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org