You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jim Barber <ji...@ddihealth.com> on 2013/09/16 09:42:46 UTC
Filtering HTTP OPTIONS request method from logs?
Hi all.
I'm hoping someone on this list can help me since I've been reading docs,
mailing lists, FAQs, and so on for hours now, and I'm not having much luck
finding an answer to my question.
I am using Tomcat version 7.0.42 as packaged in Debian Linux.
In front of my Tomcat servers, I am using haproxy for load balancing.
The haproxy load balancers are using the HTTP OPTIONS request method to check
if the Tomcat servers are alive and healthy.
This results in log entries like the following in the Tomcat accesslog file:
10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
At the moment I'm getting one of these every 2seconds, but I haven't enabled
the second load balancer for HA purposes yet.
When I do that, I'll be getting twice as many hits of this type.
This is going to result in rather large log files full of noise that I'm not
interested in.
I've been trying to work out how to filter these out.
Basically I don't want to log anything that is using the HTTP OPTIONS Request
Method, but still want to log anything else that Tomcat usually logs.
I have a feeling it will come down to modifying the following entry in the
/etc/tomcat7/server.xml file:
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
Specifically adding the condition="<VALUE>" attribute, but I have no idea what to set
<VALUE> to.
The docs say that if ServletRequest.getAttribute(<VALUE>) returns null for the
attribute defined in condition, then the item will be logged.
Is there an ServletRequest attribute that is null when the http request method
is not using "OPTIONS"?
Or am I completely off track and there is a different way to filter these
access log messages?
Regards,
--
Jim Barber
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Jim Barber <ji...@ddihealth.com>.
On 17/09/2013 5:05 PM, André Warnier wrote:
> Jim Barber wrote:
>> All,
>>
>> On 16/09/2013 10:52 PM, Christopher Schultz wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> Felix,
>>>
>>> On 9/16/13 10:25 AM, Felix Schumacher wrote:
>>>> Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher
>>>> Schultz:
>>>>> Jim,
>>>>>
>>>>> On 9/16/13 3:42 AM, Jim Barber wrote:
>>>>>> I'm hoping someone on this list can help me since I've been
>>>>>> reading docs, mailing lists, FAQs, and so on for hours now, and
>>>>>> I'm not having much luck finding an answer to my question.
>>>>>>
>>>>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>>>>> In front of my Tomcat servers, I am using haproxy for load
>>>>>> balancing. The haproxy load balancers are using the HTTP
>>>>>> OPTIONS request method to check if the Tomcat servers are alive
>>>>>> and healthy.
>>>>>>
>>>>>> This results in log entries like the following in the Tomcat
>>>>>> accesslog file:
>>>>>>
>>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>>> [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS /
>>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000]
>>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>>> [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS /
>>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000]
>>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>>> [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS /
>>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000]
>>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>>> [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>>
>>>>>> At the moment I'm getting one of these every 2seconds, but I
>>>>>> haven't enabled the second load balancer for HA purposes yet.
>>>>>> When I do that, I'll be getting twice as many hits of this
>>>>>> type.
>>>>>>
>>>>>> This is going to result in rather large log files full of
>>>>>> noise that I'm not interested in.
>>>>>
>>>>> Playing the devil's advocate here a bit...
>>>>>
>>>>> Why wouldn't you be interested in getting these logs? They are
>>>>> requests being handled by your web server. They require (a small
>>>>> amount of) time and resources to process, and indicate that your
>>>>> lb is still reaching-out to determine the status of the app
>>>>> server.
>>>>>
>>>>> My recommendation would be to leave those logs in there (they
>>>>> accurately describe a real request) and filter them out if you
>>>>> want to do some kind of analytics against your log files and
>>>>> consider those OPTIONS requests to be noise.
>>>> I have had one case where I wanted to get rid of those requests
>>>> too, so I can understand the OP. But I have to admint I had a scary
>>>> feeling about it.
>>>>
>>>>>
>>>>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>>>> directory="logs" prefix="localhost_access_log." suffix=".txt"
>>>>>> pattern="%h %l %u %t "%r" %s %b" />
>>>>>>
>>>>>> Specifically adding the condition="<VALUE>" attribute, but I
>>>>>> have no idea what to set <VALUE> to.
>>>>>
>>>>> It's not that simple: if you want to use "condition", then you
>>>>> have a write a Valve (can't be a Filter, since it must run
>>>>> *before* the AccessLogValve) that tests the request and sets a
>>>>> request attribute that will then trigger this condition.
>>>>
>>>> That is not true, you can use a filter, since the logging will
>>>> happen *after* the request and can and will check the request
>>>> attribute then.
>>>
>>> Thanks for pointing that out. After I had sent my message, I realized
>>> that and decided not to post a followup after reading yours.
>>>
>>> - -chris
>>
>> Thank you all for your responses.
>> It looks like I'll just have to put up with these messages.
>> There seems to be no easy way to filter them out.
>>
>
> Actually, based on earlier responses and on the URLrewrite filter (http://http://tuckey.org/urlrewrite/), there may be a solution after all, which does not involve additional Java programming, as long as you are willing to do some research by yourself.
>
> (Short intro : the URLrewrite filter is a bit of a workhorse, simioar to mod_rewrite for Apache httpd, and which can do a multitude of things when it comes to filter/modify HTTP requests in Tomcat)
>
> First, get the URLrewrite User's Manual at http://urlrewritefilter.googlecode.com/svn/trunk/src/doc/manual/4.0/index.html
>
> and then search for :
>
> <condition> element (see "method" and "remote-addr")
>
> and
>
> <set> element (see "request (default)") : The same as request.setAttribute([name], [value]) (note, name must be set).
>
> So, the requests that you want to "not log" look like this :
> >>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
> >>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>
> and thus they address the "/" (default) webapp, originate from the client IP 10.122.32.4, and have the method "OPTIONS".
>
> And on the other hand, the AccessLogValve has an attribute which allows you to specify that if the request has an attribute (e.g.) "nolog" set, it will not be logged.
>
> So, by configuring your ROOT (default) webapp to include the URLrewrite filter, and configuring this filter properly, you should be able to attain your goal.
>
> Enough tips now, get to work..
>
>
> (And apart from your current issue, it really does not hurt to get to know the URLrewrite filter, because it is really a very useful and powerful piece of software).
Thanks for the info André.
I'll bear this in mind and hopefully get time to take a look at the URLrewrite filte in the near future.
Regards,
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by André Warnier <aw...@ice-sa.com>.
Jim Barber wrote:
> All,
>
> On 16/09/2013 10:52 PM, Christopher Schultz wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Felix,
>>
>> On 9/16/13 10:25 AM, Felix Schumacher wrote:
>>> Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher
>>> Schultz:
>>>> Jim,
>>>>
>>>> On 9/16/13 3:42 AM, Jim Barber wrote:
>>>>> I'm hoping someone on this list can help me since I've been
>>>>> reading docs, mailing lists, FAQs, and so on for hours now, and
>>>>> I'm not having much luck finding an answer to my question.
>>>>>
>>>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>>>> In front of my Tomcat servers, I am using haproxy for load
>>>>> balancing. The haproxy load balancers are using the HTTP
>>>>> OPTIONS request method to check if the Tomcat servers are alive
>>>>> and healthy.
>>>>>
>>>>> This results in log entries like the following in the Tomcat
>>>>> accesslog file:
>>>>>
>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000]
>>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>>> [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>>
>>>>> At the moment I'm getting one of these every 2seconds, but I
>>>>> haven't enabled the second load balancer for HA purposes yet.
>>>>> When I do that, I'll be getting twice as many hits of this
>>>>> type.
>>>>>
>>>>> This is going to result in rather large log files full of
>>>>> noise that I'm not interested in.
>>>>
>>>> Playing the devil's advocate here a bit...
>>>>
>>>> Why wouldn't you be interested in getting these logs? They are
>>>> requests being handled by your web server. They require (a small
>>>> amount of) time and resources to process, and indicate that your
>>>> lb is still reaching-out to determine the status of the app
>>>> server.
>>>>
>>>> My recommendation would be to leave those logs in there (they
>>>> accurately describe a real request) and filter them out if you
>>>> want to do some kind of analytics against your log files and
>>>> consider those OPTIONS requests to be noise.
>>> I have had one case where I wanted to get rid of those requests
>>> too, so I can understand the OP. But I have to admint I had a scary
>>> feeling about it.
>>>
>>>>
>>>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>>> directory="logs" prefix="localhost_access_log." suffix=".txt"
>>>>> pattern="%h %l %u %t "%r" %s %b" />
>>>>>
>>>>> Specifically adding the condition="<VALUE>" attribute, but I
>>>>> have no idea what to set <VALUE> to.
>>>>
>>>> It's not that simple: if you want to use "condition", then you
>>>> have a write a Valve (can't be a Filter, since it must run
>>>> *before* the AccessLogValve) that tests the request and sets a
>>>> request attribute that will then trigger this condition.
>>>
>>> That is not true, you can use a filter, since the logging will
>>> happen *after* the request and can and will check the request
>>> attribute then.
>>
>> Thanks for pointing that out. After I had sent my message, I realized
>> that and decided not to post a followup after reading yours.
>>
>> - -chris
>
> Thank you all for your responses.
> It looks like I'll just have to put up with these messages.
> There seems to be no easy way to filter them out.
>
Actually, based on earlier responses and on the URLrewrite filter
(http://http://tuckey.org/urlrewrite/), there may be a solution after all, which does not
involve additional Java programming, as long as you are willing to do some research by
yourself.
(Short intro : the URLrewrite filter is a bit of a workhorse, simioar to mod_rewrite for
Apache httpd, and which can do a multitude of things when it comes to filter/modify HTTP
requests in Tomcat)
First, get the URLrewrite User's Manual at
http://urlrewritefilter.googlecode.com/svn/trunk/src/doc/manual/4.0/index.html
and then search for :
<condition> element (see "method" and "remote-addr")
and
<set> element (see "request (default)") : The same as request.setAttribute([name],
[value]) (note, name must be set).
So, the requests that you want to "not log" look like this :
>>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
and thus they address the "/" (default) webapp, originate from the client IP 10.122.32.4,
and have the method "OPTIONS".
And on the other hand, the AccessLogValve has an attribute which allows you to specify
that if the request has an attribute (e.g.) "nolog" set, it will not be logged.
So, by configuring your ROOT (default) webapp to include the URLrewrite filter, and
configuring this filter properly, you should be able to attain your goal.
Enough tips now, get to work..
(And apart from your current issue, it really does not hurt to get to know the URLrewrite
filter, because it is really a very useful and powerful piece of software).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Jim Barber <ji...@ddihealth.com>.
All,
On 16/09/2013 10:52 PM, Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Felix,
>
> On 9/16/13 10:25 AM, Felix Schumacher wrote:
>> Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher
>> Schultz:
>>> Jim,
>>>
>>> On 9/16/13 3:42 AM, Jim Barber wrote:
>>>> I'm hoping someone on this list can help me since I've been
>>>> reading docs, mailing lists, FAQs, and so on for hours now, and
>>>> I'm not having much luck finding an answer to my question.
>>>>
>>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>>> In front of my Tomcat servers, I am using haproxy for load
>>>> balancing. The haproxy load balancers are using the HTTP
>>>> OPTIONS request method to check if the Tomcat servers are alive
>>>> and healthy.
>>>>
>>>> This results in log entries like the following in the Tomcat
>>>> accesslog file:
>>>>
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>> [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS /
>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000]
>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>> [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS /
>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000]
>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>> [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS /
>>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000]
>>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>>> [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>
>>>> At the moment I'm getting one of these every 2seconds, but I
>>>> haven't enabled the second load balancer for HA purposes yet.
>>>> When I do that, I'll be getting twice as many hits of this
>>>> type.
>>>>
>>>> This is going to result in rather large log files full of
>>>> noise that I'm not interested in.
>>>
>>> Playing the devil's advocate here a bit...
>>>
>>> Why wouldn't you be interested in getting these logs? They are
>>> requests being handled by your web server. They require (a small
>>> amount of) time and resources to process, and indicate that your
>>> lb is still reaching-out to determine the status of the app
>>> server.
>>>
>>> My recommendation would be to leave those logs in there (they
>>> accurately describe a real request) and filter them out if you
>>> want to do some kind of analytics against your log files and
>>> consider those OPTIONS requests to be noise.
>> I have had one case where I wanted to get rid of those requests
>> too, so I can understand the OP. But I have to admint I had a scary
>> feeling about it.
>>
>>>
>>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>> directory="logs" prefix="localhost_access_log." suffix=".txt"
>>>> pattern="%h %l %u %t "%r" %s %b" />
>>>>
>>>> Specifically adding the condition="<VALUE>" attribute, but I
>>>> have no idea what to set <VALUE> to.
>>>
>>> It's not that simple: if you want to use "condition", then you
>>> have a write a Valve (can't be a Filter, since it must run
>>> *before* the AccessLogValve) that tests the request and sets a
>>> request attribute that will then trigger this condition.
>>
>> That is not true, you can use a filter, since the logging will
>> happen *after* the request and can and will check the request
>> attribute then.
>
> Thanks for pointing that out. After I had sent my message, I realized
> that and decided not to post a followup after reading yours.
>
> - -chris
Thank you all for your responses.
It looks like I'll just have to put up with these messages.
There seems to be no easy way to filter them out.
Regards,
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Felix,
On 9/16/13 10:25 AM, Felix Schumacher wrote:
> Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher
> Schultz:
>> Jim,
>>
>> On 9/16/13 3:42 AM, Jim Barber wrote:
>>> I'm hoping someone on this list can help me since I've been
>>> reading docs, mailing lists, FAQs, and so on for hours now, and
>>> I'm not having much luck finding an answer to my question.
>>>
>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>> In front of my Tomcat servers, I am using haproxy for load
>>> balancing. The haproxy load balancers are using the HTTP
>>> OPTIONS request method to check if the Tomcat servers are alive
>>> and healthy.
>>>
>>> This results in log entries like the following in the Tomcat
>>> accesslog file:
>>>
>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS /
>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000]
>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>> [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS /
>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000]
>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>> [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS /
>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000]
>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>> [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS /
>>> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000]
>>> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
>>> [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>
>>> At the moment I'm getting one of these every 2seconds, but I
>>> haven't enabled the second load balancer for HA purposes yet.
>>> When I do that, I'll be getting twice as many hits of this
>>> type.
>>>
>>> This is going to result in rather large log files full of
>>> noise that I'm not interested in.
>>
>> Playing the devil's advocate here a bit...
>>
>> Why wouldn't you be interested in getting these logs? They are
>> requests being handled by your web server. They require (a small
>> amount of) time and resources to process, and indicate that your
>> lb is still reaching-out to determine the status of the app
>> server.
>>
>> My recommendation would be to leave those logs in there (they
>> accurately describe a real request) and filter them out if you
>> want to do some kind of analytics against your log files and
>> consider those OPTIONS requests to be noise.
> I have had one case where I wanted to get rid of those requests
> too, so I can understand the OP. But I have to admint I had a scary
> feeling about it.
>
>>
>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>> directory="logs" prefix="localhost_access_log." suffix=".txt"
>>> pattern="%h %l %u %t "%r" %s %b" />
>>>
>>> Specifically adding the condition="<VALUE>" attribute, but I
>>> have no idea what to set <VALUE> to.
>>
>> It's not that simple: if you want to use "condition", then you
>> have a write a Valve (can't be a Filter, since it must run
>> *before* the AccessLogValve) that tests the request and sets a
>> request attribute that will then trigger this condition.
>
> That is not true, you can use a filter, since the logging will
> happen *after* the request and can and will check the request
> attribute then.
Thanks for pointing that out. After I had sent my message, I realized
that and decided not to post a followup after reading yours.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJSNxs2AAoJEBzwKT+lPKRYJlIQAIZmTn6OA6aeNuENOAbXiXYs
U+RrWYN4HKIStoVY50Z+2x5r/5ji7j/Juqlyas649Onn05mr3p4stOhzXJXXq9v4
+jeCJLm7VK19tIB12HmjMoTKdf6GapimC0X/WQcEVXVPU2i7R9NAsltcAJGkx4cA
mmpCR/y15B4qt41UgteiIEV+8G7GGka1Zqr+J9qu/HWpAg0MWtWuVdggBjOYNRA/
RPGc2Ya3UUokaj4UAodyyQlOU4fWBXUb56w3MRlZ3z0ueA3noHT+2m8xTeAhHKrj
sYJR7bDUFkFM2z8xN3+AIcAWE5wM/AA+shuXGQDtKGvxd88rOIQhFJOFHr8h2Mkl
PIdxPcGGoj2Cn5RUkOdauhzc0TuwS7+e0zpXGJiTXdKQ0VjjPupqkde6z1v/q4IS
swzKBOPYf5xlhhEaI1hm7SvjTTulNfwT428nWMFlu3Z/9nXdkLQFcbKs3LRzYT0I
K6Y4GKX89IGCNbv3nplRQayHWYphtHEQ2eKB005hWo9Lw9IEduGMKdBX8fwcFrAL
IR3tSQoM6JaVfi2q0S6x4U/bk1asSc4FpD6UPjA4FXDHZ1cTyH7+j8IF/J5fzqlq
AxBXDSeA82CFtz+9hqc45F5ifQv6FSDR5JI1NZckYcKzXkHvtLmOyA4EP0BuiG6p
Pe7DPMTkGK3G46/9zaym
=Ggr6
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Felix Schumacher <fe...@internetallee.de>.
Am Montag, den 16.09.2013, 10:02 -0400 schrieb Christopher Schultz:
> Jim,
>
> On 9/16/13 3:42 AM, Jim Barber wrote:
> > I'm hoping someone on this list can help me since I've been reading
> > docs, mailing lists, FAQs, and so on for hours now, and I'm not
> > having much luck finding an answer to my question.
> >
> > I am using Tomcat version 7.0.42 as packaged in Debian Linux. In
> > front of my Tomcat servers, I am using haproxy for load balancing.
> > The haproxy load balancers are using the HTTP OPTIONS request
> > method to check if the Tomcat servers are alive and healthy.
> >
> > This results in log entries like the following in the Tomcat
> > accesslog file:
> >
> > 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0"
> > 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS /
> > HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000]
> > "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:55
> > +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
> > [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4
> > - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0"
> > 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS /
> > HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000]
> > "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:07
> > +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
> > [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4
> > - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
> >
> > At the moment I'm getting one of these every 2seconds, but I
> > haven't enabled the second load balancer for HA purposes yet. When
> > I do that, I'll be getting twice as many hits of this type.
> >
> > This is going to result in rather large log files full of noise
> > that I'm not interested in.
>
> Playing the devil's advocate here a bit...
>
> Why wouldn't you be interested in getting these logs? They are
> requests being handled by your web server. They require (a small
> amount of) time and resources to process, and indicate that your lb is
> still reaching-out to determine the status of the app server.
>
> My recommendation would be to leave those logs in there (they
> accurately describe a real request) and filter them out if you want to
> do some kind of analytics against your log files and consider those
> OPTIONS requests to be noise.
I have had one case where I wanted to get rid of those requests too, so
I can understand the OP. But I have to admint I had a scary feeling
about it.
>
> > <Valve className="org.apache.catalina.valves.AccessLogValve"
> > directory="logs" prefix="localhost_access_log." suffix=".txt"
> > pattern="%h %l %u %t "%r" %s %b" />
> >
> > Specifically adding the condition="<VALUE>" attribute, but I have
> > no idea what to set <VALUE> to.
>
> It's not that simple: if you want to use "condition", then you have a
> write a Valve (can't be a Filter, since it must run *before* the
> AccessLogValve) that tests the request and sets a request attribute
> that will then trigger this condition.
That is not true, you can use a filter, since the logging will happen
*after* the request and can and will check the request attribute then.
>
> Honestly, it's not worth it IMO.
>
> Just use logrotate + gzip and don't worry about disk space.
>
> If you filter-out those requests, there will come a time when you'll
> look back and say "wow, I wish we had all those lb requests in the log
> so we could tell what's happening".
As I admitted above that may very well be the case :)
Felix
>
> -chris
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Jim,
On 9/16/13 3:42 AM, Jim Barber wrote:
> I'm hoping someone on this list can help me since I've been reading
> docs, mailing lists, FAQs, and so on for hours now, and I'm not
> having much luck finding an answer to my question.
>
> I am using Tomcat version 7.0.42 as packaged in Debian Linux. In
> front of my Tomcat servers, I am using haproxy for load balancing.
> The haproxy load balancers are using the HTTP OPTIONS request
> method to check if the Tomcat servers are alive and healthy.
>
> This results in log entries like the following in the Tomcat
> accesslog file:
>
> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0"
> 200 - 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS /
> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000]
> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:12:55
> +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
> [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4
> - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0"
> 200 - 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS /
> HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000]
> "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - - [16/Sep/2013:17:13:07
> +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4 - -
> [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 - 10.122.32.4
> - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>
> At the moment I'm getting one of these every 2seconds, but I
> haven't enabled the second load balancer for HA purposes yet. When
> I do that, I'll be getting twice as many hits of this type.
>
> This is going to result in rather large log files full of noise
> that I'm not interested in.
Playing the devil's advocate here a bit...
Why wouldn't you be interested in getting these logs? They are
requests being handled by your web server. They require (a small
amount of) time and resources to process, and indicate that your lb is
still reaching-out to determine the status of the app server.
My recommendation would be to leave those logs in there (they
accurately describe a real request) and filter them out if you want to
do some kind of analytics against your log files and consider those
OPTIONS requests to be noise.
> <Valve className="org.apache.catalina.valves.AccessLogValve"
> directory="logs" prefix="localhost_access_log." suffix=".txt"
> pattern="%h %l %u %t "%r" %s %b" />
>
> Specifically adding the condition="<VALUE>" attribute, but I have
> no idea what to set <VALUE> to.
It's not that simple: if you want to use "condition", then you have a
write a Valve (can't be a Filter, since it must run *before* the
AccessLogValve) that tests the request and sets a request attribute
that will then trigger this condition.
Honestly, it's not worth it IMO.
Just use logrotate + gzip and don't worry about disk space.
If you filter-out those requests, there will come a time when you'll
look back and say "wow, I wish we had all those lb requests in the log
so we could tell what's happening".
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJSNw90AAoJEBzwKT+lPKRY5a0QAJBJK54sxoOAgmRwA3ZD7+O+
X1ckD5i/8oqhw3WSpK3F2+C979RZ0aWnt8/htAin9/Rq97MF0CxzDZIa/TYDKt+m
e4I4Mt/42Df604zRGM5pIXTj74wlCsaTdiDGgAalOgRZoj96w9bt+MM8hmNb91wK
/7UBHVXRaJbmlpLTPng+5d6R5f73LydPzcxbKGMExm889Qr3DVFLmQggx8+Gr6Ge
q36bCadfAXRNUQ606fj71XLwLENBQovHd25oF34l4ikwLiNrbh1RrCKYmk5n2QS2
nN6Lk98cb7hWIRae6XuMUPkVzm8W2dQS7gWUWZ2VEfLY1QKV4tYVma6PQaCP051N
tZ06wElAI/jqfRhoiYp4wJQdVw10z+BjD/3OyNpJhDydmwNAzZB80IsooLHAMFyf
3kIr/bDHL4sPcCXSxrn6XQEbb85bPYajccE2yKF7lisrNDMf049DMld+e94kaph6
tKoW+OqJDhFNYLfwlTBbJKGkh6zxkFadV621ltq++bHVSB4Xrg2Ut5w0ivv8g8CE
0lZnvZJdZigHRvhqwmbFE4xMfEdE1kQaZW2zkgDri4PXbftQV2SBZxacFE1T8SmH
02c4OMkXtzhlCBibg2SWXCrKql+nytGRpnfnmXqwaVWQfFFYIroksd15ATXKhS55
UeZ31WFFuZ+r05GAJobG
=lfwU
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by André Warnier <aw...@ice-sa.com>.
Jim Barber wrote:
> On 16/09/2013 4:46 PM, André Warnier wrote:
>> Apologies for top posting, just following the trend.
>>
>> OPTIONS are used quite a bit by e.g. DAV clients.
>> Won't you want also to add an IP filter then, to be able to block
>> selectively only the requests from the proxies themselves ?
>>
>> Cédric Couralet wrote:
>>> Hi,
>>>
>>> I'm also interested in a method to filter those OPTIONS.
>>> With the same setup, I basically created my own AccessLogValve wich
>>> does the filtering, something like :
>>>
>>> /**
>>> * Don't log request when HTTP Method is one of the exclude List
>>> */
>>> @Override
>>> public void log(Request request, Response response, long time) {
>>>
>>> if
>>> (Arrays.asList(exclude.split(",")).contains(request.getMethod())) {
>>> return;
>>> }
>>>
>>> super.log(request, response, time);
>>> }
>>>
>>> But there must be something better.
>>>
>>>
>>> 2013/9/16 Jim Barber <ji...@ddihealth.com>:
>>>> Hi all.
>>>>
>>>> I'm hoping someone on this list can help me since I've been reading
>>>> docs,
>>>> mailing lists, FAQs, and so on for hours now, and I'm not having
>>>> much luck
>>>> finding an answer to my question.
>>>>
>>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>>> In front of my Tomcat servers, I am using haproxy for load balancing.
>>>> The haproxy load balancers are using the HTTP OPTIONS request method to
>>>> check
>>>> if the Tomcat servers are alive and healthy.
>>>>
>>>> This results in log entries like the following in the Tomcat
>>>> accesslog file:
>>>>
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>> 10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>>
>>>> At the moment I'm getting one of these every 2seconds, but I haven't
>>>> enabled
>>>> the second load balancer for HA purposes yet.
>>>> When I do that, I'll be getting twice as many hits of this type.
>>>>
>>>> This is going to result in rather large log files full of noise that
>>>> I'm not
>>>> interested in.
>>>> I've been trying to work out how to filter these out.
>>>> Basically I don't want to log anything that is using the HTTP OPTIONS
>>>> Request
>>>> Method, but still want to log anything else that Tomcat usually logs.
>>>>
>>>> I have a feeling it will come down to modifying the following entry
>>>> in the
>>>> /etc/tomcat7/server.xml file:
>>>>
>>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>> directory="logs"
>>>> prefix="localhost_access_log." suffix=".txt"
>>>> pattern="%h %l %u %t "%r" %s %b" />
>>>>
>>>> Specifically adding the condition="<VALUE>" attribute, but I have no
>>>> idea
>>>> what to set
>>>> <VALUE> to.
>>>> The docs say that if ServletRequest.getAttribute(<VALUE>) returns
>>>> null for
>>>> the
>>>> attribute defined in condition, then the item will be logged.
>>>> Is there an ServletRequest attribute that is null when the http request
>>>> method
>>>> is not using "OPTIONS"?
>>>>
>>>> Or am I completely off track and there is a different way to filter
>>>> these
>>>> access log messages?
>>>>
>>>> Regards,
>>>>
>>>> --
>>>> Jim Barber
>
> Hi André.
>
> I'm not sure I follow what you're saying.
> I don't want an IP filter, since I need the HTTP OPTIONS check from the
> load
> balancers to reach the Tomcat servers and a response to come back, or
> else the
> load balancers will think the tomcat instance is unhealthy.
> I just don't want that check to be logged at all.
>
> Although there are other things that use the HTTP OPTIONS check, these load
> balancers are only exposed to internal traffic requesting specific servlets
> from the Tomcat servers, and so there won't be anything else of interest
> using
> the HTTP OPTIONS request methods to the Tomcat servers.
>
>
> Hi Cédric.
>
> What you posted is some Java code that needs to be compiled and then the
> resulting class file put somewhere where Tomcat can find it right?
yes.
> Is it only partial code where 'exclude' was some sort of pre-populated
> comma separated string?
yes, it was only the basic idea.
> Just checking as it doesn't look like anything that you can put direct
> into a
> Tomcat configuration file to me.
> Or is it?
No.
There isn't any configuration option currently that I know of, which answers your need.
So the solution would be indeed to either modify the AccessLogValve code (which is openly
available), or override it (as Cedric seems to have done).
The remark that I made about the filtering of the OPTIONS requests in the logs by origin
IP was generic, not specific to your case.
I do see a lot of such OPTIONS requests being logged also on servers which I manage,
coming from internal watchdog software processes. These are uninteresting for access log
purposes, and it would be nice to be able to filter them out.
On the other hand, and in general, there can also be OPTIONS requests coming from
legitimate clients, which need to be logged.
So I did not suggest to filter all requests via IP, only to selectively log according to
client IP. The IP filtering would only concern the logging, not the request processing.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Jim Barber <ji...@ddihealth.com>.
On 16/09/2013 4:46 PM, André Warnier wrote:
> Apologies for top posting, just following the trend.
>
> OPTIONS are used quite a bit by e.g. DAV clients.
> Won't you want also to add an IP filter then, to be able to block selectively only the requests from the proxies themselves ?
>
> Cédric Couralet wrote:
>> Hi,
>>
>> I'm also interested in a method to filter those OPTIONS.
>> With the same setup, I basically created my own AccessLogValve wich
>> does the filtering, something like :
>>
>> /**
>> * Don't log request when HTTP Method is one of the exclude List
>> */
>> @Override
>> public void log(Request request, Response response, long time) {
>>
>> if (Arrays.asList(exclude.split(",")).contains(request.getMethod())) {
>> return;
>> }
>>
>> super.log(request, response, time);
>> }
>>
>> But there must be something better.
>>
>>
>> 2013/9/16 Jim Barber <ji...@ddihealth.com>:
>>> Hi all.
>>>
>>> I'm hoping someone on this list can help me since I've been reading docs,
>>> mailing lists, FAQs, and so on for hours now, and I'm not having much luck
>>> finding an answer to my question.
>>>
>>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>>> In front of my Tomcat servers, I am using haproxy for load balancing.
>>> The haproxy load balancers are using the HTTP OPTIONS request method to
>>> check
>>> if the Tomcat servers are alive and healthy.
>>>
>>> This results in log entries like the following in the Tomcat accesslog file:
>>>
>>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
>>> 10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>>
>>> At the moment I'm getting one of these every 2seconds, but I haven't enabled
>>> the second load balancer for HA purposes yet.
>>> When I do that, I'll be getting twice as many hits of this type.
>>>
>>> This is going to result in rather large log files full of noise that I'm not
>>> interested in.
>>> I've been trying to work out how to filter these out.
>>> Basically I don't want to log anything that is using the HTTP OPTIONS
>>> Request
>>> Method, but still want to log anything else that Tomcat usually logs.
>>>
>>> I have a feeling it will come down to modifying the following entry in the
>>> /etc/tomcat7/server.xml file:
>>>
>>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>>> directory="logs"
>>> prefix="localhost_access_log." suffix=".txt"
>>> pattern="%h %l %u %t "%r" %s %b" />
>>>
>>> Specifically adding the condition="<VALUE>" attribute, but I have no idea
>>> what to set
>>> <VALUE> to.
>>> The docs say that if ServletRequest.getAttribute(<VALUE>) returns null for
>>> the
>>> attribute defined in condition, then the item will be logged.
>>> Is there an ServletRequest attribute that is null when the http request
>>> method
>>> is not using "OPTIONS"?
>>>
>>> Or am I completely off track and there is a different way to filter these
>>> access log messages?
>>>
>>> Regards,
>>>
>>> --
>>> Jim Barber
Hi André.
I'm not sure I follow what you're saying.
I don't want an IP filter, since I need the HTTP OPTIONS check from the load
balancers to reach the Tomcat servers and a response to come back, or else the
load balancers will think the tomcat instance is unhealthy.
I just don't want that check to be logged at all.
Although there are other things that use the HTTP OPTIONS check, these load
balancers are only exposed to internal traffic requesting specific servlets
from the Tomcat servers, and so there won't be anything else of interest using
the HTTP OPTIONS request methods to the Tomcat servers.
Hi Cédric.
What you posted is some Java code that needs to be compiled and then the
resulting class file put somewhere where Tomcat can find it right?
Is it only partial code where 'exclude' was some sort of pre-populated
comma separated string?
Just checking as it doesn't look like anything that you can put direct into a
Tomcat configuration file to me.
Or is it?
Regards,
Jim Barber
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Cédric Couralet <ce...@gmail.com>.
2013/9/16 André Warnier <aw...@ice-sa.com>:
> Apologies for top posting, just following the trend.
>
> OPTIONS are used quite a bit by e.g. DAV clients.
> Won't you want also to add an IP filter then, to be able to block
> selectively only the requests from the proxies themselves ?
>
>
Sorry for the top-post, i have got to find a better client ...
If you are talking about my message, I agree, I didn't do it because
in my case, there cannot be any other OPTION than for the proxy itself
(we don't use all those new technologies like DAV :) ).
And again, I'm really looking for a better way to handle that.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by André Warnier <aw...@ice-sa.com>.
Apologies for top posting, just following the trend.
OPTIONS are used quite a bit by e.g. DAV clients.
Won't you want also to add an IP filter then, to be able to block selectively only the
requests from the proxies themselves ?
Cédric Couralet wrote:
> Hi,
>
> I'm also interested in a method to filter those OPTIONS.
> With the same setup, I basically created my own AccessLogValve wich
> does the filtering, something like :
>
> /**
> * Don't log request when HTTP Method is one of the exclude List
> */
> @Override
> public void log(Request request, Response response, long time) {
>
> if (Arrays.asList(exclude.split(",")).contains(request.getMethod())) {
> return;
> }
>
> super.log(request, response, time);
> }
>
> But there must be something better.
>
>
> 2013/9/16 Jim Barber <ji...@ddihealth.com>:
>> Hi all.
>>
>> I'm hoping someone on this list can help me since I've been reading docs,
>> mailing lists, FAQs, and so on for hours now, and I'm not having much luck
>> finding an answer to my question.
>>
>> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
>> In front of my Tomcat servers, I am using haproxy for load balancing.
>> The haproxy load balancers are using the HTTP OPTIONS request method to
>> check
>> if the Tomcat servers are alive and healthy.
>>
>> This results in log entries like the following in the Tomcat accesslog file:
>>
>> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
>> 10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>>
>> At the moment I'm getting one of these every 2seconds, but I haven't enabled
>> the second load balancer for HA purposes yet.
>> When I do that, I'll be getting twice as many hits of this type.
>>
>> This is going to result in rather large log files full of noise that I'm not
>> interested in.
>> I've been trying to work out how to filter these out.
>> Basically I don't want to log anything that is using the HTTP OPTIONS
>> Request
>> Method, but still want to log anything else that Tomcat usually logs.
>>
>> I have a feeling it will come down to modifying the following entry in the
>> /etc/tomcat7/server.xml file:
>>
>> <Valve className="org.apache.catalina.valves.AccessLogValve"
>> directory="logs"
>> prefix="localhost_access_log." suffix=".txt"
>> pattern="%h %l %u %t "%r" %s %b" />
>>
>> Specifically adding the condition="<VALUE>" attribute, but I have no idea
>> what to set
>> <VALUE> to.
>> The docs say that if ServletRequest.getAttribute(<VALUE>) returns null for
>> the
>> attribute defined in condition, then the item will be logged.
>> Is there an ServletRequest attribute that is null when the http request
>> method
>> is not using "OPTIONS"?
>>
>> Or am I completely off track and there is a different way to filter these
>> access log messages?
>>
>> Regards,
>>
>> --
>> Jim Barber
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Felix Schumacher <fe...@internetallee.de>.
Am Montag, den 16.09.2013, 10:01 +0200 schrieb Cédric Couralet:
> Hi,
>
> I'm also interested in a method to filter those OPTIONS.
> With the same setup, I basically created my own AccessLogValve wich
> does the filtering, something like :
>
> /**
> * Don't log request when HTTP Method is one of the exclude List
> */
> @Override
> public void log(Request request, Response response, long time) {
>
> if (Arrays.asList(exclude.split(",")).contains(request.getMethod())) {
> return;
> }
>
> super.log(request, response, time);
> }
>
> But there must be something better.
If you look at the documentation fot the AccessLogValve
(http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Access_Log_Valve) you will find three config-parameters "conditionIf", "conditionUnless" and the old one "condition".
In your case you could set "conditionUnless" to "junk" and use a filter
to mark every request you don't want to log by setting an attribute
"junk" to a non-null value on that request.
In your Filter-class you would then implement the doFilter method
something like this
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
if (request instanceof HttpServletRequest) {
HttpServletRequest httpRequest = (HttpServletRequest) request;
if ("OPTIONS".equals(httpRequest.getMethod())) {
request.setAttribute("junk", true);
}
}
chain.doFilter(request, response);
}
You would probably check for the remoteIp, too, to be sure to only
exclude the HAProxy requests from your log-files.
Regards
Felix
>
>
> 2013/9/16 Jim Barber <ji...@ddihealth.com>:
> > Hi all.
> >
> > I'm hoping someone on this list can help me since I've been reading docs,
> > mailing lists, FAQs, and so on for hours now, and I'm not having much luck
> > finding an answer to my question.
> >
> > I am using Tomcat version 7.0.42 as packaged in Debian Linux.
> > In front of my Tomcat servers, I am using haproxy for load balancing.
> > The haproxy load balancers are using the HTTP OPTIONS request method to
> > check
> > if the Tomcat servers are alive and healthy.
> >
> > This results in log entries like the following in the Tomcat accesslog file:
> >
> > 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
> > 10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
> >
> > At the moment I'm getting one of these every 2seconds, but I haven't enabled
> > the second load balancer for HA purposes yet.
> > When I do that, I'll be getting twice as many hits of this type.
> >
> > This is going to result in rather large log files full of noise that I'm not
> > interested in.
> > I've been trying to work out how to filter these out.
> > Basically I don't want to log anything that is using the HTTP OPTIONS
> > Request
> > Method, but still want to log anything else that Tomcat usually logs.
> >
> > I have a feeling it will come down to modifying the following entry in the
> > /etc/tomcat7/server.xml file:
> >
> > <Valve className="org.apache.catalina.valves.AccessLogValve"
> > directory="logs"
> > prefix="localhost_access_log." suffix=".txt"
> > pattern="%h %l %u %t "%r" %s %b" />
> >
> > Specifically adding the condition="<VALUE>" attribute, but I have no idea
> > what to set
> > <VALUE> to.
> > The docs say that if ServletRequest.getAttribute(<VALUE>) returns null for
> > the
> > attribute defined in condition, then the item will be logged.
> > Is there an ServletRequest attribute that is null when the http request
> > method
> > is not using "OPTIONS"?
> >
> > Or am I completely off track and there is a different way to filter these
> > access log messages?
> >
> > Regards,
> >
> > --
> > Jim Barber
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Filtering HTTP OPTIONS request method from logs?
Posted by Cédric Couralet <ce...@gmail.com>.
Hi,
I'm also interested in a method to filter those OPTIONS.
With the same setup, I basically created my own AccessLogValve wich
does the filtering, something like :
/**
* Don't log request when HTTP Method is one of the exclude List
*/
@Override
public void log(Request request, Response response, long time) {
if (Arrays.asList(exclude.split(",")).contains(request.getMethod())) {
return;
}
super.log(request, response, time);
}
But there must be something better.
2013/9/16 Jim Barber <ji...@ddihealth.com>:
> Hi all.
>
> I'm hoping someone on this list can help me since I've been reading docs,
> mailing lists, FAQs, and so on for hours now, and I'm not having much luck
> finding an answer to my question.
>
> I am using Tomcat version 7.0.42 as packaged in Debian Linux.
> In front of my Tomcat servers, I am using haproxy for load balancing.
> The haproxy load balancers are using the HTTP OPTIONS request method to
> check
> if the Tomcat servers are alive and healthy.
>
> This results in log entries like the following in the Tomcat accesslog file:
>
> 10.122.32.4 - - [16/Sep/2013:17:12:49 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:12:51 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:12:53 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:12:55 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:12:57 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:12:59 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:01 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:03 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:05 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:07 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:09 +1000] "OPTIONS / HTTP/1.0" 200 -
> 10.122.32.4 - - [16/Sep/2013:17:13:11 +1000] "OPTIONS / HTTP/1.0" 200 -
>
> At the moment I'm getting one of these every 2seconds, but I haven't enabled
> the second load balancer for HA purposes yet.
> When I do that, I'll be getting twice as many hits of this type.
>
> This is going to result in rather large log files full of noise that I'm not
> interested in.
> I've been trying to work out how to filter these out.
> Basically I don't want to log anything that is using the HTTP OPTIONS
> Request
> Method, but still want to log anything else that Tomcat usually logs.
>
> I have a feeling it will come down to modifying the following entry in the
> /etc/tomcat7/server.xml file:
>
> <Valve className="org.apache.catalina.valves.AccessLogValve"
> directory="logs"
> prefix="localhost_access_log." suffix=".txt"
> pattern="%h %l %u %t "%r" %s %b" />
>
> Specifically adding the condition="<VALUE>" attribute, but I have no idea
> what to set
> <VALUE> to.
> The docs say that if ServletRequest.getAttribute(<VALUE>) returns null for
> the
> attribute defined in condition, then the item will be logged.
> Is there an ServletRequest attribute that is null when the http request
> method
> is not using "OPTIONS"?
>
> Or am I completely off track and there is a different way to filter these
> access log messages?
>
> Regards,
>
> --
> Jim Barber
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org