You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Grant Peel <gp...@thenetnow.com> on 2008/07/30 01:22:03 UTC
[users@httpd] Here's a new one (to me).
Hi all,
Just when you thought you'd seen it all.
On this past Saturday, my server started seeing sporadic spikes in CPU
usage. As it turns out, somehow, some bot or something somewhere is
connecting to me server and relaying messages to another server. The logs
below have been Googled several times with no real answer, other than, the
mod_proxy issue, which is not the case on this server as mod_proxy is not
installed (confirmed in httpd.conf and httpd -M).
When that (attack) hits, I wind up having to stop Apache, Exim, and clearing
the mailq, by which time there are several hundred thousand of bounces (for
'www') which can't be delivered.
That having been said, I did have 'www' set as a trusted user in my Exim
configuration, which allows the Apache user to set the "From:" line in
outgoing emails (Webforms etc). I have since removed that trusted user, but,
that stops my clients from sending email via webforms etc.
That having been said, this is a production webserver, and it handles
several hundred thousand hits a day, and sends/recieves serveral thousand
emails a day, a functionality I cant afford to loose.
I am hoping there is some simple fix somewhere I can find that someone here
might suggest. Any help would be greatly appreciated.
Log lines:
66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "GET
http://www.microsoft.com/ HTTP/1.0" 200 1401 "-"
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
1.1.4322)"
66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "POST
http://lti-mail01.ltinetworks.com:25/ HTTP/1.0"
200 1401 "-" "-"
66.139.69.201 - - [29/Jul/2008:04:02:00 -0400] "CONNECT
http://lti-mail01.ltinetworks.com:25 HTTP/1.
0" 400 226 "-" "-"
-Grant
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Graeme Fowler <gr...@graemef.net>.
On Wed, 2008-07-30 at 20:50 -0400, Grant Peel wrote:
> Is there any better logging knobs I can turn to try and track down an
> offending script?
Possibly, but you'd be better off running something over your access
logs in order to see what you can see.
Assuming a standard, as-supplied "combined" log format of:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
combined
Then something like this should help you find the script that's a
problem:
awk '{ print $6 " " $7 " " $8 }' < /var/log/httpd/access_log.1 | sort |
uniq -c | sort -n
That'll throw a list of requests at you - you should be able to look
through that to find the script name. If you extend it a bit (for
example you have the vhost logged at position 1) then you could add $1
as the first awk element and sort on that, too.
If all your logs are separated into directories per vhost, you'll have
to iterate over them all.
Graeme
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Eric Covener <co...@gmail.com>.
On Wed, Jul 30, 2008 at 8:50 PM, Grant Peel <gp...@thenetnow.com> wrote:
> Joshua,
>
> Thanks for the reply, but I have already confirmed that mod_proxy is not
> running on the server.
Your situation is specifically addressed in the ProxyAbuse link.
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Grant Peel <gp...@thenetnow.com>.
Joshua,
Thanks for the reply, but I have already confirmed that mod_proxy is not
running on the server.
Mod_Security is installed and is up to date.
I am thinking now I may have an unsecure form to email script in a virtual
host somewhere.
Is there any better logging knobs I can turn to try and track down an
offending script?
-Grant
----- Original Message -----
From: "Joshua Slive" <jo...@slive.ca>
To: <us...@httpd.apache.org>; "Grant Peel" <gp...@thenetnow.com>
Sent: Wednesday, July 30, 2008 5:40 PM
Subject: Re: [users@httpd] Here's a new one (to me).
> On Wed, Jul 30, 2008 at 4:20 PM, Grant Peel <gp...@thenetnow.com> wrote:
>> OK, understood,
>>
>> So does that mean that the original problem (my Apache server connecting
>> to
>> an open relay), is starting AT my server itself?
>
> This is discussed quite thoroughly at
> http://wiki.apache.org/httpd/ProxyAbuse
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Joshua Slive <jo...@slive.ca>.
On Wed, Jul 30, 2008 at 4:20 PM, Grant Peel <gp...@thenetnow.com> wrote:
> OK, understood,
>
> So does that mean that the original problem (my Apache server connecting to
> an open relay), is starting AT my server itself?
This is discussed quite thoroughly at
http://wiki.apache.org/httpd/ProxyAbuse
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Grant Peel <gp...@thenetnow.com>.
OK, understood,
So does that mean that the original problem (my Apache server connecting to
an open relay), is starting AT my server itself?
i.e. Being sent to my server on port 80 but POSTing to another server on
port 25?
66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "POST
http://lti-mail01.ltinetworks.com:25/ HTTP/1.0"
200 1401 "-" "-"
66.139.69.201 - - [29/Jul/2008:04:02:00 -0400] "CONNECT
http://lti-mail01.ltinetworks.com:25 HTTP/1.
0" 400 226 "-" "-"
I am really in need of stopping this!
mod_security didnt seem to help.
-Grant
----- Original Message -----
From: "William A. Rowe, Jr." <wr...@rowe-clan.net>
To: <us...@httpd.apache.org>
Sent: Tuesday, July 29, 2008 11:34 PM
Subject: Re: [users@httpd] Here's a new one (to me).
> Grant Peel wrote:
>> index.html = 1401 bytes.
>>
>> ?
>
> Well that's good, you have to understand that without proxy module
> enabled,
> these are /local/ requests...
>
>>> Grant Peel wrote:
>>>>
>>>> On this past Saturday, my server started seeing sporadic spikes in CPU
>>>> usage. As it turns out, somehow, some bot or something somewhere is
>>>> connecting to me server and relaying messages to another server.
>>>
>>> Don't freak out...
>>>
>>>> Log lines:
>>>>
>>>> 66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "GET
>>>> http://www.microsoft.com/ HTTP/1.0" 200 1401 "-"
>>>> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
>>>> 1.1.4322)"
>>>
>>> How large is your index.html?
>
>
> go ahead and (without using DNS) just
>
> telnet {your-ip} 80
>
> GET http://www.microsoft.com/ HTTP/1.0
> Host: www.microsoft.com
>
> and see what the server comes back with; it will be your own index.html.
>
> Without proxy; the http://hostname is stripped off, and your server
> simply served up /
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Grant Peel wrote:
> index.html = 1401 bytes.
>
> ?
Well that's good, you have to understand that without proxy module enabled,
these are /local/ requests...
>> Grant Peel wrote:
>>>
>>> On this past Saturday, my server started seeing sporadic spikes in
>>> CPU usage. As it turns out, somehow, some bot or something somewhere
>>> is connecting to me server and relaying messages to another server.
>>
>> Don't freak out...
>>
>>> Log lines:
>>>
>>> 66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "GET
>>> http://www.microsoft.com/ HTTP/1.0" 200 1401 "-"
>>> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
>>> 1.1.4322)"
>>
>> How large is your index.html?
go ahead and (without using DNS) just
telnet {your-ip} 80
GET http://www.microsoft.com/ HTTP/1.0
Host: www.microsoft.com
and see what the server comes back with; it will be your own index.html.
Without proxy; the http://hostname is stripped off, and your server
simply served up /
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by Grant Peel <gp...@thenetnow.com>.
index.html = 1401 bytes.
?
-Grant
----- Original Message -----
From: "William A. Rowe, Jr." <wr...@rowe-clan.net>
To: <us...@httpd.apache.org>
Sent: Tuesday, July 29, 2008 7:46 PM
Subject: Re: [users@httpd] Here's a new one (to me).
> Grant Peel wrote:
>>
>> On this past Saturday, my server started seeing sporadic spikes in CPU
>> usage. As it turns out, somehow, some bot or something somewhere is
>> connecting to me server and relaying messages to another server.
>
> Don't freak out...
>
>> Log lines:
>>
>> 66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "GET
>> http://www.microsoft.com/ HTTP/1.0" 200 1401 "-"
>> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
>> 1.1.4322)"
>
> How large is your index.html?
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Here's a new one (to me).
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Grant Peel wrote:
>
> On this past Saturday, my server started seeing sporadic spikes in CPU
> usage. As it turns out, somehow, some bot or something somewhere is
> connecting to me server and relaying messages to another server.
Don't freak out...
> Log lines:
>
> 66.139.69.201 - - [29/Jul/2008:04:01:58 -0400] "GET
> http://www.microsoft.com/ HTTP/1.0" 200 1401 "-"
> "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
> 1.1.4322)"
How large is your index.html?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org