You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by seb hould <ap...@gmail.com> on 2005/03/29 14:18:37 UTC
[users@httpd] apache attack
I believe I was recently attacked but still there seems to be
something missing. Yesterday my web server went pretty slow at a
certain point. When I checked my Linux process list there we're
roughly 10 times as much processes as usual (maxed from the apache
configs) and Apache was killing the oldest processes. This is not
normal traffic, and I for sure thought I was either attacked or
someone made a very bad script. Strangely enough, there are
absolutely no sign of additional requests in the apache logfile. By
looking at the file there are no more traffic at the time of the
incident than in normal circumstances. There ain't no sign of a bad
script (same source IP, same URI). So I'm supposing it was a DOS
attack but can someone explain why it wouldn't show up in the logs.
Is it that we recieved so many requests all at the same time and
Apache wasn't able to process them ? The load average on my server
went over 33 and the MySQL server was also quite busy (it is located
on another server).
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] apache attack
Posted by Markus Mayer <my...@gmx.at>.
Update apache to 1.3.33 - security patches.
Update PHP to 4.3.10 - security patches. Consider applying the hardened PHP
patches written by Stefan Esser http://www.hardened-php.net/
Unless the php script is threaded or forked or something like that, which I
doubt, your problem is most likely not there - apache started falling over at
15:10, the php mailing script was finished at 14:47. If you want to be sure,
test this by copying the script to a test area and setting up a test scenario
where you get the mails at some different mail accounts.
Look at what scripts were called in the few minutes before apache started
having problems and up until the time where you stopped apache. If there is
something solid to find, it will most likely be in this time frame.
Consider setting the apache log level to info until you've found the source of
the problem, or at least while you're testing. Maybe with this logging level
you can get more information should the situation come up again. Make
similar settings with mysql.
Without sufficient logging, it's difficult to see what you problem was. At
best you can try to reproduce the problem, then take steps to prevent it
happening again.
Markus
On Tuesday 29 March 2005 19:17, seb hould wrote:
> Apache 1.3.29
> RedHat Linux 9, kernel 2.4.20-31.9
> PHP 4.3.4
> MySQL 4.1.7
>
> First all thanks for the tips. I checked the MySQL error logs but
> unfortunately it is not as verbose as I wish it would be. The
> /var/log/message tells me the first apache process killed was at 15:10
> (so it maxed at that time). So I looked at all the requests from that
> time up until 15:27 when I stopped Apache. There were continuous
> requests but I believe these were requests that Apache could serve
> without the help of the db (this is not 100% checked but it does seem
> like it) and there seem to be less requests than in usual
> circumstances. The Apache error log reports nothing unusual. I did
> get a tip though that a user ran a newsletter-sending PHP script
> roughly at 14:47. The script has been used many times before and
> never failed but it does issue many SQL requests. The culprit is
> probably the MySQL server.
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] apache attack
Posted by herauthon <he...@home.nl>.
[newb at MySQL]
are there buffers settings?
are you using squid?
how is your port range (wide enough?)
{very wild guess)
----- Original Message -----
From: "seb hould" <ap...@gmail.com>
To: <us...@httpd.apache.org>
Sent: Tuesday, March 29, 2005 7:17 PM
Subject: Re: [users@httpd] apache attack
> Apache 1.3.29
> RedHat Linux 9, kernel 2.4.20-31.9
> PHP 4.3.4
> MySQL 4.1.7
>
> First all thanks for the tips. I checked the MySQL error logs but
> unfortunately it is not as verbose as I wish it would be. The
> /var/log/message tells me the first apache process killed was at 15:10
> (so it maxed at that time). So I looked at all the requests from that
> time up until 15:27 when I stopped Apache. There were continuous
> requests but I believe these were requests that Apache could serve
> without the help of the db (this is not 100% checked but it does seem
> like it) and there seem to be less requests than in usual
> circumstances. The Apache error log reports nothing unusual. I did
> get a tip though that a user ran a newsletter-sending PHP script
> roughly at 14:47. The script has been used many times before and
> never failed but it does issue many SQL requests. The culprit is
> probably the MySQL server.
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] apache attack
Posted by seb hould <ap...@gmail.com>.
Apache 1.3.29
RedHat Linux 9, kernel 2.4.20-31.9
PHP 4.3.4
MySQL 4.1.7
First all thanks for the tips. I checked the MySQL error logs but
unfortunately it is not as verbose as I wish it would be. The
/var/log/message tells me the first apache process killed was at 15:10
(so it maxed at that time). So I looked at all the requests from that
time up until 15:27 when I stopped Apache. There were continuous
requests but I believe these were requests that Apache could serve
without the help of the db (this is not 100% checked but it does seem
like it) and there seem to be less requests than in usual
circumstances. The Apache error log reports nothing unusual. I did
get a tip though that a user ran a newsletter-sending PHP script
roughly at 14:47. The script has been used many times before and
never failed but it does issue many SQL requests. The culprit is
probably the MySQL server.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] apache attack
Posted by herauthon <he...@home.nl>.
Apache 2+
Linux Kernel version?
crontab -e
0 * * * * * analog -f /var/logs/access_log
keep backups of the logs, saved to non-user
owned map, rotating
logname=log`date +%d%m%y%H%M%S
----- Original Message -----
From: "seb hould" <ap...@gmail.com>
To: <us...@httpd.apache.org>
Sent: Tuesday, March 29, 2005 2:18 PM
Subject: [users@httpd] apache attack
> I believe I was recently attacked but still there seems to be
> something missing. Yesterday my web server went pretty slow at a
> certain point. When I checked my Linux process list there we're
> roughly 10 times as much processes as usual (maxed from the apache
> configs) and Apache was killing the oldest processes. This is not
> normal traffic, and I for sure thought I was either attacked or
> someone made a very bad script. Strangely enough, there are
> absolutely no sign of additional requests in the apache logfile. By
> looking at the file there are no more traffic at the time of the
> incident than in normal circumstances. There ain't no sign of a bad
> script (same source IP, same URI). So I'm supposing it was a DOS
> attack but can someone explain why it wouldn't show up in the logs.
> Is it that we recieved so many requests all at the same time and
> Apache wasn't able to process them ? The load average on my server
> went over 33 and the MySQL server was also quite busy (it is located
> on another server).
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] apache attack
Posted by Markus Mayer <my...@gmx.at>.
Not likely. We've seen this type of thing on our servers when mysql gets too
busy or doesn't have enough resources to do its job. Apache will keep the
connections open until it's completed the request. In apache2, which I
assume you're using, the threads (again an assumption based on the guess that
you're using php) have to wait for an answer from the mysql server, so they
stay there until they can complete the request. Apache writes to the logs
when a request is complete, not before, which may be why you don't see the
requests. I don't know if you can change this behaviour. Look in the error
log file, maybe there's something there. Also look at your error log level
config for apache.
The exact cause is something you will have to look at. If mysql falls into
swap and is running slow, you need to either work on its configuration or do
something with the hardware. It may also be useful to look at the logs in
mysql, assuming you have logging turned on. Also, turn on the slow-request
log feature in mysql to see requests that take too long (read the manual for
info).
In your next posts, please include more info about what versions of what
software you're using.
Markus.
On Tuesday 29 March 2005 14:18, seb hould wrote:
> I believe I was recently attacked but still there seems to be
> something missing. Yesterday my web server went pretty slow at a
> certain point. When I checked my Linux process list there we're
> roughly 10 times as much processes as usual (maxed from the apache
> configs) and Apache was killing the oldest processes. This is not
> normal traffic, and I for sure thought I was either attacked or
> someone made a very bad script. Strangely enough, there are
> absolutely no sign of additional requests in the apache logfile. By
> looking at the file there are no more traffic at the time of the
> incident than in normal circumstances. There ain't no sign of a bad
> script (same source IP, same URI). So I'm supposing it was a DOS
> attack but can someone explain why it wouldn't show up in the logs.
> Is it that we recieved so many requests all at the same time and
> Apache wasn't able to process them ? The load average on my server
> went over 33 and the MySQL server was also quite busy (it is located
> on another server).
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org