You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Manuel Lemos <ml...@acm.org> on 2001/12/22 15:28:10 UTC
Anti-leech IP blocking
Hello,
I have a busy site that once in a while is accessed by users that want
to leech its content. That would not be a problem if that would not
cause the site to spend so much CPU resources to serve that content.
The only solution that I have for that is to block the IP address of the
leeching computers. That is not very efficient because my ISP warns me
before I can do anything.
I was wondering if is there any module or some other automatic way to
avoid this problem.
Thanks in advance,
Manuel Lemos
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: Anti-leech IP blocking
Posted by Daniel Lopez <da...@rawbyte.com>.
Hi Manuel,
Check the references at
http://www.linuxdoc.org/HOWTO/Apache-Overview-HOWTO-5.html
On Sat, Dec 22, 2001 at 12:28:10PM -0200, Manuel Lemos wrote:
> Hello,
>
> I have a busy site that once in a while is accessed by users that want
> to leech its content. That would not be a problem if that would not
> cause the site to spend so much CPU resources to serve that content.
>
> The only solution that I have for that is to block the IP address of the
> leeching computers. That is not very efficient because my ISP warns me
> before I can do anything.
>
> I was wondering if is there any module or some other automatic way to
> avoid this problem.
>
> Thanks in advance,
> Manuel Lemos
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: CGI Handling ErrorDocument 404/Redirect to external URI
Posted by Fred Koschara <wf...@L5Development.com>.
At 07:35 PM 12/24/01 -1000, Sivakatirswami <ka...@hindu.org> wrote:
>on 12/24/01 7:58 AM, Fred Koschara at wfredk@L5Development.com wrote:
>
> > In the HTTP protocol, the first strings returned by the server to the
> > browser are the "header" lines, describing what's being sent. The header
> > lines are terminated by a blank line, followed by the body of the
> > document. In the case of a "Location:" header, the document body is
> > empty: The server is telling the browser to go look at a different
> > location (hence the name). In some environments you will see a string
> such as
> > print "Content-type: text/html\n\n";
> > Here the server is sending a header line; indeed, the last one: The "\n\n"
> > is a pair of newlines - one ending the "Content-type" header line, the
> > second one being the blank line indicating the end of the headers. If you
> > see something like that, replace it with
> > print "Location: $URI\n\n";
> > where $URI is a variable containing the location you want to redirect to.
>
>OK, I did that, using Metacard CGI syntax (see below--this is a simple echo
>script which I have commented out and put the simple header in which is sent
>to standard out/i.e. back to the browser) And it works in the sense that I
>don't get any error message, the request is processed, but the browser tells
>me: "Document contains no data" so... I think there must be something else
>in that header besides "Location: $URI\n\n" to cause the browser to take it
>as a directive and not assume it is supposed to render the body, which is
>non-existent...So, there must be more to this "Location Header"
Sparing the details, PR number 4008 finally provided a clue: A "Status:"
header is required, in addition to the "Location:" one: Use a script similar
to the one shown here:
#!/usr/bin/perl
print "Status: 301 Moved Permanently\n";
print "Location: http://www.HimalayanAcademy.com/index.html\n\n";
You may want to use a different status code. For more information, see
ftp://ftp.isi.edu/in-notes/rfc2616.txt
Look at section 10.3, "Redirection 3xx" for the choices.
-- Fred Koschara, President
L5 Development Group
________________________________________________________________________
For private sector (commercial) space development, visit
http://www.L5Development.com
L5 Software Development - "out of this world" sites and software
http://www.L5Software.com
StudioLines.com - Your place on the Internet for local music
Music, feedback, connections. Tap the power of the Internet!
http://www.StudioLines.com
How much did your last traffic ticket cost you?
http://www.StopHighwayRobbery.com
ThmIndxr(tm), the *only* HTML thumbnail/indexer you need!
http://www.L5Software.com/go?ThmIndxr
wCapLock(tm), makes CapsLock work like it does on a typewriter
http://www.L5Software.com/go?wCapLock
KeywordGo(tm), provides keyword access to your popular pages
http://www.L5Software.com/go?KeywordGo
BannerAds(tm), join multiple affiliate programs with one banner
http://www.L5Software.com/go?BannerAds
My personal Web page is http://www.L5Development.com/wfredk
Stop by some time!
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: CGI Handling ErrorDocument 404/Redirect to external URI
Posted by Sivakatirswami <ka...@hindu.org>.
on 12/24/01 7:58 AM, Fred Koschara at wfredk@L5Development.com wrote:
> In the HTTP protocol, the first strings returned by the server to the
> browser are the "header" lines, describing what's being sent. The header
> lines are terminated by a blank line, followed by the body of the
> document. In the case of a "Location:" header, the document body is
> empty: The server is telling the browser to go look at a different
> location (hence the name). In some environments you will see a string such as
> print "Content-type: text/html\n\n";
> Here the server is sending a header line; indeed, the last one: The "\n\n"
> is a pair of newlines - one ending the "Content-type" header line, the
> second one being the blank line indicating the end of the headers. If you
> see something like that, replace it with
> print "Location: $URI\n\n";
> where $URI is a variable containing the location you want to redirect to.
>
OK, I did that, using Metacard CGI syntax (see below--this is a simple echo
script which I have commented out and put the simple header in which is sent
to standard out/i.e. back to the browser) And it works in the sense that I
don't get any error message, the request is processed, but the browser tells
me: "Document contains no data" so... I think there must be something else
in that header besides "Location: $URI\n\n" to cause the browser to take it
as a directive and not assume it is supposed to render the body, which is
non-existent...So, there must be more to this "Location Header"
#!/export/vhost/org/o/oursite/www/public_html/cgi-bin/mc
on startup
-- note the following is irrelevant, but shows how Metacard
-- is written... comments as #; command syntax as --
# This MetaTalk CGI script loops over all the environment variables
# set by the server when it runs a CGI application printing out
# its name and value.
# loop over all of the global variables, getting name and value
-- repeat for each item i in the globals
-- put i && "=" && value(i) & return after buffer
-- end repeat
## trap any data from the browser and add to ENV variables
-- read from stdin until empty
-- put it after buffer
# write minimal set of HTTP headers to stdout
-- put "Content-Type: text/plain" & cr
-- put "Content-Length:" && the length of buffer & cr & cr
-- put buffer
## the above is a minimal two line header for an HTML document
## and works fine.
## OK next is where I try to send a simple location header
## with no body back to the browser...
## it works to an extent,
## I don't get any server error, so the CGI is not breaking
## but the browser says
## Document contains no data... so I believe the header
## is missing something
## poke the URI
put "http://www.HimalayanAcademy.com/index.html" into newURL
## the following is directly equivalent to PERL:
## print "Location: $URI\n\n";
## but seems too simple... something is missing... what?
put "Location: " & newURL & cr & cr
end startup
======
Client browser responds: "Document contains no data"
Obviously.. the body is empty... so how to make the client/browser take this
as a directive to initiate another http GET seeking the new URI?
??
Hinduism Today
Sivakatirswami
Editor's Assistant/Production Manager
katir@hindu.org
www.HinduismToday.com, www.HimalayanAcademy.com,
www.Gurudeva.org, www.hindu.org
Read The Master Course Lesson of the Day at
http://www.gurudeva.org/lesson.shtml
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: CGI Handling ErrorDocument 404/Redirect to external URI
Posted by Fred Koschara <wf...@L5Development.com>.
At 05:25 AM 12/24/01 -1000, Sivakatirswami <ka...@hindu.org> wrote:
>on 12/23/01 8:32 PM, Fred Koschara at wfredk@L5Development.com wrote:
>
> > At 07:30 PM 12/23/01 -1000, Sivakatirswami <ka...@hindu.org> wrote:
> >> I want to use the CGI method of handling 404's and serving alternative
> pages
> >> or inform the client browser of the needful. FYI, I am a greenhorn
> >> Apache/CGI writer... but after study of the docs for a few hours I can get
> >> myself this far:
> >>
> >> 1) .htaccess file with this
> >>
> >> errorDocument 404 /cgi-bin/echo.mt
> >>
> >> [Note, I will be using Metacard scripts and not PERL script for the CGI
> >> work... it's a lot easier and more facile, but the issues are the same]
> >
> > You need to send a "Location:" header to the browser, which is then sent
> > back to the appropriate server by the client. Both Perl and PHP provide
> > similar mechanisms for generating headers (e.g., call header("Location:
> > $URI\n\n") in both languages.) If your Metacard scripts do not provide
> > such a mechanism, you will need to change your programming language.
>
>Aloha, (I'm in Hawaii) Fred:
>
>Thanks... well, as we xTalk buffs like to say "You can do anything with
>Metacard, easier" ... i.e. isn't anything, in this case a header, sent to a
>browser just simple a text string? Metacard does have whatever mechanism is
>needed, I just need a complete example of such a "Location:" header. I'll
>start searching...
In the HTTP protocol, the first strings returned by the server to the
browser are the "header" lines, describing what's being sent. The header
lines are terminated by a blank line, followed by the body of the
document. In the case of a "Location:" header, the document body is
empty: The server is telling the browser to go look at a different
location (hence the name). In some environments you will see a string such as
print "Content-type: text/html\n\n";
Here the server is sending a header line; indeed, the last one: The "\n\n"
is a pair of newlines - one ending the "Content-type" header line, the
second one being the blank line indicating the end of the headers. If you
see something like that, replace it with
print "Location: $URI\n\n";
where $URI is a variable containing the location you want to redirect to.
HTH
Happy Holidays :)
-- Fred Koschara, President
L5 Development Group
________________________________________________________________________
For private sector (commercial) space development, visit
http://www.L5Development.com
L5 Software Development - "out of this world" sites and software
http://www.L5Software.com
StudioLines.com - Your place on the Internet for local music
Music, feedback, connections. Tap the power of the Internet!
http://www.StudioLines.com
How much did your last traffic ticket cost you?
http://www.StopHighwayRobbery.com
ThmIndxr(tm), the *only* HTML thumbnail/indexer you need!
http://www.L5Software.com/go?ThmIndxr
wCapLock(tm), makes CapsLock work like it does on a typewriter
http://www.L5Software.com/go?wCapLock
KeywordGo(tm), provides keyword access to your popular pages
http://www.L5Software.com/go?KeywordGo
BannerAds(tm), join multiple affiliate programs with one banner
http://www.L5Software.com/go?BannerAds
My personal Web page is http://www.L5Development.com/wfredk
Stop by some time!
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: CGI Handling ErrorDocument 404/Redirect to external URI
Posted by Fred Koschara <wf...@L5Development.com>.
At 07:30 PM 12/23/01 -1000, Sivakatirswami <ka...@hindu.org> wrote:
>I want to use the CGI method of handling 404's and serving alternative pages
>or inform the client browser of the needful. FYI, I am a greenhorn
>Apache/CGI writer... but after study of the docs for a few hours I can get
>myself this far:
>
>1) .htaccess file with this
>
>errorDocument 404 /cgi-bin/echo.mt
>
>[Note, I will be using Metacard scripts and not PERL script for the CGI
>work... it's a lot easier and more facile, but the issues are the same]
You need to send a "Location:" header to the browser, which is then sent
back to the appropriate server by the client. Both Perl and PHP provide
similar mechanisms for generating headers (e.g., call header("Location:
$URI\n\n") in both languages.) If your Metacard scripts do not provide
such a mechanism, you will need to change your programming language.
>2) My CGI dutifully echoes back all the environment variable that Apache now
>delivers with the errorDocument directive, which is wonderful.
>
>3) It is very easy to parse the variable name/value pairs, in particular
>
>$REDIRECT_URL = /foo/blooper.html ##this being the 404 requested doc.
>
>4) Now, it's a simple matter to read the correct *.html file from the server
>and send that to standardOut and the client browser will be looking at a
>proper HTML page and not get a 404...but... this is the tricky part
>
>5) How do I redirect to an external URI? From what I read in the "voodoo" of
>mod_alias/rewrite directive... it appears to me that I have to "re-inject"
>something back to the Apache kernal if I want Apache to send something to
>the client browser which tells the client browser to go fetch a document at
>another domain no longer on this server...
>
>Goal is: to make the CGI work so that robots and spiders will slowly log in
>the permanently moved pages and after a month or two requests to URI's on
>this server that no longer exist (paths were restructured... content split
>out to several other domains with separate public_html folders etc.)
>
>so the cgi has to do something like this
>
>redirect /foo/nolongerExists.html http://www.HinduismToday/index.html
>
>but, in this case I will create an array that the CGI will access of of the
>path URI's as keys and the new URI's to go to as the elements for each
>key... having thus a variable with the new URI (some of which will be on
>the same machine and same domain, and other which will be external full
>URI's... in any case, suffice it to say that for every bad URI I will be
>able to generate the correct URI... what do I do with it then?
>
>I am happy to do the home work, just point me in the right direction... but
>I suspect I will have open some apache process and send some command back to
>Apache with the new URI... how would that be done??
-- Fred Koschara, President
L5 Development Group
________________________________________________________________________
For private sector (commercial) space development, visit
http://www.L5Development.com
L5 Software Development - "out of this world" sites and software
http://www.L5Software.com
StudioLines.com - Your place on the Internet for local music
Music, feedback, connections. Tap the power of the Internet!
http://www.StudioLines.com
How much did your last traffic ticket cost you?
http://www.StopHighwayRobbery.com
ThmIndxr(tm), the *only* HTML thumbnail/indexer you need!
http://www.L5Software.com/go?ThmIndxr
wCapLock(tm), makes CapsLock work like it does on a typewriter
http://www.L5Software.com/go?wCapLock
KeywordGo(tm), provides keyword access to your popular pages
http://www.L5Software.com/go?KeywordGo
BannerAds(tm), join multiple affiliate programs with one banner
http://www.L5Software.com/go?BannerAds
My personal Web page is http://www.L5Development.com/wfredk
Stop by some time!
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
CGI Handling ErrorDocument 404/Redirect to external URI
Posted by Sivakatirswami <ka...@hindu.org>.
I want to use the CGI method of handling 404's and serving alternative pages
or inform the client browser of the needful. FYI, I am a greenhorn
Apache/CGI writer... but after study of the docs for a few hours I can get
myself this far:
1) .htaccess file with this
errorDocument 404 /cgi-bin/echo.mt
[Note, I will be using Metacard scripts and not PERL script for the CGI
work... it's a lot easier and more facile, but the issues are the same]
2) My CGI dutifully echoes back all the environment variable that Apache now
delivers with the errorDocument directive, which is wonderful.
3) It is very easy to parse the variable name/value pairs, in particular
$REDIRECT_URL = /foo/blooper.html ##this being the 404 requested doc.
4) Now, it's a simple matter to read the correct *.html file from the server
and send that to standardOut and the client browser will be looking at a
proper HTML page and not get a 404...but... this is the tricky part
5) How do I redirect to an external URI? From what I read in the "voodoo" of
mod_alias/rewrite directive... it appears to me that I have to "re-inject"
something back to the Apache kernal if I want Apache to send something to
the client browser which tells the client browser to go fetch a document at
another domain no longer on this server...
Goal is: to make the CGI work so that robots and spiders will slowly log in
the permanently moved pages and after a month or two requests to URI's on
this server that no longer exist (paths were restructured... content split
out to several other domains with separate public_html folders etc.)
so the cgi has to do something like this
redirect /foo/nolongerExists.html http://www.HinduismToday/index.html
but, in this case I will create an array that the CGI will access of of the
path URI's as keys and the new URI's to go to as the elements for each
key... having thus a variable with the new URI (some of which will be on
the same machine and same domain, and other which will be external full
URI's... in any case, suffice it to say that for every bad URI I will be
able to generate the correct URI... what do I do with it then?
I am happy to do the home work, just point me in the right direction... but
I suspect I will have open some apache process and send some command back to
Apache with the new URI... how would that be done??
Hinduism Today
Sivakatirswami
Editor's Assistant/Production Manager
katir@hindu.org
www.HinduismToday.com, www.HimalayanAcademy.com,
www.Gurudeva.org, www.hindu.org
Read The Master Course Lesson of the Day at
http://www.gurudeva.org/lesson.shtml
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org