You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by mrl <mr...@psfc.mit.edu> on 2013/07/12 17:34:22 UTC
[users@httpd] Can I block.crawlers from seeing files with certain suffixes, i.e. .php? Thanks.
Is there a way to block .php from being indexed by crawlers, but allow other
type files to be indexed? When the crawlers access the php files, they are
executed, creating lots of error messages (and taking up cpu cycles). Thanks.
- Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Can I block.crawlers from seeing files with
certain suffixes, i.e. .php? Thanks.
Posted by "D'Arcy J.M. Cain" <da...@Vex.Net>.
On Fri, 12 Jul 2013 15:34:22 +0000 (UTC)
mrl <mr...@psfc.mit.edu> wrote:
> Is there a way to block .php from being indexed by crawlers, but
> allow other type files to be indexed? When the crawlers access the
> php files, they are executed, creating lots of error messages (and
> taking up cpu cycles). Thanks.
Google for "robots.txt".
--
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:darcy@Vex.Net
Voip: sip:darcy@Vex.Net
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org