You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Michael Klama <no...@tampabay.rr.com> on 2003/02/18 05:43:36 UTC
[users@httpd] robots.txt
Can someone please tell me what robots.txt is and why I keep getting 404
errors for this file in my access and error logs?
Mike
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] robots.txt
Posted by Zac Stevens <zt...@cryptocracy.com>.
On Mon, Feb 17, 2003 at 11:43:36PM -0500, Michael Klama wrote:
> Can someone please tell me what robots.txt is and why I keep getting 404
> errors for this file in my access and error logs?
A couple of minutes on Google would have given you the answer, but take a
look at the following for a brief introduction and links to further
reading.
http://httpd.apache.org/docs/misc/howto.html#stoprob
Cheers,
Zac
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] robots.txt
Posted by Chris Meadors <cl...@hereintown.net>.
Michael Klama wrote:
> Can someone please tell me what robots.txt is and why I keep getting 404
> errors for this file in my access and error logs?
To quote, Dharmendra.T (who answered last time someone asked this exact
same question):
This link should help you:
http://www.searchengineworld.com/robots/robots_tutorial.htm
--
Chris
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] robots.txt
Posted by individual <in...@mi.cl>.
On Tuesday, February 18, 2003, at 01:43 AM, Michael Klama wrote:
> Can someone please tell me what robots.txt is and why I keep getting
> 404
> errors for this file in my access and error logs?
>
STFW.
Paul
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] robots.txt
Posted by Michael Klama <no...@tampabay.rr.com>.
Nevermind, I found the answer. Sorry for any inconvenience.
Mike
-----Original Message-----
From: Michael Klama [mailto:nosedive@tampabay.rr.com]
Sent: Monday, February 17, 2003 11:44 PM
To: users@httpd.apache.org
Subject: [users@httpd] robots.txt
Can someone please tell me what robots.txt is and why I keep getting 404
errors for this file in my access and error logs?
Mike
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org