You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by Juan Jose Pablos <ch...@che-che.com> on 2004/07/20 17:46:52 UTC
Re: svn commit: rev 23083 - in forrest/trunk/src/forrestbot/webapp/src/webapp:
. common
brondsem@apache.org escribió:
> Author: brondsem
> Date: Tue Jul 20 05:40:54 2004
> New Revision: 23083
>
> Modified:
> forrest/trunk/src/forrestbot/webapp/src/webapp/common/layout.jsp
> forrest/trunk/src/forrestbot/webapp/src/webapp/common/simple.jsp
> forrest/trunk/src/forrestbot/webapp/src/webapp/common/viewlog_frame.jsp
> forrest/trunk/src/forrestbot/webapp/src/webapp/viewlog_header.jsp
> Log:
> robot meta tags to discourage indexing the generated websites
what about a robots.txt file?
Re: svn commit: rev 23083 - in forrest/trunk/src/forrestbot/webapp/src/webapp:
. common
Posted by Dave Brondsema <da...@brondsema.net>.
On Tue, 20 Jul 2004, Juan Jose Pablos wrote:
> brondsem@apache.org escribió:
> > Author: brondsem
> > Date: Tue Jul 20 05:40:54 2004
> > New Revision: 23083
> >
> > Modified:
> > forrest/trunk/src/forrestbot/webapp/src/webapp/common/layout.jsp
> > forrest/trunk/src/forrestbot/webapp/src/webapp/common/simple.jsp
> > forrest/trunk/src/forrestbot/webapp/src/webapp/common/viewlog_frame.jsp
> > forrest/trunk/src/forrestbot/webapp/src/webapp/viewlog_header.jsp
> > Log:
> > robot meta tags to discourage indexing the generated websites
>
> what about a robots.txt file?
>
I thought about that, but that only works at domain.com/robots.txt and not
at a directory like: domaind.com/forrestbot/robots.txt. Meta tags are not
as widely supported as robots.txt but googlebot, msnbot and inktomi slurp
support it so I think that's good enough.
--
Dave Brondsema : dave@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
http://csx.calvin.edu : student org