You are viewing a plain text version of this content. The canonical link for it is here.

Posted to agent@nutch.apache.org by div div <di...@yahoo.co.uk> on 2007/12/01 15:47:04 UTC

carpages.co.uk - your robot does not seem to obay our robots.txt file

Hi

Can you please stop your robot indexing carpages.co.uk

Many Thanks

DiV


      ___________________________________________________________
Support the World Aids Awareness campaign this month with Yahoo! For Good http://uk.promotions.yahoo.com/forgood/

Re: carpages.co.uk - your robot does not seem to obay our robots.txt file

Posted by Pierre-Luc Bacon <pi...@aqra.ca>.

You are sending a message to a mailing list. Nutch is not a search
engine website but rather a software allowing people to build such
websites.

2007/12/1, div div <di...@yahoo.co.uk>:
> Hi
>
> Can you please stop your robot indexing carpages.co.uk
>
> Many Thanks
>
> DiV
>
>
>      ___________________________________________________________
> Support the World Aids Awareness campaign this month with Yahoo! For Good http://uk.promotions.yahoo.com/forgood/


-- 
Pierre-Luc Bacon
http://eurobot2008.aqra.ca