You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by MoD <w...@ant.com> on 2009/10/14 03:33:23 UTC

Why this domain isn't fetched

Hi All,


I'm trying to crawl http://www.tvshack.net
why nutch default configuration doesn't crawl it ?

I change the robots rules to only my bot
http.robots.agents is set to "GoogleBot"

I tryed also with and without '*'

Any idea ?

Regards,
Louis