You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "byron miller (JIRA)" <ji...@apache.org> on 2005/04/20 20:08:23 UTC

[jira] Commented: (NUTCH-13) If dns points to 127.0.0.1, the url is also crawled

     [ http://issues.apache.org/jira/browse/NUTCH-13?page=comments#action_63305 ]
     
byron miller commented on NUTCH-13:
-----------------------------------

Could 127.0.0.1 & localhost be added to the urlfilter as a default?

> If dns points to 127.0.0.1, the url is also crawled
> ---------------------------------------------------
>
>          Key: NUTCH-13
>          URL: http://issues.apache.org/jira/browse/NUTCH-13
>      Project: Nutch
>         Type: Bug
>   Components: fetcher
>     Reporter: Matthias Jaekle
>     Priority: Minor

>
> For example www.tik24.de points to 127.0.0.1.
> If you follow a link to www.tik24.de fetcher will crawl content from your own machine.
> Wrong DNS entries could create unwanted entries in segments.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira