You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sami Siren (JIRA)" <ji...@apache.org> on 2006/09/07 20:14:25 UTC
[jira] Commented: (NUTCH-273) When a page is redirected, the
original url is NOT updated.
[ http://issues.apache.org/jira/browse/NUTCH-273?page=comments#action_12433183 ]
Sami Siren commented on NUTCH-273:
----------------------------------
+1 for not following redirects immediately - simplify fetcher logic.
I would also like to see a flexible (configurable?) solution not a one size fits all because there's conflicting requirements (or atleast opinions) around this topic.
> When a page is redirected, the original url is NOT updated.
> -----------------------------------------------------------
>
> Key: NUTCH-273
> URL: http://issues.apache.org/jira/browse/NUTCH-273
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.8
> Environment: n/a
> Reporter: Lukas Vlcek
>
> [Excerpt from maillist, sender: Andrzej Bialecki]
> When a page is redirected, the original url is NOT updated - so, CrawlDB will never know that a redirect occured, it won't even know that a fetch occured... This looks like a bug.
> In 0.7 this was recorded in the segment, and then it would affect the Page status during updatedb. It should do so 0.8, too...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira