You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Stefan Groschupf (JIRA)" <ji...@apache.org> on 2006/11/25 11:40:05 UTC
[jira] Updated: (NUTCH-273) When a page is redirected, the original
url is NOT updated.
[ http://issues.apache.org/jira/browse/NUTCH-273?page=all ]
Stefan Groschupf updated NUTCH-273:
-----------------------------------
Priority: Blocker (was: Major)
I agree this is a serious problem for any production use of nutch - a blocker since you end up refetching again and again the same pages.
> When a page is redirected, the original url is NOT updated.
> -----------------------------------------------------------
>
> Key: NUTCH-273
> URL: http://issues.apache.org/jira/browse/NUTCH-273
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.8
> Environment: n/a
> Reporter: Lukas Vlcek
> Priority: Blocker
>
> [Excerpt from maillist, sender: Andrzej Bialecki]
> When a page is redirected, the original url is NOT updated - so, CrawlDB will never know that a redirect occured, it won't even know that a fetch occured... This looks like a bug.
> In 0.7 this was recorded in the segment, and then it would affect the Page status during updatedb. It should do so 0.8, too...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira