You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Lewis John McGibbney (JIRA)" <ji...@apache.org> on 2013/01/12 21:12:14 UTC
[jira] [Updated] (NUTCH-690) bug in
DomContentUtils.shouldThrowAwayLink?
[ https://issues.apache.org/jira/browse/NUTCH-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-690:
---------------------------------------
Fix Version/s: 1.7
> bug in DomContentUtils.shouldThrowAwayLink?
> -------------------------------------------
>
> Key: NUTCH-690
> URL: https://issues.apache.org/jira/browse/NUTCH-690
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Peter Sparks
> Priority: Minor
> Fix For: 1.7
>
>
> I found a potential bug in DomContentUtils.shouldThrowAwayLink. It returns true for the 5 links at the top of the home page for www.aksteel.com. Here are the links in the source:
> <a id="Search" href="/search/default.aspx"></a>
> <a id="Investor" style="height: 15px;" href="/investor_information/"></a>
> <a id="Markets" href="/markets_products/"></a>
> <a id="Production" href="/production_facilities/"></a>
> <a id="News" href="/news/"></a>
> Perhaps I am just ignorant of what this function is supposed to do but returning true for these 5 links on that site make that site impossible to crawl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira