[jira] Updated: (NUTCH-230) OPIC score for outlinks should be based on # of valid links, not total # of links.

["Andrzej Bialecki (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/NUTCH-230?page=all ]

Andrzej Bialecki  updated NUTCH-230:
------------------------------------

    Attachment: patch.txt

Please review this patch, if it's ok I'll commit it.

> OPIC score for outlinks should be based on # of valid links, not total # of links.
> ----------------------------------------------------------------------------------
>
>          Key: NUTCH-230
>          URL: http://issues.apache.org/jira/browse/NUTCH-230
>      Project: Nutch
>         Type: Improvement
>     Versions: 0.8-dev
>     Reporter: Ken Krugler
>     Priority: Minor
>  Attachments: patch.txt
>
> In ParseOutputFormat.java, the write() method currently divides the page score by the # of outlinks:
>           score /= links.length;
> It then loops over the links, and any that pass the normalize/filter gauntlet get added to the crawl output.
> But this means that any filtered links result in some amount of the page's OPIC score being "lost".
> For Nutch 0.7, I built a list of valid (post-filter) links, and then used that to determine the per-link OPIC score, after which I iterated over the list, adding entries to the crawl output.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira