[jira] Commented: (NUTCH-468) Scoring filter should distribute score to all outlinks at once

["Nicolás Lichtmaier (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491051 ] 

Nicolás Lichtmaier commented on NUTCH-468:
------------------------------------------

This patch would be useful to me.

Just one very minor thing:

Here:

-  public CrawlDatum distributeScoreToOutlink(Text fromUrl, Text toUrl, ParseData parseData, CrawlDatum target, CrawlDatum adjust, int allCount, int validCount) throws ScoringFilterException {
+  public CrawlDatum distributeScoreToOutlinks(Text fromUrl, ParseData parseData, List<Entry<Text, CrawlDatum>> targets, CrawlDatum adjust, int allCount) throws ScoringFilterException {

Why don't you just use Collection instead of List? It's enough for iterating and could, in the future, be passed a Set of entries (as Map provides).


> Scoring filter should distribute score to all outlinks at once
> --------------------------------------------------------------
>
>                 Key: NUTCH-468
>                 URL: https://issues.apache.org/jira/browse/NUTCH-468
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 1.0.0
>            Reporter: Doğacan Güney
>            Priority: Minor
>             Fix For: 1.0.0
>
>         Attachments: scoring.patch
>
>
> Currently ScoringFilter.distributeScoreToOutlink, as its name implies, takes only a single outlink and works on that. I would suggest that we change it to distributeScoreToOutlink_s_ so that it would take all the outlinks of a page at once. This has several advantages:
> 1) A ScoringFilter plugin returns a single adjust datum to set its score instead of returning several.
> 2) A ScoringFilter plugin can change the score of the original page (via adjust datum) even if there are no outlinks. This is useful if you have a ScoringFilter plugin that, say, scores pages based on content instead of outlinks.
> 3) Since the ScoringFilter plugin recieves all outlinks at once, it can make better decisions on how to distribute the score. For example, right now it is not possible to create a plugin that always distributes exactly a page's 'cash' to outlinks(that is, if a page has score 5, it will always distribute exactly 5 points to its outlinks no matter what the internal/external factors are) if internal / external score factors are not 1.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.