[jira] [Updated] (NUTCH-1335) OutlinkDB to collect unique URL's only

["Markus Jelsma (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Markus Jelsma updated NUTCH-1335:
---------------------------------

    Description: The aggregating code in the Outlink reducer does not take care of incoming duplicates. When the input segments contain duplicates of a single URL they are collected.  (was: The OutlinkDB may contain duplicates if a segment is added more than once. The aggregating code in the reducer is does not take care of removing duplicates.

See: http://mail-archives.apache.org/mod_mbox/nutch-user/201204.mbox/%3C39d7bed10f572c3211c3ad91c8a37145@openindex.io%3E)
     Patch Info: Patch Available
        Summary: OutlinkDB to collect unique URL's only  (was: OutlinkDB to emit unique URL's only)
    
> OutlinkDB to collect unique URL's only
> --------------------------------------
>
>                 Key: NUTCH-1335
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1335
>             Project: Nutch
>          Issue Type: Bug
>    Affects Versions: 1.5
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>             Fix For: 1.6
>
>
> The aggregating code in the Outlink reducer does not take care of incoming duplicates. When the input segments contain duplicates of a single URL they are collected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira