You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Linh Pham (JIRA)" <ji...@apache.org> on 2007/04/27 23:25:15 UTC
[jira] Created: (NUTCH-476) Would like to add a field to the
document class for its MD5 signature
Would like to add a field to the document class for its MD5 signature
----------------------------------------------------------------------
Key: NUTCH-476
URL: https://issues.apache.org/jira/browse/NUTCH-476
Project: Nutch
Issue Type: Improvement
Components: indexer
Environment: all
Reporter: Linh Pham
Priority: Minor
During indexing a file, if an MD5 signature was calculated and stored along with the document as a default,
it could then be used to remove duplicates from the results on retrieval.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-476) Would like to add a field to the
document class for its MD5 signature
Posted by "Sami Siren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494537 ]
Sami Siren commented on NUTCH-476:
----------------------------------
md5 sum (or any other configurable "digest") is already calculated in fetcher or parser and dedup can be used to remove duplicates.
> Would like to add a field to the document class for its MD5 signature
> ----------------------------------------------------------------------
>
> Key: NUTCH-476
> URL: https://issues.apache.org/jira/browse/NUTCH-476
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Environment: all
> Reporter: Linh Pham
> Priority: Minor
>
> During indexing a file, if an MD5 signature was calculated and stored along with the document as a default,
> it could then be used to remove duplicates from the results on retrieval.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (NUTCH-476) Would like to add a field to the
document class for its MD5 signature
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doğacan Güney closed NUTCH-476.
-------------------------------
Resolution: Invalid
Such a field is already stored in index (as "digest"). You can change how it is calculated by db.signature.class option.
> Would like to add a field to the document class for its MD5 signature
> ----------------------------------------------------------------------
>
> Key: NUTCH-476
> URL: https://issues.apache.org/jira/browse/NUTCH-476
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Environment: all
> Reporter: Linh Pham
> Priority: Minor
>
> During indexing a file, if an MD5 signature was calculated and stored along with the document as a default,
> it could then be used to remove duplicates from the results on retrieval.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.