You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Stefan Groschupf (JIRA)" <ji...@apache.org> on 2006/01/31 01:30:33 UTC

[jira] Updated: (NUTCH-192) meta data support for CrawlDatum

     [ http://issues.apache.org/jira/browse/NUTCH-192?page=all ]

Stefan Groschupf updated NUTCH-192:
-----------------------------------

    Attachment: metadata300106.patch

Attached a first suggestion for a patch to adding meta data support into crawlDatum. 
In general I created a MapWritable and add this to the CrawlDatum. If no meta data are added to CrawlDatum there will be only one more int written to the output stream. The MapWritable works like a HashMap but requre Writables as key and value. Beside the key and the value size it writes two addition int's into the stream to identify the classes of  key and value. If we may be more change the WritableName we can minimize that to two addidtional bytes for storing classes (this would limit us but i guess we will neve so mache writable object types. :-o). However I started with a patch that changes as less as possible and I'm sure there is space for improvements. So feedback and improvement suggestions are welcome.



> meta data support for CrawlDatum
> --------------------------------
>
>          Key: NUTCH-192
>          URL: http://issues.apache.org/jira/browse/NUTCH-192
>      Project: Nutch
>         Type: Improvement
>     Versions: 0.8-dev
>     Reporter: Stefan Groschupf
>      Fix For: 0.8-dev
>  Attachments: metadata300106.patch
>
> Supporting meta data in CrawlDatum would help to get a set of new nutch features realized and makes a lot possible to smaller special focused search engines.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira