[jira] Reopened: (NUTCH-855) ScoringFilter and IndexingFilter: To allow for the propagation of URL Metatags and their subsequent indexing.

["Scott Gonyea (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gonyea reopened NUTCH-855:
--------------------------------


If it wasn't clear from my prior comment, the property for urlmeta in nutch-site should look like:
<property>
<name>urlmeta.tags</name>
<value>damn,you,doug,cutting</value>
</property>

It might be nice if someone updates the "nutch-default.xml" entry for "urlmeta.tags" to the following:

<property>
<name>urlmeta.tags</name>
<value></value>
<description>
To be used in conjunction with features introduced in NUTCH-655, which allows
for custom metatags to be injected alongside your crawl URLs. Specifying those
custom tags here will allow for their propagation into a pages outlinks, as
well as allow for them to be included as part of an index.
Values should be comma-delimited. ("tag1,tag2,tag3") Do not pad the tags with
white-space at their boundaries, if you are using anything earlier than Hadoop-0.21.
</description>
</property>

Unless, of course, Nutch-1.2 ships with Hadoop-0.21... Then it's a wash. I do think it's good to note that in there, as someone may stumble across that tidbit while troubleshooting some unrelated godawful bug. I'm looking out for you, long lost not-twin.


> ScoringFilter and IndexingFilter: To allow for the propagation of URL Metatags and their subsequent indexing.
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: NUTCH-855
>                 URL: https://issues.apache.org/jira/browse/NUTCH-855
>             Project: Nutch
>          Issue Type: New Feature
>          Components: generator, indexer
>    Affects Versions: 1.1
>            Reporter: Scott Gonyea
>            Assignee: Chris A. Mattmann
>             Fix For: 1.2
>
>         Attachments: nutch-855.txt
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> This plugin is designed to enhance the NUTCH-655 patch, by doing two things:
> 1. Meta Tags that are supplied with your Crawl URLs, during injection, will be propagated throughout the outlinks of those Crawl URLs.
> 2. When you index your URLs, the meta tags that you specified with your URLs will be indexed alongside those URLs--and can be directly queried, assuming you have done everything else correctly.
> The flat-file of URLs you are injecting should, per NUTCH-655, be tab-delimited in the form of:
> www.url.com\tkey1=value1\tkey2=value2\t...\tkeyN=valueN
> or:
> http://slashdot.org/	corp_owner=Geeknet	will_it_blend=indubitably
> http://engadget.com/	corp_owner=Weblogs	genre=geeksquad_thriller
> To activate this plugin, you must modify two properties in your nutch-sites.xml:
> 1. plugin.includes
>    add: urlmeta
>    to:   <value>...</value>
>    ie: <value>urlmeta|parse-tika|scoring-opic|...</value>
> 2. urlmeta.tags
>    Insert a comma-delimited list of metatags. Using the above example:
>    <value>corp_owner, will_it_blend, genre</value>
>    Note that you do not need to include the tag with every URL. However, you must specify each tag if you want it to be propagated and later indexed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.