Populating outlinks with CrawlDatum Metadata

[Lewis John Mcgibbney <le...@gmail.com>]

Hi Folks,
The above has been discussed a few times with the following thread [0]
being probably most helpful.
Is anyone else working with another mechanism for populating CrawlDatum
Metadata to Outlinks?
I thought we (Julien) had implemented this feature and it was a case of
turning it on within nutch-site.xml?
We are currently looking into implementing a feature for adaptive protocol
invocation based on CrawlDatum Metadata key describing which protocol
plugin should be called for which URL. Of particular interest is when to
utilize protocol-selenium over protocol-http.

Any convo is much appreciated.
Thanks

[0] http://lucene.472066.n3.nabble.com/Outlink-with-metadata-td4134243.html

-- 
*Lewis*

Re: Populating outlinks with CrawlDatum Metadata

[Julien Nioche <li...@gmail.com>]

Hi Lewis

Can't you achieve this already with the url-meta plugin and urlmeta.tags
config?

Julien



On 3 November 2015 at 22:09, Lewis John Mcgibbney <lewis.mcgibbney@gmail.com
> wrote:

> Hi Folks,
> The above has been discussed a few times with the following thread [0]
> being probably most helpful.



> Is anyone else working with another mechanism for populating CrawlDatum
> Metadata to Outlinks?
> I thought we (Julien) had implemented this feature and it was a case of
> turning it on within nutch-site.xml?

We are currently looking into implementing a feature for adaptive protocol
> invocation based on CrawlDatum Metadata key describing which protocol
> plugin should be called for which URL. Of particular interest is when to
> utilize protocol-selenium over protocol-http.
>
> Any convo is much appreciated.
> Thanks
>
> [0]
> http://lucene.472066.n3.nabble.com/Outlink-with-metadata-td4134243.html
>
> --
> *Lewis*
>



-- 

*Open Source Solutions for Text Engineering*

http://www.digitalpebble.com
http://digitalpebble.blogspot.com/
#digitalpebble <http://twitter.com/digitalpebble>