Re: [Nutch-dev] Re: Starting a non-profit organisation running Nutch with a thousand or more sponsored servers

[David Spencer <da...@tropo.com>]

Michael Wechner wrote:

> Stefan Groschupf wrote:
> 
>>> have you collected these offers somewhere?
>>
>>
>> Check the source-forge mail archive.
> 
> 
> 
> thanks, will do.
> 
> btw, is there an interface within Nutch, where a CMS (e.g. Apache Lenya) 
> can notify Nutch about content changes (or deletion of pages or renaming 
> of URLs)?


I'm new to Nutch and haven't tried this yet, but I've had the same kind 
of question.

I think the answer is: IWebDBWriter.addPageIfNotPresent()
or maybe just addPage() in your case, for updates.

http://nutch.sourceforge.net/docs/api/net/nutch/db/IWebDBWriter.html#addPageIfNotPresent(net.nutch.db.Page)

See WebDBInjector for sample usage, esp as now you have to create a Page 
to get this call to work right.



> 
> I guess this would make crawling "obsolete" to a certain point (at least 
> for pages
> being created by content management systems).
> 
> Thanks
> 
> Michi
> 
>>
>>
> 
> 

Notifying Nutch about content changes [WAS: Re: [Nutch-dev] Re: Starting a non-profit organisation running Nutch with a thousand or more sponsored servers]

[Michael Wechner <mi...@wyona.com>]

David Spencer wrote:

> Michael Wechner wrote:
>
>> Stefan Groschupf wrote:
>>
>>>> have you collected these offers somewhere?
>>>
>>>
>>>
>>> Check the source-forge mail archive.
>>
>>
>>
>>
>> thanks, will do.
>>
>> btw, is there an interface within Nutch, where a CMS (e.g. Apache 
>> Lenya) can notify Nutch about content changes (or deletion of pages 
>> or renaming of URLs)?
>
>
>
> I'm new to Nutch and haven't tried this yet, but I've had the same 
> kind of question.
>
> I think the answer is: IWebDBWriter.addPageIfNotPresent()
> or maybe just addPage() in your case, for updates.
>
> http://nutch.sourceforge.net/docs/api/net/nutch/db/IWebDBWriter.html#addPageIfNotPresent(net.nutch.db.Page) 
>
>
> See WebDBInjector for sample usage, esp as now you have to create a 
> Page to get this call to work right.


thanks very much for the hint

Michi


-- 
Michael Wechner
Wyona Inc.  -   Open Source Content Management   -   Apache Lenya
http://www.wyona.com                      http://lenya.apache.org
michael.wechner@wyona.com                        michi@apache.org