You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Xiaolong Yang <ya...@gmail.com> on 2012/04/10 05:00:53 UTC
question about ObjectCache
Hi，all
I'm reading source code of nutch and I have some puzzled about the
ObjectCache.java in package org.apache.nutch.util.I just find it may be
little benefit to use it in urlnormalizers and urlfiters.I also have read
some discuss about cache in Nutch-169 and Nutch-501.But I can't understand
it.
Can anyone tell me where ObjectCache be used and get a good benefit
in nutch ?
Appreciate your time and help.
Re: question about ObjectCache
Posted by Xiaolong Yang <ya...@gmail.com>.
Hi,Andrzej
Thank you for your detail answers.I have understood it uses.[?]
2012/4/10 Andrzej Bialecki <ab...@getopt.org>
> On 10/04/2012 05:00, Xiaolong Yang wrote:
>
>> Hi，all
>>
>> I'm reading source code of nutch and I have some puzzled about the
>> ObjectCache.java in package org.apache.nutch.util.I just find it may be
>> little benefit to use it in urlnormalizers and urlfiters.I also have
>> read some discuss about cache in Nutch-169 and Nutch-501.But I can't
>> understand it.
>>
>> Can anyone tell me where ObjectCache be used and get a good benefit in
>> nutch ?
>>
>
> ObjectCache is designed to cache ready-to-use instances of Nutch plugins.
> The process of finding, instantiating and initializing plugins is
> inefficient, because it involves parsing plugin descriptors, initializing
> plugins, collecting the ones that implement correct extension points, etc.
>
> It would kill performance if this process were invoked each time you want
> to run all plugins of a given type (e.g. URLNormalizer-s). The facade
> URLNormalizers/URLFilters and others make sure that plugin instances of a
> given type are initialized once per lifetime of a JVM, and then they are
> cached in ObjectCache, so that next time you want to use them they can be
> retrieved from a cache, instead of going again through the process of
> parsing/instantiating/**initializing.
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ ______________________________**____
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
Re: question about ObjectCache
Posted by Andrzej Bialecki <ab...@getopt.org>.
On 10/04/2012 05:00, Xiaolong Yang wrote:
> Hi，all
>
> I'm reading source code of nutch and I have some puzzled about the
> ObjectCache.java in package org.apache.nutch.util.I just find it may be
> little benefit to use it in urlnormalizers and urlfiters.I also have
> read some discuss about cache in Nutch-169 and Nutch-501.But I can't
> understand it.
>
> Can anyone tell me where ObjectCache be used and get a good benefit in
> nutch ?
ObjectCache is designed to cache ready-to-use instances of Nutch
plugins. The process of finding, instantiating and initializing plugins
is inefficient, because it involves parsing plugin descriptors,
initializing plugins, collecting the ones that implement correct
extension points, etc.
It would kill performance if this process were invoked each time you
want to run all plugins of a given type (e.g. URLNormalizer-s). The
facade URLNormalizers/URLFilters and others make sure that plugin
instances of a given type are initialized once per lifetime of a JVM,
and then they are cached in ObjectCache, so that next time you want to
use them they can be retrieved from a cache, instead of going again
through the process of parsing/instantiating/initializing.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com