[jira] Commented: (NUTCH-501) Implement a different caching mechanism for objects cached in configuration

["Doğacan Güney (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505815 ] 

Doğacan Güney commented on NUTCH-501:
-------------------------------------

Thanks for the quick review, Andrzej.

I guess we can solve that problem by going down the PluginRepository way. We add a static WeakHashMap<Configuration, ObjectCache> to ObjectCache, make CACHE an instance member and add a static ObjectCache.get(Configuration) method. So one would use ObjectCache like:

ObjectCache.get(conf).getObject(...)

If that particular configuration instance does not have an associated ObjectCache, get will first create a new ObjectCache then put it in WeakHashMap. When that configuration is no longer used, GC will collect it.

As you know, this doesn't work for PluginRepository. New configurations instantiate new PluginRepository-s while, for some reason that I don't understand (I should really study how dynamic class loading works), old PluginRepository-s don't get collected. But this should not be a problem for ObjectCache, since whatever is not collected in ObjectCache will also be in a PluginRepository(thus, it wouldn't have been collected anyway) and everything else will be collected.

What do you think?

> Implement a different caching mechanism for objects cached in configuration
> ---------------------------------------------------------------------------
>
>                 Key: NUTCH-501
>                 URL: https://issues.apache.org/jira/browse/NUTCH-501
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 1.0.0
>            Reporter: Doğacan Güney
>             Fix For: 1.0.0
>
>         Attachments: NUTCH-501_draft.patch
>
>
> As per HADOOP-1343, Configuration.setObject and Configuration.getObject (which are used by Nutch to cache arbitrary objects) are deprecated and will be removed soon. We have to implement an alternative caching mechanism and replace all usages of Configuration.{getObject,setObject} with the new mechanism.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.