[jira] [Updated] (IMPALA-6354) Consider using Guava LoadingCache to cache metadata objects opposed to a ConcurrentHashMap

["Tim Armstrong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong updated IMPALA-6354:
----------------------------------
    Component/s: Catalog

> Consider using Guava LoadingCache to cache metadata objects opposed to a ConcurrentHashMap
> ------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-6354
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6354
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>            Reporter: Mostafa Mokhtar
>            Assignee: Vuk Ercegovac
>            Priority: Critical
>              Labels: catalog, catalog-server, scalability
>
> Look into replacing the ConcurrentHashMap(s) used in the Catalog service to cache metadata with [LoadingCache|https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html].
> Caching metadata using a LoadingCache will allow:
> * Better coherency by adding expireAfter and refreshAfter clause to the cache
> * Weak reference eviction to respond to garbage collection
> * Putting a cap on maximum number of caches entires to avoid OOMs and excessive GC
> * Assigning different weights for each entry (For more efficient eviction)
> https://github.com/google/guava/wiki/CachesExplained
> https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/CacheBuilder.html
> https://google.github.io/guava/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org