[jira] [Commented] (CALCITE-1808) JaninoRelMetadataProvider loading cache might cause OOM error

["Jesus Camacho Rodriguez (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026138#comment-16026138 ] 

Jesus Camacho Rodriguez commented on CALCITE-1808:
--------------------------------------------------

I think one possible solution is to use _maximumSize_ when creating the cache, since most systems will have an idea on the maximum number of queries that will be planned concurrently, and make that number configurable.

Another possible better solution might be linking the cache to the query planning cycle. However, I am not sure if that would imply major changes in that code. Or would it be as simple as including the _RelOptCluster_ object in the key of the cache? [~julianhyde], what do you think?

> JaninoRelMetadataProvider loading cache might cause OOM error
> -------------------------------------------------------------
>
>                 Key: CALCITE-1808
>                 URL: https://issues.apache.org/jira/browse/CALCITE-1808
>             Project: Calcite
>          Issue Type: Bug
>          Components: core
>            Reporter: Jagruti Varia
>            Assignee: Julian Hyde
>         Attachments: calcite-retained-strong-refs.png
>
>
> JaninoRelMetadataProvider has a static handler cache with size increasing over time for long running HS2, ending up causing OOM errors.
> {code:java}
>   /** Cache of pre-generated handlers by provider and kind of metadata.
>    * For the cache to be effective, providers should implement identity
>    * correctly. */
>   private static final LoadingCache<Key, MetadataHandler> HANDLERS =
>       CacheBuilder.newBuilder().build(
>           new CacheLoader<Key, MetadataHandler>() {
>             public MetadataHandler load(@Nonnull Key key) {
>               //noinspection unchecked
>               return load3(key.def, key.provider.handlers(key.def),
>                   key.relClasses);
>             }
>           });
> ...
>   /** Key for the cache. */
>   private static class Key {
>     public final MetadataDef def;
>     public final RelMetadataProvider provider;
>     public final ImmutableList<Class<? extends RelNode>> relClasses;
> ...
> {code}
> The lifecycle for providers is per query and we have multiple providers instantiated on the lifecycle of a query. The entries are retained in the cache even when query planning has finished.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)