[jira] [Commented] (KYLIN-1843) LRU cache for in memory dimensions

["Abhilash L L (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454248#comment-15454248 ] 

Abhilash L L commented on KYLIN-1843:
-------------------------------------

There needs to two levels of caching ?

1) Number of entries of each dimension to hold in memory
- There will be a hit when all the values for the dimension is required. Would be nice if this can be tuned based on size in MB or number of records. Former is better as if its based on number of rows, it can grow to any size depending on the number of columns and column avg size

2) Number of dimension to hold in memory
- Dimensions frequently involved in queries would reside in memory

> LRU cache for in memory dimensions
> ----------------------------------
>
>                 Key: KYLIN-1843
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1843
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Query Engine
>    Affects Versions: v1.2, v1.5.2
>            Reporter: Abhilash L L
>            Assignee: Yiming Liu
>         Attachments: KYLIN-1843.patch
>
>
> The dimensions are kept in memory to fill up the result set.
> Now as and when more cubes are created, all the dimensions keep accumulating in the memory
> This means tomcat memory requirements always keep increasing
> It would help if less frequently used dimensions are swapped out to keep memory in check.
> We can have a Cache Manager which gets to decide which dimensions are kept in memory and brings in as and when its required. The size of the cache should be tuneable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)