You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2016/07/08 07:38:11 UTC

[jira] [Updated] (KYLIN-1844) Hold huge dictionary in 2nd storage like disk/hbase

     [ https://issues.apache.org/jira/browse/KYLIN-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

liyang updated KYLIN-1844:
--------------------------
    Summary: Hold huge dictionary in 2nd storage like disk/hbase  (was: High cardinality dimensions in memory)

> Hold huge dictionary in 2nd storage like disk/hbase
> ---------------------------------------------------
>
>                 Key: KYLIN-1844
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1844
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Query Engine
>    Affects Versions: v1.2, v1.5.2
>            Reporter: Abhilash L L
>            Assignee: liyang
>
> A whole dimension is kept in memory.
> We should have a way to keep only certain number / size of total rows to be kept in memory. A LRU cache for rows in the dimension will help keep memory in check.
> Why not store all the dimensions data in hbase in a different table with a prefix of dimensionid, and all calls to the dimensions (get based on dim key), is mapped to hbase.
> This does mean it will cost more time on a miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)