You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kylin.apache.org by "Abhilash L L (JIRA)" <ji...@apache.org> on 2016/07/01 06:51:11 UTC

[jira] [Created] (KYLIN-1844) High cardinality dimensions in memory

Abhilash L L created KYLIN-1844:
-----------------------------------

             Summary: High cardinality dimensions in memory
                 Key: KYLIN-1844
                 URL: https://issues.apache.org/jira/browse/KYLIN-1844
             Project: Kylin
          Issue Type: Improvement
          Components: Query Engine
    Affects Versions: v1.5.2, v1.2
            Reporter: Abhilash L L
            Assignee: liyang


A whole dimension is kept in memory.

We should have a way to keep only certain number / size of total rows to be kept in memory. A LRU cache for rows in the dimension will help keep memory in check.

Why not store all the dimensions data in hbase in a different table with a prefix of dimensionid, and all calls to the dimensions (get based on dim key), is mapped to hbase.

This does mean it will cost more time on a miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)