You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2017/08/09 05:44:00 UTC

[jira] [Closed] (KYLIN-1178) Build dictionary in Hadoop cluster

     [ https://issues.apache.org/jira/browse/KYLIN-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shaofeng SHI closed KYLIN-1178.
-------------------------------

> Build dictionary in Hadoop cluster
> ----------------------------------
>
>                 Key: KYLIN-1178
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1178
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: Shaofeng SHI
>            Assignee: Shaofeng SHI
>
> Kylin build dictionary in job engine node, usually this is okay. But if there is some high cardinality dimentions, the JVM heap couldn't fit in all distinct values, then job engine instance will crash with OOM error.
> Need to enhance on this, move the dictionary building to another process or a hadoop node. Ideally only need modify "CreateDictionaryJob.java", move the dictionary building to a mapper task.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)