You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "XIE FAN (JIRA)" <ji...@apache.org> on 2016/11/21 15:18:58 UTC

[jira] [Assigned] (KYLIN-2217) Reducers build dictionaries locally

     [ https://issues.apache.org/jira/browse/KYLIN-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

XIE FAN reassigned KYLIN-2217:
------------------------------

    Assignee: XIE FAN

> Reducers build dictionaries locally
> -----------------------------------
>
>                 Key: KYLIN-2217
>                 URL: https://issues.apache.org/jira/browse/KYLIN-2217
>             Project: Kylin
>          Issue Type: Improvement
>    Affects Versions: v1.5.4.1
>            Reporter: XIE FAN
>            Assignee: XIE FAN
>             Fix For: Future
>
>
> In KYLIN-1851, we reduce the peek memory usage of the dictionary-building procedure by splitting a single Trie tree structure to Trie forest. But there still exist a bottleneck that all the dictionaries are built in Kylin client. In this issue, we want to use multi reducers to build different dictionaries locally and concurrently，which can further reduce the peek memory usage as well as speed up the dictionary-building procedure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)