You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kylin.apache.org by "ShengJun Zheng (Jira)" <ji...@apache.org> on 2021/06/28 10:02:00 UTC

[jira] [Created] (KYLIN-5019) Avoid building global dictionary from all data of fact table each time

ShengJun Zheng created KYLIN-5019:
-------------------------------------

             Summary: Avoid building global dictionary from all data of fact table each time
                 Key: KYLIN-5019
                 URL: https://issues.apache.org/jira/browse/KYLIN-5019
             Project: Kylin
          Issue Type: New Feature
            Reporter: ShengJun Zheng


Global dictionaries will be resized to avoid single dictionary bucket exceeding its threashold in each cubing job. To judge whether a dictionary should be resized, Kylin will read all data from fact table and do count distinct operation. Then rebuild dictionary from all history data.

It’s better to use a lightweight approach, since dictionary don't need to be resized for incremental building in most cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)