You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2019/03/27 03:31:00 UTC
[jira] [Resolved] (KYLIN-3905) Enable shrunken dictionary default
[ https://issues.apache.org/jira/browse/KYLIN-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shaofeng SHI resolved KYLIN-3905.
---------------------------------
Resolution: Fixed
Excellent, thank you xiaoxiang!
> Enable shrunken dictionary default
> ----------------------------------
>
> Key: KYLIN-3905
> URL: https://issues.apache.org/jira/browse/KYLIN-3905
> Project: Kylin
> Issue Type: Improvement
> Components: Measure - Count Distinct
> Reporter: XiaoXiang Yu
> Assignee: XiaoXiang Yu
> Priority: Minor
> Fix For: v2.6.2
>
> Attachments: image-2019-03-25-11-26-59-198.png, image-2019-03-25-11-27-26-149.png, image-2019-03-25-11-27-46-175.png, image-2019-03-25-11-28-14-256.png, image-2019-03-25-11-29-19-383.png
>
>
> In dev mail list's discussion, I suggest to enable shrunken dictionary by default, and received some dev's aggrement.
> When using bitmap measure on a large cardinality column(require global dictionaty), build base cuboid step need frequent cache swap so it cannot finished within a reasonable period.
> When shrunken dictionary enabled, a new step will be added to build separated dictionary for each `InputSplit`, Mapper of **BuildBaseCuboid** step only has to fetch a smaller dictionary for itself, instead of a larger global dictionary. It will reduce cache swap and make **BuildBaseCuboid** step run as quicker as possible.
>
> http://mail-archives.apache.org/mod_mbox/kylin-dev//201903.mbox/%3c62EFCB72-B235-4FC3-9ADD-0FC510D97768@kyligence.io%3e
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)