You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/07/21 15:33:00 UTC
[jira] [Commented] (KYLIN-3905) Enable shrunken dictionary default
[ https://issues.apache.org/jira/browse/KYLIN-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889753#comment-16889753 ]
ASF GitHub Bot commented on KYLIN-3905:
---------------------------------------
hit-lacus commented on pull request #765: KYLIN-3905 Disable shrunken dict if Mr-Hive dict is enabled & code format
URL: https://github.com/apache/kylin/pull/765
1. Shutdown shrunken dict if Mr-Hive dict is enabled
2. Add comment & reformat code for kylin-metrics module
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Enable shrunken dictionary default
> ----------------------------------
>
> Key: KYLIN-3905
> URL: https://issues.apache.org/jira/browse/KYLIN-3905
> Project: Kylin
> Issue Type: Improvement
> Components: Measure - Count Distinct
> Reporter: Xiaoxiang Yu
> Assignee: Xiaoxiang Yu
> Priority: Minor
> Fix For: v2.6.2, v3.0.0-alpha
>
> Attachments: image-2019-03-25-11-26-59-198.png, image-2019-03-25-11-27-26-149.png, image-2019-03-25-11-27-46-175.png, image-2019-03-25-11-28-14-256.png, image-2019-03-25-11-29-19-383.png
>
>
> In dev mail list's discussion, I suggest to enable shrunken dictionary by default, and received some dev's aggrement.
> When using bitmap measure on a large cardinality column(require global dictionaty), build base cuboid step need frequent cache swap so it cannot finished within a reasonable period.
> When shrunken dictionary enabled, a new step will be added to build separated dictionary for each `InputSplit`, Mapper of **BuildBaseCuboid** step only has to fetch a smaller dictionary for itself, instead of a larger global dictionary. It will reduce cache swap and make **BuildBaseCuboid** step run as quicker as possible.
>
> http://mail-archives.apache.org/mod_mbox/kylin-dev//201903.mbox/%3c62EFCB72-B235-4FC3-9ADD-0FC510D97768@kyligence.io%3e
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)