You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "huangsheng (Jira)" <ji...@apache.org> on 2023/07/03 06:45:00 UTC

[jira] [Comment Edited] (KYLIN-5601) V2 dictionary workaround fixes

    [ https://issues.apache.org/jira/browse/KYLIN-5601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739461#comment-17739461 ] 

huangsheng edited comment on KYLIN-5601 at 7/3/23 6:44 AM:
-----------------------------------------------------------

Fix Design：

New configuration: kylin.engine.global-dict-aqe-enabled, disabled by default

Add ss.conf.set(AQE, false) to the code of real build dict to temporarily close aqe

Then after the build is over, restore aqe to the initial value and add an internal switch: kylin.engine.global-dict-aqe-enabled, which is disabled by default

Add ss.conf.set(AQE, false) to the code of real build dict to temporarily close aqe

Then after the build is over, restore aqe to the initial value


was (Author: JIRAUSER297438):
Fix Design：

New internal switch: kylin.engine.global-dict-aqe-enabled, disabled by default

Add ss.conf.set(AQE, false) to the code of real build dict to temporarily close aqe

Then after the build is over, restore aqe to the initial value and add an internal switch: kylin.engine.global-dict-aqe-enabled, which is disabled by default

Add ss.conf.set(AQE, false) to the code of real build dict to temporarily close aqe

Then after the build is over, restore aqe to the initial value

> V2 dictionary workaround fixes
> ------------------------------
>
>                 Key: KYLIN-5601
>                 URL: https://issues.apache.org/jira/browse/KYLIN-5601
>             Project: Kylin
>          Issue Type: Bug
>          Components: Job Engine
>    Affects Versions: 5.0-alpha
>            Reporter: huangsheng
>            Priority: Major
>             Fix For: 5.0-alpha
>
>
> In some scenarios, AQE optimizes the step of writing the dictionary from repartition in the execution plan, skips the encoding of some partitioned dictionaries, and there is no abnormal prompt in the task status, which eventually leads to an error in the calculation of the count distinct metric.
> So consider turning off AQE during dictionary construction



--
This message was sent by Atlassian Jira
(v8.20.10#820010)