You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/03/25 12:38:00 UTC
[jira] [Commented] (KYLIN-4433) When uhc step is turned on, Build
Dimension Dictionary job cannot get correct configuration
[ https://issues.apache.org/jira/browse/KYLIN-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066644#comment-17066644 ]
ASF GitHub Bot commented on KYLIN-4433:
---------------------------------------
ChangjiGuo commented on pull request #1168: KYLIN-4433 use cube config when uhc step enabled
URL: https://github.com/apache/kylin/pull/1168
issue: https://issues.apache.org/jira/browse/KYLIN-4433
If the system level configuration is used, it will cause the dictionary to be built repeatedly, so I think the configuration of the cube should be used here.
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> When uhc step is turned on, Build Dimension Dictionary job cannot get correct configuration
> -------------------------------------------------------------------------------------------
>
> Key: KYLIN-4433
> URL: https://issues.apache.org/jira/browse/KYLIN-4433
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v2.6.4
> Reporter: ChangjiGuo
> Priority: Major
>
> Hi guys :
> In the _Configuration overrides_ step of _Cube Designer_, I added the following configuration:
> {code:java}
> kylin.engine.mr.build-uhc-dict-in-additional-step = true
> {code}
> When building the cube, it will take an extra step to handle the UHC dimensions, but I found that in the step of 'Build Dimension Dictionary', it still builds the dictionary that has been built in the previous step which takes double the time to build the dictionary.
> When I traced the source code, I found that our expected code was not executed due to _config.isBuildUHCDictWithMREnabled() == false_ :
> {code:java}
> if (config.isBuildUHCDictWithMREnabled() && uhcColumns.contains(col)) {
> colDir = new Path(dictPath, col.getIdentity());
> } else {
> colDir = new Path(factColumnsInputPath, col.getIdentity());
> }
> {code}
> The root cause is that the configuration used in this step is the environment configuration, but our parameters are set in the cube:
> {code:java}
> final KylinConfig config = KylinConfig.getInstanceFromEnv();{code}
> I think the configuration of the cube should be used here instead of the environment configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)