You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Xiaoxiang Yu (Jira)" <ji...@apache.org> on 2020/11/04 01:59:00 UTC
[jira] [Commented] (KYLIN-4804) Hive Global Dictionary - with
measure (PRICE) NumberFormatException
[ https://issues.apache.org/jira/browse/KYLIN-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225773#comment-17225773 ]
Xiaoxiang Yu commented on KYLIN-4804:
-------------------------------------
Really appreciate your report, we do not have enough test on various cases. I think we may try to fix this in next release, if you do know how to fix, a patch is welcome.
> Hive Global Dictionary - with measure (PRICE) NumberFormatException
> -------------------------------------------------------------------
>
> Key: KYLIN-4804
> URL: https://issues.apache.org/jira/browse/KYLIN-4804
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v3.1.1
> Reporter: Alexander
> Priority: Major
> Attachments: spark_executor.log
>
>
> If i try to use Global dict with measure (PRICE)
> Cubing with spark - exception
> {code:java}
> java.lang.NumberFormatException: For input string: "7623.0000"java.lang.NumberFormatException: For input string: "7623.0000" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:118) at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:99) at org.apache.kylin.engine.mr.common.BaseCuboidBuilder.buildValueObjects(BaseCuboidBuilder.java:116) at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:297) at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:257)
> {code}
> it's because
> {code:java}
> // In order to keep compatibility with old version, tinyint/smallint/int column use value directly, without dictionary
> private boolean needDictionaryColumn(FunctionDesc functionDesc) {
> DataType dataType = functionDesc.getParameter().getColRefs().get(0).getType();
> if (functionDesc.isMrDict()) {
> // If isMrDict set to true, it means related column has been
> // encoded in previous step by Hive Global Dictionary
> return false;
> }
> if (dataType.isIntegerFamily() && !dataType.isBigInt()) {
> return false;
> }
> return true;
> }
> {code}
> returns false due to isMrDict is true in this case and it goes id = Integer.parseInt(values[0]);
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)