You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Luke Han (JIRA)" <ji...@apache.org> on 2015/08/25 14:44:51 UTC
[jira] [Updated] (KYLIN-275) Kylin need validate the data type
before building the cube;
[ https://issues.apache.org/jira/browse/KYLIN-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Han updated KYLIN-275:
---------------------------
Fix Version/s: (was: v1.0)
v1.1
> Kylin need validate the data type before building the cube;
> -----------------------------------------------------------
>
> Key: KYLIN-275
> URL: https://issues.apache.org/jira/browse/KYLIN-275
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Reporter: Luke Han
> Assignee: ZhouQianhao
> Labels: github-import
> Fix For: v1.1
>
>
> As Hive doesn't validate data type, it has the possibility that a column is declared as number type while its value is not number; Such dirty data will cause Kylin failed in building the base cuboid, the error looks like:
> 2014-12-16 00:50:42,172 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.NumberFormatException: For input string: "3917898514a0a356848d035afffa092e"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:441)
> at java.lang.Long.parseLong(Long.java:483)
> at com.kylinolap.cube.measure.LongSerializer.valueOf(LongSerializer.java:51)
> at com.kylinolap.cube.measure.LongSerializer.valueOf(LongSerializer.java:30)
> at com.kylinolap.job.hadoop.cube.BaseCuboidMapper.buildValue(BaseCuboidMapper.java:151)
> at com.kylinolap.job.hadoop.cube.BaseCuboidMapper.map(BaseCuboidMapper.java:210)
> at com.kylinolap.job.hadoop.cube.BaseCuboidMapper.map(BaseCuboidMapper.java:52)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> So, Kylin may need do some kind of data validation before the calculation.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/KylinOLAP/Kylin/issues/230
> Created by: [shaofengshi|https://github.com/shaofengshi]
> Labels:
> Created at: Tue Dec 16 16:25:50 CST 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)