You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by jyzheng <jy...@iflytek.com> on 2016/04/14 04:11:25 UTC
Precise distinct count meet NumberFormatException
I upgrad kylin to 1.5.1 version. When I build cube meet this exception:
The input hive table is orc file, and uid is bigint. The Hivesql “"select *
from sdk_log where uid = 2627100446" return the expected result.
2016-04-14 09:25:11,730 ERROR [IPC Server handler 2 on 49399]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1452760018394_272831_m_000001_1 - exited :
java.lang.NumberFormatException: For input string: "2627100446"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65
)
at java.lang.Integer.parseInt(Integer.java:495)
at java.lang.Integer.parseInt(Integer.java:527)
at
org.apache.kylin.measure.bitmap.BitmapCounter.add(BitmapCounter.java:63)
at
org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureTyp
e.java:106)
at
org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureTyp
e.java:98)
at
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValueOf(BaseCuboi
dMapperBase.java:189)
at
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValue(BaseCuboidM
apperBase.java:159)
at
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMap
perBase.java:206)
at
org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboid
Mapper.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
--------------------------------
郑江雨 云平台
Phone: 15155195496
Re: Precise distinct count meet NumberFormatException
Posted by Yerui Sun <su...@gmail.com>.
For now, precise distinct count only support int, but not bigint. We’re working on it to support other data types.
> 在 2016年4月14日，10:11，jyzheng <jy...@iflytek.com> 写道：
>
> I upgrad kylin to 1.5.1 version. When I build cube meet this exception:
>
> The input hive table is orc file, and uid is bigint. The Hivesql “"select * from sdk_log where uid = 2627100446" return the expected result.
> 2016-04-14 09:25:11,730 ERROR [IPC Server handler 2 on 49399] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1452760018394_272831_m_000001_1 - exited : java.lang.NumberFormatException: For input string: "2627100446"
>
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>
> at java.lang.Integer.parseInt(Integer.java:495)
>
> at java.lang.Integer.parseInt(Integer.java:527)
>
> at org.apache.kylin.measure.bitmap.BitmapCounter.add(BitmapCounter.java:63)
>
> at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:106)
>
> at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:98)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValueOf(BaseCuboidMapperBase.java:189)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValue(BaseCuboidMapperBase.java:159)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMapperBase.java:206)
>
> at org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboidMapper.java:53)
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
>
> --------------------------------
> 郑江雨 云平台
> Phone: 15155195496
Re: Precise distinct count meet NumberFormatException
Posted by Yerui Sun <su...@gmail.com>.
For now, precise distinct count only support int, but not bigint. We’re working on it to support other data types.
> 在 2016年4月14日，10:11，jyzheng <jy...@iflytek.com> 写道：
>
> I upgrad kylin to 1.5.1 version. When I build cube meet this exception:
>
> The input hive table is orc file, and uid is bigint. The Hivesql “"select * from sdk_log where uid = 2627100446" return the expected result.
> 2016-04-14 09:25:11,730 ERROR [IPC Server handler 2 on 49399] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1452760018394_272831_m_000001_1 - exited : java.lang.NumberFormatException: For input string: "2627100446"
>
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>
> at java.lang.Integer.parseInt(Integer.java:495)
>
> at java.lang.Integer.parseInt(Integer.java:527)
>
> at org.apache.kylin.measure.bitmap.BitmapCounter.add(BitmapCounter.java:63)
>
> at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:106)
>
> at org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:98)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValueOf(BaseCuboidMapperBase.java:189)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValue(BaseCuboidMapperBase.java:159)
>
> at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMapperBase.java:206)
>
> at org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboidMapper.java:53)
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
>
> --------------------------------
> 郑江雨 云平台
> Phone: 15155195496