You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by rahulsingh <ra...@i-link.co.in> on 2017/03/07 14:01:43 UTC
get
Hi all,
getting this error while building a cube
Please help.
Thanks
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.io.IOException: Failed to build cube in mapper 9
at
org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:145)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:148)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1714)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.concurrent.ExecutionException:
java.nio.BufferUnderflowException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:143)
... 8 more
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:492)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:135)
at org.apache.kylin.common.util.BytesUtil.readVLong(BytesUtil.java:208)
at org.apache.kylin.common.util.BytesUtil.readVInt(BytesUtil.java:223)
at
org.apache.kylin.metadata.datatype.BigDecimalSerializer.peekLength(BigDecimalSerializer.java:83)
at
org.apache.kylin.cube.gridtable.CubeCodeSystem.codeLength(CubeCodeSystem.java:91)
at org.apache.kylin.gridtable.GTRecord.loadColumns(GTRecord.java:294)
at
org.apache.kylin.cube.inmemcubing.ConcurrentDiskStore$Reader$2.hasNext(ConcurrentDiskStore.java:233)
at
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$MergeSlot.fetchNext(DoggedCubeBuilder.java:412)
at
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$Merger.mergeAndOutput(DoggedCubeBuilder.java:328)
at
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:119)
at
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:75)
at
org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:82)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/get-tp7378.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: get
Posted by ShaoFeng Shi <sh...@apache.org>.
Open JIRA for tracking this:
https://issues.apache.org/jira/browse/KYLIN-2512
Before this be resolved, as a work-round, you can switch to "by-layer"
cubing algorithm by setting: kylin.cube.algorithm=layer
2017-03-16 16:13 GMT+08:00 rahulsingh <ra...@i-link.co.in>:
> Hi shaofengshi,
>
> I have attached the Cube Metadata file in the attachment.
>
> Thank You
>
> cube_metadata.txt
> <http://apache-kylin.74782.x6.nabble.com/file/n7452/cube_metadata.txt>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Getting-this-exception-at-step-18-Build-Cube-tp7378p7452.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
--
Best regards,
Shaofeng Shi 史少锋
Re: get
Posted by rahulsingh <ra...@i-link.co.in>.
Hi shaofengshi,
I have attached the Cube Metadata file in the attachment.
Thank You
cube_metadata.txt
<http://apache-kylin.74782.x6.nabble.com/file/n7452/cube_metadata.txt>
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Getting-this-exception-at-step-18-Build-Cube-tp7378p7452.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: get
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Rahul, could you provide the Cube metadata (JSON format) for analysis?
You can do confusion on the column names, but keep the structure. I have
seen Cubes with over hundred measures, which doesn't have this error, so
want to double check what is special in your case.
2017-03-16 1:57 GMT+08:00 rahulsingh <ra...@i-link.co.in>:
> hi,
>
> Still i am facing the above exception.
> Please, give me if you have any solution for this.
>
> Thank You,
> Rahul Singh
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Getting-this-exception-at-step-18-Build-Cube-tp7378p7444.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
--
Best regards,
Shaofeng Shi 史少锋
Re: get
Posted by rahulsingh <ra...@i-link.co.in>.
hi,
Still i am facing the above exception.
Please, give me if you have any solution for this.
Thank You,
Rahul Singh
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Getting-this-exception-at-step-18-Build-Cube-tp7378p7444.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: get
Posted by rahulsingh <ra...@i-link.co.in>.
hi,
thanks for replying.
Cube consist 57 measure in which some have integer data type and some have
double.
Thank You
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/Getting-this-exception-at-step-18-Build-Cube-tp7378p7385.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: get
Posted by ShaoFeng Shi <sh...@apache.org>.
The estimated buffer is shorter than expected. Usually this won't happen as
each data type serializer has reported a max lenght, but some (like topN)
may not estimate that precisely. Need to know the data type and measures in
your cube, could you please share that?
2017-03-07 22:01 GMT+08:00 rahulsingh <ra...@i-link.co.in>:
> Hi all,
>
> getting this error while building a cube
> Please help.
>
> Thanks
> org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.io.IOException: Failed to build cube in mapper 9
> at
> org.apache.kylin.engine.mr.steps.InMemCuboidMapper.
> cleanup(InMemCuboidMapper.java:145)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:148)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1714)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException:
> java.nio.BufferUnderflowException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.kylin.engine.mr.steps.InMemCuboidMapper.
> cleanup(InMemCuboidMapper.java:143)
> ... 8 more
> Caused by: java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:492)
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:135)
> at org.apache.kylin.common.util.BytesUtil.readVLong(BytesUtil.
> java:208)
> at org.apache.kylin.common.util.BytesUtil.readVInt(BytesUtil.
> java:223)
> at
> org.apache.kylin.metadata.datatype.BigDecimalSerializer.peekLength(
> BigDecimalSerializer.java:83)
> at
> org.apache.kylin.cube.gridtable.CubeCodeSystem.codeLength(CubeCodeSystem.
> java:91)
> at org.apache.kylin.gridtable.GTRecord.loadColumns(GTRecord.
> java:294)
> at
> org.apache.kylin.cube.inmemcubing.ConcurrentDiskStore$Reader$2.
> hasNext(ConcurrentDiskStore.java:233)
> at
> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$MergeSlot.fetchNext(
> DoggedCubeBuilder.java:412)
> at
> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$Merger.mergeAndOutput(
> DoggedCubeBuilder.java:328)
> at
> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(
> DoggedCubeBuilder.java:119)
> at
> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.
> build(DoggedCubeBuilder.java:75)
> at
> org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.
> run(AbstractInMemCubeBuilder.java:82)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/get-tp7378.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
--
Best regards,
Shaofeng Shi 史少锋