You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "JiangQin (JIRA)" <ji...@apache.org> on 2017/09/05 02:33:00 UTC
[jira] [Created] (KYLIN-2845) Merging cube get error
IllegalStateException
JiangQin created KYLIN-2845:
-------------------------------
Summary: Merging cube get error IllegalStateException
Key: KYLIN-2845
URL: https://issues.apache.org/jira/browse/KYLIN-2845
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v2.0.0
Reporter: JiangQin
Assignee: Dong Li
When trying to merge cubes, I encountered following errors :
Error: java.lang.IllegalStateException at com.google.common.base.Preconditions.checkState(Preconditions.java:129) at org.apache.kylin.cube.kv.RowKeyEncoder.encode(RowKeyEncoder.java:123) at org.apache.kylin.engine.mr.steps.MergeCuboidMapper.doMap(MergeCuboidMapper.java:246)
I digged into the source code and found the following code:
{code:java}
public void encode(ByteArray bodyBytes, ByteArray outputBuf) {
Preconditions.checkState(bodyBytes.length() == bodyLength);
Preconditions.checkState(bodyBytes.length() + getHeaderLength() == outputBuf.length(), //
"bodybytes length: " + bodyBytes.length() + " outputBuf length: " + outputBuf.length() + " header length: " + getHeaderLength());
System.arraycopy(bodyBytes.array(), bodyBytes.offset(), outputBuf.array(), getHeaderLength(), bodyLength);
//fill shard and cuboid
fillHeader(outputBuf.array());
}
{code}
And also I found an old similar bug [https://issues.apache.org/jira/browse/KYLIN-1004].
The cube can build correctly, but the merge just does not work.
Here is the detail for this bug
2017-09-05 09:35:57,801 INFO [main] org.apache.kylin.cube.CubeManager: Reloaded cube cube_fact_dui_assemble_log being CUBE[name=cube_fact_dui_assemble_log] having 7 segments
2017-09-05 09:35:57,801 INFO [main] org.apache.kylin.cube.CubeManager: Loaded 1 cubes, fail on 0 cubes
2017-09-05 09:35:57,805 INFO [main] org.apache.kylin.dict.DictionaryManager: DictionaryManager(1908999587) loading DictionaryInfo(loadDictObj:true) at /dict/DIM_DUI.DIM_CALENDAR/P_DAY/7dbd2f63-ffc6-4eb6-972a-b712954bcff0.dict
2017-09-05 09:35:57,821 INFO [main] org.apache.kylin.dict.DictionaryManager: DictionaryManager(1908999587) loading DictionaryInfo(loadDictObj:true) at /dict/DIM_DUI.DIM_DEVELOPER/DEVELOPER_ID/31e3578a-94fe-4d99-b00f-c523f59b8eb2.dict
2017-09-05 09:35:57,833 INFO [main] org.apache.kylin.dict.DictionaryManager: DictionaryManager(1908999587) loading DictionaryInfo(loadDictObj:true) at /dict/DIM_DUI.DIM_DEVELOPER/DEVELOPER_ID/9e365c29-52d8-4397-acb9-63cb9214a05e.dict
2017-09-05 09:35:57,834 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Accepting Mapper Key with ordinal: 1
2017-09-05 09:35:57,837 ERROR [main] org.apache.kylin.engine.mr.KylinMapper:
java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at org.apache.kylin.cube.kv.RowKeyEncoder.encode(RowKeyEncoder.java:123)
at org.apache.kylin.engine.mr.steps.MergeCuboidMapper.doMap(MergeCuboidMapper.java:246)
at org.apache.kylin.engine.mr.steps.MergeCuboidMapper.doMap(MergeCuboidMapper.java:63)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:48)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
2017-09-05 09:35:57,866 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output
2017-09-05 09:35:57,958 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2017-09-05 09:35:57,964 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at org.apache.kylin.cube.kv.RowKeyEncoder.encode(RowKeyEncoder.java:123)
at org.apache.kylin.engine.mr.steps.MergeCuboidMapper.doMap(MergeCuboidMapper.java:246)
at org.apache.kylin.engine.mr.steps.MergeCuboidMapper.doMap(MergeCuboidMapper.java:63)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:48)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)