You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shao Feng Shi (Jira)" <ji...@apache.org> on 2020/05/27 14:06:00 UTC
[jira] [Updated] (KYLIN-2231) Layer cubing tolerates some insane
records, but in-mem cubing doesn't
[ https://issues.apache.org/jira/browse/KYLIN-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shao Feng Shi updated KYLIN-2231:
---------------------------------
Labels: newbie (was: )
> Layer cubing tolerates some insane records, but in-mem cubing doesn't
> ---------------------------------------------------------------------
>
> Key: KYLIN-2231
> URL: https://issues.apache.org/jira/browse/KYLIN-2231
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Reporter: Shao Feng Shi
> Priority: Minor
> Labels: newbie
>
> User pengwen yuan in user@kylin.apache.org mailing list reports that, he got error when using "in-mem" cubing, but got success after switch to "layer" cubing algorithm.
> Checked the code I think it is because the layer cubing tolerates on minor insane records (see BaseCuboidMapperBase.handleErrorRecord()), but in-mem doesn't.
> There should be no difference in the behavior. Need make them consistent.
> The error in in-mem cubing is:
> {code}
> 2016-11-25 16:23:10,198 ERROR [Thread-12] org.apache.kylin.dict.TrieDictionary: Not a valid value: 532
> 2016-11-25 16:23:11,199 ERROR [pool-5-thread-1] org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build error
> java.io.IOException: java.lang.IllegalArgumentException: Value not exists!
> at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.abort(DoggedCubeBuilder.java:194)
> at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.checkException(DoggedCubeBuilder.java:167)
> at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:114)
> at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:73)
> at org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:80)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Value not exists!
> at org.apache.kylin.common.util.Dictionary.getIdFromValueBytes(Dictionary.java:162)
> at org.apache.kylin.dict.TrieDictionary.getIdFromValueImpl(TrieDictionary.java:167)
> at org.apache.kylin.common.util.Dictionary.getIdFromValue(Dictionary.java:98)
> at org.apache.kylin.dimension.DictionaryDimEnc$DictionarySerializer.serialize(DictionaryDimEnc.java:121)
> at org.apache.kylin.cube.gridtable.CubeCodeSystem.encodeColumnValue(CubeCodeSystem.java:121)
> at org.apache.kylin.cube.gridtable.CubeCodeSystem.encodeColumnValue(CubeCodeSystem.java:110)
> at org.apache.kylin.gridtable.GTRecord.setValues(GTRecord.java:93)
> at org.apache.kylin.gridtable.GTRecord.setValues(GTRecord.java:81)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilderInputConverter.convert(InMemCubeBuilderInputConverter.java:74)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder$InputConverter$1.next(InMemCubeBuilder.java:542)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder$InputConverter$1.next(InMemCubeBuilder.java:523)
> at org.apache.kylin.gridtable.GTAggregateScanner.iterator(GTAggregateScanner.java:139)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.createBaseCuboid(InMemCubeBuilder.java:339)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:166)
> at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:135)
> at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$SplitThread.run(DoggedCubeBuilder.java:282)
> 2016-11-25 16:23:11,202 INFO [pool-5-thread-1] org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build end, totally took 3410 ms
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)