You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Li Yang <li...@apache.org> on 2017/09/03 07:58:24 UTC
Re: merge cube Invalid input data. Unordered data cannot be split
into multi trees
Thanks for reporting! This is a known issue. Should fix in next release.
https://issues.apache.org/jira/browse/KYLIN-2794
On Wed, Aug 23, 2017 at 2:59 PM, yuyong.zhai <yu...@ele.me> wrote:
> kylin:2.0.0 hbase 0.98 hadoop:2.6.0-cdh5.8.2
>
>
> 2017-08-23 14:53:00,353 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> execution.AbstractExecutable:110 : Executing AbstractExecutable
> (Order_Cube - 20170601000000_2017061600
>
> 0000 - MERGE - GMT+08:00 2017-08-23 14:52:58)
>
> 2017-08-23 14:53:00,353 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dao.ExecutableDao:217 : updating job output, id: a27f40cb-14f6-4472-a192-
> 3d33cf77d34d
>
> 2017-08-23 14:53:00,354 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> execution.ExecutableManager:389 : job id:a27f40cb-14f6-4472-a192-3d33cf77d34d
> from READY to RUNNING
>
> 2017-08-23 14:53:00,355 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> execution.AbstractExecutable:110 : Executing AbstractExecutable (Merge
> Cuboid Dictionary)
>
> 2017-08-23 14:53:00,355 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dao.ExecutableDao:217 : updating job output, id: a27f40cb-14f6-4472-a192-
> 3d33cf77d34d-00
>
> 2017-08-23 14:53:00,356 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> execution.ExecutableManager:389 : job id:a27f40cb-14f6-4472-a192-3d33cf77d34d-00
> from READY to RUNNING
>
> 2017-08-23 14:53:00,356 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> steps.MergeDictionaryStep:123 : Merging fact table dictionary on :
> ST.ST_LOG_APP_USER_GROUP_ORDER_DAY_I
>
> NC.RESTAURANT_NAME
>
> 2017-08-23 14:53:00,356 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> steps.MergeDictionaryStep:126 : Including fact table dictionary of segment
> : Order_Cube[20170601000000_
>
> 20170615000000]
>
> 2017-08-23 14:53:00,356 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> steps.MergeDictionaryStep:126 : Including fact table dictionary of segment
> : Order_Cube[20170615000000_
>
> 20170616000000]
>
> 2017-08-23 14:53:00,360 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.TrieDictionaryForestBuilder:89 : values not in ascending order,
> previous '\xE5\x85\xB6\xE4\xBB\x96
>
> ', current '1003996_\xE9\xA6\x99\xE6\xB8\xAF\xE4\xB9\x9D\xE9\xBE\x99\
> xE5\x8C\x85\xEF\xBC\x88\xE8\xB4\xA2\xE7\xBB\x8F\xEF\xBC\x89'
>
> 2017-08-23 14:53:00,366 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryGenerator:96 : Dictionary value samples:
> 1000016_我呀便当（滨江区府店）=>0, 1000086_每味
>
> 每客（日月光店）=>1, 1001022_华莱士（现代店）=>2, 1001410_简餐厅=>3, 1001539_水果大王=>4
>
> 2017-08-23 14:53:00,366 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryGenerator:97 : Dictionary cardinality: 4152
>
> 2017-08-23 14:53:00,366 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryGenerator:98 : Dictionary builder class:
> org.apache.kylin.dict.DictionaryGenerator$Strin
>
> gTrieDictForestBuilder
>
> 2017-08-23 14:53:00,366 DEBUG [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryGenerator:99 : Dictionary class: org.apache.kylin.dict.
> TrieDictionaryForest
>
> 2017-08-23 14:53:00,367 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:185 : 61 existing dictionaries of the same column
>
> 2017-08-23 14:53:00,368 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:445 : DictionaryManager(49776108) loading
> DictionaryInfo(loadDictObj:true) at /dict/ST.ST_LOG_APP_USER_
> GROUP_ORDER_DAY_INC/RESTAURANT_NAME/c571da70-84d4-
> 4231-ba43-b182268e19a5.dict
>
> 2017-08-23 14:53:00,369 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:445 : DictionaryManager(49776108) loading
> DictionaryInfo(loadDictObj:true) at /dict/ST.ST_LOG_APP_USER_
> GROUP_ORDER_DAY_INC/RESTAURANT_NAME/c6090e98-d20f-
> 4291-87a5-33d888a4a965.dict
>
> 2017-08-23 14:53:00,369 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:445 : DictionaryManager(49776108) loading
> DictionaryInfo(loadDictObj:true) at /dict/ST.ST_LOG_APP_USER_
> GROUP_ORDER_DAY_INC/RESTAURANT_NAME/cff448ae-cf4f-
> 45cd-83ef-61dae4c8b972.dict
>
> 2017-08-23 14:53:00,370 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:445 : DictionaryManager(49776108) loading
> DictionaryInfo(loadDictObj:true) at /dict/ST.ST_LOG_APP_USER_
> GROUP_ORDER_DAY_INC/RESTAURANT_NAME/d0ceebd3-9b9f-
> 4917-a6b0-8283bb0f9931.dict
>
> 2017-08-23 14:53:00,370 INFO [Job a27f40cb-14f6-4472-a192-3d33cf77d34d-1199]
> dict.DictionaryManager:445 : DictionaryManager(49776108) loading
> DictionaryInfo(loadDictObj:true) at /dict/ST.ST_LOG_APP_USER_
> GROUP_ORDER_DAY_INC/RESTAURANT_NAME/d1a39987-9bad-
> 4d12-bc25-2d0e830e30a9.dict
>
>
>
> org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException:
> java.lang.IllegalStateException: Invalid input data. Unordered data
> cannot be split into multi trees
>
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:134)
>
> at org.apache.kylin.job.impl.threadpool.DefaultScheduler$
> JobRunner.run(DefaultScheduler.java:142)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.IllegalStateException:
> Invalid input data. Unordered data cannot be split into multi trees
>
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:134)
>
> at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(
> DefaultChainedExecutable.java:64)
>
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:124)
>
> ... 4 more
>
> Caused by: java.lang.IllegalStateException: Invalid input data. Unordered
> data cannot be split into multi trees
>
> at org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(
> TrieDictionaryForestBuilder.java:92)
>
> at org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(
> TrieDictionaryForestBuilder.java:78)
>
> at org.apache.kylin.dict.DictionaryGenerator$
> StringTrieDictForestBuilder.addValue(DictionaryGenerator.java:212)
>
> at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(
> DictionaryGenerator.java:79)
>
> at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(
> DictionaryGenerator.java:64)
>
> at org.apache.kylin.dict.DictionaryGenerator.mergeDictionaries(
> DictionaryGenerator.java:104)
>
> at org.apache.kylin.dict.DictionaryManager.mergeDictionary(
> DictionaryManager.java:267)
>
> at org.apache.kylin.engine.mr.steps.MergeDictionaryStep.
> mergeDictionaries(MergeDictionaryStep.java:146)
>
> at org.apache.kylin.engine.mr.steps.MergeDictionaryStep.
> makeDictForNewSegment(MergeDictionaryStep.java:136)
>
> at org.apache.kylin.engine.mr.steps.MergeDictionaryStep.
> doWork(MergeDictionaryStep.java:68)
>
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:124)
>
> ... 6 more
>
>