Re: Re: org.apache.kylin.dict.TrieDictionary: Not a valid value

[Li Yang <li...@apache.org>]

Note https://issues.apache.org/jira/browse/KYLIN-1834

was fixed again in v1.6.0

Try the latest version if not yet.

On Wed, Nov 30, 2016 at 4:40 PM, ypw89724@126.com <yp...@126.com> wrote:

> Hi Shaofeng,
>
> I modify models, but it's still error.
>
> And I found in normal operation,when job error(not discard) you cann't
> build this cube(show warning). (build->job error->build(ERROR))
> But when chose purge the cube can build(show build page).(build -> job
> error -> cube purge -> build(OK))
>
>
> ------------------------------
> 袁鹏文  Pengwen Yuan
>
>
> *From:* ShaoFeng Shi <sh...@apache.org>
> *Date:* 2016-11-26 15:59
> *To:* user <us...@kylin.apache.org>
> *Subject:* Re: Re: org.apache.kylin.dict.TrieDictionary: Not a valid value
> Hi pengwen,
>
> I carefully checked that and I think it was caused by some dirtly data on
> fact table (fk but doesn't exist in lookup table); The invalid value "532"
> should be a FK column value. Since the lookup table is defined as "inner
> join" with the fact table, Kylin delegates building the dictionary from
> lookup table's PK; so if there is unmatched record, this error may happen.
> The discussion about this issue is in JIRA KYLIN-1855.
>
> Regarding why in-mem building got failed but layer cubing got success,
> that is because the layer cubing tolerates some insane record (< 100 in one
> mapper), but in-mem building didn't. I opened JIRA KYLIN-2231 for tracking
> this.
>
> Thanks for the reporting!
>
>
> 2016-11-25 23:33 GMT+08:00 ypw89724@126.com <yp...@126.com>:
>
>> This attachment is diagnosis.
>> Tonight I tried to use the layer cubing (kylin.cube.algorithm:layer),I
>> found its ok.
>>
>> ------------------------------
>> 袁鹏文  Pengwen Yuan
>>
>>
>> *From:* ShaoFeng Shi <sh...@apache.org>
>> *Date:* 2016-11-25 17:31
>> *To:* user <us...@kylin.apache.org>
>> *Subject:* Re: org.apache.kylin.dict.TrieDictionary: Not a valid value
>> Hi Pengwen,
>>
>> This question also be asked by other people before, but we wasn't able to
>> reproduce. Could you share your model and cube definition JSON? Thanks!
>>
>> 2016-11-25 15:41 GMT+08:00 ypw89724@126.com <yp...@126.com>:
>>
>>> hi all:
>>>
>>> When I build cube at "#15 Step Name: Build Cube", I got an error :
>>>
>>>
>>> 2016-11-25 14:17:23,724 INFO [main] org.apache.kylin.engine.mr.steps.InMemCuboidMapper: Reserve 1079 MB = 200 (MR reserve) + 879 (SYS reserve)
>>> 2016-11-25 14:17:23,732 INFO [pool-5-thread-1] org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build start
>>> 2016-11-25 14:17:23,758 INFO [pool-5-thread-1] org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Split #1 kickoff
>>> 2016-11-25 14:17:23,771 INFO [Thread-12] org.apache.kylin.cube.inmemcubing.InMemCubeBuilder: In Mem Cube Build start, CUBE_OLP_SGM_HOTELMONTHREPORT
>>> 2016-11-25 14:17:25,984 WARN [Thread-12] org.apache.kylin.common.util.MemoryBudgetController: Higher system avail 8886 MB in markLow()
>>> 2016-11-25 14:17:25,987 INFO [Thread-12] org.apache.kylin.cube.inmemcubing.InMemCubeBuilder: Calculating base cuboid 1272016-11-25 14:17:26,134 ERROR [Thread-12] org.apache.kylin.dict.TrieDictionary: Not a valid value: 532
>>> 2016-11-25 14:17:27,136 ERROR [pool-5-thread-1] org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build error
>>> java.io.IOException: java.lang.IllegalArgumentException: Value not exists!
>>> 	at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.abort(DoggedCubeBuilder.java:194)
>>> 	at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.checkException(DoggedCubeBuilder.java:167)
>>> 	at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:114)
>>> 	at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:73)
>>> 	at org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:80)
>>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> 	at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.lang.IllegalArgumentException: Value not exists!
>>> 	at org.apache.kylin.common.util.Dictionary.getIdFromValueBytes(Dictionary.java:162)
>>> 	at org.apache.kylin.dict.TrieDictionary.getIdFromValueImpl(TrieDictionary.java:167)
>>> 	at org.apache.kylin.common.util.Dictionary.getIdFromValue(Dictionary.java:98)
>>>
>>>
>>> My Kylin version : 1.5.4.1
>>>
>>> hadoop : 2.6.0-cdh5.8.0
>>>
>>> HBase : 1.2.0-cdh5.8.0
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> 袁鹏文  Pengwen Yuan
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
> [image: 提示图标] 邮件带有附件预览链接，若您转发或回复此邮件时不希望对方预览附件，建议您手动删除链接。
> 共有 *1* 个附件
> Catch0E0F.jpg(53K) 极速下载
> <http://preview.mail.126.com/xdownload?filename=Catch0E0F.jpg&mid=1tbikAguFFag1GbqlQAAsZ&part=3&sign=0990a222c8b70fa5e1ddebe695c24a52&time=1480147278&uid=ypw89724%40126.com>
> 在线预览
> <http://preview.mail.126.com/preview?mid=1tbikAguFFag1GbqlQAAsZ&part=3&sign=0990a222c8b70fa5e1ddebe695c24a52&time=1480147278&uid=ypw89724%40126.com>
>
>