You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by liangmeng <13...@139.com> on 2015/08/06 02:13:48 UTC
回复: exception at step 3:Build Dimension Dictionary
should i select false and fill the length of the dimension string length at "Advanced Setting" when defining a cube?
what's the meaning for these two?
梁猛
中国移动广东公司 网管维护中心 网管支撑室
电话：13802880779
邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
邮编：510623
发件人： liangmeng
发送时间： 2015-08-06 00:56
收件人： dev
主题： exception at step 3:Build Dimension Dictionary
i define a normal dimension from fact table which might have distinct values up to 100 millions, but the cube build failed at step 3, error log as below:
java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary -- DEFAULT.LTE_HTTP_ETL.MSISDN cardinality: 7908567
at org.apache.kylin.dict.DictionaryGenerator.buildDictionaryFromValueList(DictionaryGenerator.java:90)
at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:125)
should i make a new lookup table to save these value or has some other ways?
梁猛
中国移动广东公司 网管维护中心 网管支撑室
电话：13802880779
邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
邮编：510623
Re: exception at step 3:Build Dimension Dictionary
Posted by hongbin ma <ma...@apache.org>.
select false and fill the length of the dimension string length at "
Advanced Setting"
On Thu, Aug 6, 2015 at 11:14 AM, Luke Han <lu...@gmail.com> wrote:
> 100 millions is very high cardinality, we have dictionary threshold for
> this, please search in JIRA or this mailing list,
> There's discussion previous.
>
> Thanks.
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Thu, Aug 6, 2015 at 8:13 AM, liangmeng <13...@139.com> wrote:
>
> > should i select false and fill the length of the dimension string length
> > at "Advanced Setting" when defining a cube?
> > what's the meaning for these two?
> >
> > ------------------------------
> > 梁猛
> > 中国移动广东公司 网管维护中心 网管支撑室
> > 电话：13802880779
> > 邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
> > 地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
> > 邮编：510623
> >
> >
> > *发件人：* liangmeng <13...@139.com>
> > *发送时间：* 2015-08-06 00:56
> > *收件人：* dev <de...@kylin.incubator.apache.org>
> > *主题：* exception at step 3:Build Dimension Dictionary
> > i define a normal dimension from fact table which might have distinct
> > values up to 100 millions, but the cube build failed at step 3, error log
> > as below:
> >
> >
> > java.lang.IllegalArgumentException: Too high cardinality is not suitable
> for dictionary -- DEFAULT.LTE_HTTP_ETL.MSISDN cardinality: 7908567
> >
> > at
> org.apache.kylin.dict.DictionaryGenerator.buildDictionaryFromValueList(DictionaryGenerator.java:90)
> >
> > at
> org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:125)
> >
> > should i make a new lookup table to save these value or has some other
> > ways?
> > ------------------------------
> > 梁猛
> > 中国移动广东公司 网管维护中心 网管支撑室
> > 电话：13802880779
> > 邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
> > 地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
> > 邮编：510623
> >
> >
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: exception at step 3:Build Dimension Dictionary
Posted by Luke Han <lu...@gmail.com>.
100 millions is very high cardinality, we have dictionary threshold for
this, please search in JIRA or this mailing list,
There's discussion previous.
Thanks.
Best Regards!
---------------------
Luke Han
On Thu, Aug 6, 2015 at 8:13 AM, liangmeng <13...@139.com> wrote:
> should i select false and fill the length of the dimension string length
> at "Advanced Setting" when defining a cube?
> what's the meaning for these two?
>
> ------------------------------
> 梁猛
> 中国移动广东公司 网管维护中心 网管支撑室
> 电话：13802880779
> 邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
> 地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
> 邮编：510623
>
>
> *发件人：* liangmeng <13...@139.com>
> *发送时间：* 2015-08-06 00:56
> *收件人：* dev <de...@kylin.incubator.apache.org>
> *主题：* exception at step 3:Build Dimension Dictionary
> i define a normal dimension from fact table which might have distinct
> values up to 100 millions, but the cube build failed at step 3, error log
> as below:
>
>
> java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary -- DEFAULT.LTE_HTTP_ETL.MSISDN cardinality: 7908567
>
> at org.apache.kylin.dict.DictionaryGenerator.buildDictionaryFromValueList(DictionaryGenerator.java:90)
>
> at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:125)
>
> should i make a new lookup table to save these value or has some other
> ways?
> ------------------------------
> 梁猛
> 中国移动广东公司 网管维护中心 网管支撑室
> 电话：13802880779
> 邮箱: liangmeng@gd.chinamobile.com ，13802880779@139.com
> 地址：广东省广州市珠江新城珠江西路11号 广东全球通大厦北3楼
> 邮编：510623
>
>