You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 沙漠火狐 <27...@qq.com> on 2016/10/11 01:16:14 UTC
回复: java.lang.OutOfMemoryError: Java heap space
thanks xwhfcenter@gmail.com
in /bin/setev.sh my setting is
export KYLIN_JVM_SETTINGS="-Xms2048M -Xmx10240M -Xss1024K -XX:MaxPermSize=10240M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
the problem is still. or i any config setting is wrong?
------------------ 原始邮件 ------------------
发件人: "Mario Copperfield";<xw...@gmail.com>;
发送时间: 2016年10月10日(星期一) 下午5:26
收件人: "dev"<de...@kylin.apache.org>;
主题: Re: java.lang.OutOfMemoryError: Java heap space
can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh
On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <27...@qq.com> wrote:
> hi
> I have a big lookuptable about 1.3G, when I cube it, in
> /logs/kylin.out say :
>
>
> Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start
> INFO: Server startup in 24004 ms
> Found segment ordrpath_goodsid[20161008000000_20161009000000]
> #
> # java.lang.OutOfMemoryError: Java heap space
> # -XX:OnOutOfMemoryError="kill -9 %p"
> # Executing /bin/sh -c "kill -9 23272"...
>
>
>
> ------------------------------------------------------------
> then the kylin is stop.
>
>
> so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m
> -XX:MaxPermSize=4096m" config in catalin.sh
>
>
> and change the config in kylin_job_conf_inmem.xml to
> <property>
> <name>mapreduce.map.memory.mb</name>
> <value>4096</value>
> <description></description>
> </property>
>
>
> <property>
> <name>mapreduce.map.java.opts</name>
> <value>-Xmx10240m</value>
> <description></description>
> </property>
>
>
> I also change the propery in kylin.properies
> kylin.dictionary.max.cardinality=50000000
> kylin.table.snapshot.max_mb=1500
>
>
>
>
>
>
>
> but the problem is still in. is there any config i should add to solve
> this problem. thanks !
--
Best regards,
Amuro Copperfield
Re: java.lang.OutOfMemoryError: Java heap space
Posted by Li Yang <li...@apache.org>.
1.3G is too big as a lookup table. Consider make it part of fact table, by
using a view to join it with the origin fact for example.
On Tue, Oct 11, 2016 at 9:31 AM, Billy(Yiming) Liu <li...@gmail.com>
wrote:
> 50,000,000 is ultra high cardinality. What kind of Dictionary did you
> choose?
>
> 2016-10-11 9:16 GMT+08:00 沙漠火狐 <27...@qq.com>:
>
> > thanks xwhfcenter@gmail.com
> >
> >
> > in /bin/setev.sh my setting is
> >
> >
> > export KYLIN_JVM_SETTINGS="-Xms2048M -Xmx10240M -Xss1024K
> > -XX:MaxPermSize=10240M -verbose:gc -XX:+PrintGCDetails
> > -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$
> > -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
> -XX:GCLogFileSize=64M"
> >
> >
> > the problem is still. or i any config setting is wrong?
> >
> >
> >
> >
> > ------------------ 原始邮件 ------------------
> > 发件人: "Mario Copperfield";<xw...@gmail.com>;
> > 发送时间: 2016年10月10日(星期一) 下午5:26
> > 收件人: "dev"<de...@kylin.apache.org>;
> >
> > 主题: Re: java.lang.OutOfMemoryError: Java heap space
> >
> >
> >
> > can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh
> >
> > On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <27...@qq.com> wrote:
> >
> > > hi
> > > I have a big lookuptable about 1.3G, when I cube it, in
> > > /logs/kylin.out say :
> > >
> > >
> > > Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start
> > > INFO: Server startup in 24004 ms
> > > Found segment ordrpath_goodsid[20161008000000_20161009000000]
> > > #
> > > # java.lang.OutOfMemoryError: Java heap space
> > > # -XX:OnOutOfMemoryError="kill -9 %p"
> > > # Executing /bin/sh -c "kill -9 23272"...
> > >
> > >
> > >
> > > ------------------------------------------------------------
> > > then the kylin is stop.
> > >
> > >
> > > so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m
> > > -XX:MaxPermSize=4096m" config in catalin.sh
> > >
> > >
> > > and change the config in kylin_job_conf_inmem.xml to
> > > <property>
> > > <name>mapreduce.map.memory.mb</name>
> > > <value>4096</value>
> > > <description></description>
> > > </property>
> > >
> > >
> > > <property>
> > > <name>mapreduce.map.java.opts</name>
> > > <value>-Xmx10240m</value>
> > > <description></description>
> > > </property>
> > >
> > >
> > > I also change the propery in kylin.properies
> > > kylin.dictionary.max.cardinality=50000000
> > > kylin.table.snapshot.max_mb=1500
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > but the problem is still in. is there any config i should add to
> solve
> > > this problem. thanks !
> >
> >
> >
> >
> > --
> > Best regards,
> > Amuro Copperfield
> >
>
>
>
> --
> With Warm regards
>
> Yiming Liu (刘一鸣)
>
Re: java.lang.OutOfMemoryError: Java heap space
Posted by "Billy(Yiming) Liu" <li...@gmail.com>.
50,000,000 is ultra high cardinality. What kind of Dictionary did you
choose?
2016-10-11 9:16 GMT+08:00 沙漠火狐 <27...@qq.com>:
> thanks xwhfcenter@gmail.com
>
>
> in /bin/setev.sh my setting is
>
>
> export KYLIN_JVM_SETTINGS="-Xms2048M -Xmx10240M -Xss1024K
> -XX:MaxPermSize=10240M -verbose:gc -XX:+PrintGCDetails
> -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$
> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
>
>
> the problem is still. or i any config setting is wrong?
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Mario Copperfield";<xw...@gmail.com>;
> 发送时间: 2016年10月10日(星期一) 下午5:26
> 收件人: "dev"<de...@kylin.apache.org>;
>
> 主题: Re: java.lang.OutOfMemoryError: Java heap space
>
>
>
> can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh
>
> On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <27...@qq.com> wrote:
>
> > hi
> > I have a big lookuptable about 1.3G, when I cube it, in
> > /logs/kylin.out say :
> >
> >
> > Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start
> > INFO: Server startup in 24004 ms
> > Found segment ordrpath_goodsid[20161008000000_20161009000000]
> > #
> > # java.lang.OutOfMemoryError: Java heap space
> > # -XX:OnOutOfMemoryError="kill -9 %p"
> > # Executing /bin/sh -c "kill -9 23272"...
> >
> >
> >
> > ------------------------------------------------------------
> > then the kylin is stop.
> >
> >
> > so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m
> > -XX:MaxPermSize=4096m" config in catalin.sh
> >
> >
> > and change the config in kylin_job_conf_inmem.xml to
> > <property>
> > <name>mapreduce.map.memory.mb</name>
> > <value>4096</value>
> > <description></description>
> > </property>
> >
> >
> > <property>
> > <name>mapreduce.map.java.opts</name>
> > <value>-Xmx10240m</value>
> > <description></description>
> > </property>
> >
> >
> > I also change the propery in kylin.properies
> > kylin.dictionary.max.cardinality=50000000
> > kylin.table.snapshot.max_mb=1500
> >
> >
> >
> >
> >
> >
> >
> > but the problem is still in. is there any config i should add to solve
> > this problem. thanks !
>
>
>
>
> --
> Best regards,
> Amuro Copperfield
>
--
With Warm regards
Yiming Liu (刘一鸣)