You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by 我是天才 <50...@qq.com> on 2020/05/07 01:47:05 UTC

回复:erros on building cube in kylin

if i want to add "kylin.engine.mr.config-override"&nbsp; in the front of them.
MY config below as:&nbsp;
kylin_job_conf.xml
&nbsp;<property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.map.java.opts.max.heap</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp;<value&gt;204800</value&gt;
&nbsp; &nbsp; </property&gt;
&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.reduce.java.opts.max.heap</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;204800</value&gt;
&nbsp; &nbsp; </property&gt;
&nbsp;<property&gt;

&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.map.java.opts</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;-Xmx8000m -XX:OnOutOfMemoryError='kill -9 %p'</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;
&nbsp; &nbsp; </property&gt;
&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.reduce.java.opts</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;-Xmx8000m -XX:OnOutOfMemoryError='kill -9 %p'</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;
&nbsp; &nbsp; </property&gt;
&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapred.child.java.opt</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;-Xmx8000m -XX:OnOutOfMemoryError='kill -9 %p'</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;



kylin_job_conf_inmem.xml
<property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.map.memory.mb</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;8192</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;
&nbsp; &nbsp; </property&gt;


&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.map.java.opts</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;-Xmx12g -XX:OnOutOfMemoryError='kill -9 %p'</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;
&nbsp; &nbsp; </property&gt;


&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;mapreduce.task.io.sort.mb</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;20</value&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <description&gt;</description&gt;
&nbsp; &nbsp; </property&gt;





------------------&nbsp;原始邮件&nbsp;------------------
发件人:&nbsp;"Xiaoxiang Yu"<xxyu@apache.org&gt;;
发送时间:&nbsp;2020年5月6日(星期三) 晚上7:57
收件人:&nbsp;"user"<user@kylin.apache.org&gt;;"我是天才"<50308969@qq.com&gt;;

主题:&nbsp;Re:erros on building  cube  in kylin



 Hi,
&nbsp; &nbsp; Have you ever check this doc : http://kylin.apache.org/docs/install/configuration.html#mr-config-override&nbsp;and increase the heap of Mapper or Reducer?







--
Best wishes to you ! 
From :Xiaoxiang Yu







At 2020-05-06 19:45:43, "我是天才" <50308969@qq.com&gt; wrote:
Hello:
&nbsp; &nbsp; &nbsp;I have a table and it has three columns,it has 5736604 rows,the three columns distinct value counts is 1843696	5736485	1742356.I build cube and got a error on 3rd step,it said:"Error: Java heap space",the step name is "&nbsp;Extract Fact Table Distinct Columns".
Can you help me? thanks!