You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Omid Vahdaty <om...@gmail.com> on 2020/08/13 11:55:30 UTC

RAM alloaction

HI,

Just installed kyling using docker.

I performed a job and got a reduce error

i have tried increasing the amount of RAM configuration for yarn get same
error

i twould seem that regardless of what i configure i keep getting yarn total
moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB of ram
and i have not limited the container to any limit.

what is the correct way to allocate RAM for a kylin job or generally
speaking?

Re: RAM alloaction

Posted by Xiaoxiang Yu <xx...@apache.org>.
Hi Omid, 
    Could you please show us which kind of trouble do you met ? We did not face similar trouble in our testing. Please show us the error log/ screenshot or other information (kylin.log)?
    On the another hand, you can edit "/etc/hadoop/conf/yarn-site.xml" and modify it; after that, you can restart Resource Manager and Node Manager to make it take effect. 










--

Best wishes to you ! 
From ：Xiaoxiang Yu




在 2020-09-20 22:26:20，"Omid Vahdaty" <om...@gmail.com> 写道：

Hi,


Thank you for fast response


1. I am using the official kylin docker, Ideally - Where would set:
  - yarn.nodemanager.resource.memory-mb
    - yarn.scheduler.maximum-allocation-mb
as I tried many config files - nothing takes. 
2. As for the JVM, thanks for the feedback, I would research that. I am not  KVM Expert
Generally speaking What would you recommend as the max size for JVM?




On Sun, Sep 20, 2020 at 4:59 PM Xiaoxiang Yu <xx...@apache.org> wrote:

Hi friend,
     Have you ever check these paramter in your Yarn Configuration? 
    - yarn.nodemanager.resource.memory-mb
    - yarn.scheduler.maximum-allocation-mb


    On the another hand, it is unusual to allocate such a huge size for a single JVM, unless you have special experience on JVM performance tuning. I wonder if you could do a rearch about Hadoop/JVM, and I guess it maybe helpful.


    
   













--

Best wishes to you ! 
From ：Xiaoxiang Yu




在 2020-09-20 18:45:58，"Omid Vahdaty" <om...@gmail.com> 写道：

Hi,


Yes, I tried to overwrite the following:
kylin.job.mr.config.override.mapreduce.map.java.opts -Xmx256000m

kylin.job.mr.config.override.mapreduce.map.memory.mb 320000

kylin.job.mr.config.override.mapreduce.reduce.memory.mb 640000

kylin.job.mr.config.override.mapreduce.reduce.java.opts-Xmx512000m



and increase set env :

export KYLIN_JVM_SETTINGS="-Xms8g -Xmx300g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"



still getting errors as if i have not changed the RAM. 
and again the machine has 1.5 TB RAM. 


Please advice, any help will be appreciated.


Thanks
Omid




On Fri, Aug 14, 2020 at 4:50 AM Yaqian Zhang <Ya...@126.com> wrote:

Hi:
Do you use this method to configure the yarn parameter in Kylin?




在 2020年8月13日，19:55，Omid Vahdaty <om...@gmail.com> 写道：


HI,


Just installed kyling using docker.


I performed a job and got a reduce error


i have tried increasing the amount of RAM configuration for yarn get same error


i twould seem that regardless of what i configure i keep getting yarn total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB of ram and i have not limited the container to any limit.


what is the correct way to allocate RAM for a kylin job or generally speaking?

Re: RAM alloaction

Posted by Omid Vahdaty <om...@gmail.com>.
Hi,

Thank you for fast response

1. I am using the official kylin docker, Ideally - Where would set:
  - yarn.nodemanager.resource.memory-mb
    - yarn.scheduler.maximum-allocation-mb
as I tried many config files - nothing takes.
2. As for the JVM, thanks for the feedback, I would research that. I am
not  KVM Expert
Generally speaking What would you recommend as the max size for JVM?


On Sun, Sep 20, 2020 at 4:59 PM Xiaoxiang Yu <xx...@apache.org> wrote:

> Hi friend,
>      Have you ever check these paramter in your Yarn Configuration?
>     - yarn.nodemanager.resource.memory-mb
>     - yarn.scheduler.maximum-allocation-mb
>
>     On the another hand, it is unusual to allocate such a huge size for a
> single JVM, unless you have special experience on JVM performance tuning. I
> wonder if you could do a rearch about Hadoop/JVM, and I guess it maybe
> helpful.
>
>
>
>
>
>
>
>
> --
> *Best wishes to you ! *
> *From ：**Xiaoxiang Yu*
>
>
> 在 2020-09-20 18:45:58，"Omid Vahdaty" <om...@gmail.com> 写道：
>
> Hi,
>
> Yes, I tried to overwrite the following:
> kylin.job.mr.config.override.mapreduce.map.java.opts -Xmx256000m
> kylin.job.mr.config.override.mapreduce.map.memory.mb 320000
> kylin.job.mr.config.override.mapreduce.reduce.memory.mb 640000
> kylin.job.mr.config.override.mapreduce.reduce.java.opts-Xmx512000m
>
> and increase set env :
>
> export KYLIN_JVM_SETTINGS="-Xms8g -Xmx300g -XX:MaxPermSize=512m
> -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4
> -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled
> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
> -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly
> -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc
> -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation
> -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M
> -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"
>
> still getting errors as if i have not changed the RAM.
> and again the machine has 1.5 TB RAM.
>
> Please advice, any help will be appreciated.
>
> Thanks
> Omid
>
>
> On Fri, Aug 14, 2020 at 4:50 AM Yaqian Zhang <Ya...@126.com> wrote:
>
>> Hi:
>> Do you use this method to configure the yarn parameter in Kylin?
>>
>>
>> 在 2020年8月13日，19:55，Omid Vahdaty <om...@gmail.com> 写道：
>>
>> HI,
>>
>> Just installed kyling using docker.
>>
>> I performed a job and got a reduce error
>>
>> i have tried increasing the amount of RAM configuration for yarn get same
>> error
>>
>> i twould seem that regardless of what i configure i keep getting yarn
>> total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB
>> of ram and i have not limited the container to any limit.
>>
>> what is the correct way to allocate RAM for a kylin job or generally
>> speaking?
>>
>>
>>
>>

Re: RAM alloaction

Posted by Xiaoxiang Yu <xx...@apache.org>.
Hi friend,
     Have you ever check these paramter in your Yarn Configuration? 
    - yarn.nodemanager.resource.memory-mb
    - yarn.scheduler.maximum-allocation-mb


    On the another hand, it is unusual to allocate such a huge size for a single JVM, unless you have special experience on JVM performance tuning. I wonder if you could do a rearch about Hadoop/JVM, and I guess it maybe helpful.


    
   













--

Best wishes to you ! 
From ：Xiaoxiang Yu




在 2020-09-20 18:45:58，"Omid Vahdaty" <om...@gmail.com> 写道：

Hi,


Yes, I tried to overwrite the following:
kylin.job.mr.config.override.mapreduce.map.java.opts -Xmx256000m

kylin.job.mr.config.override.mapreduce.map.memory.mb 320000

kylin.job.mr.config.override.mapreduce.reduce.memory.mb 640000

kylin.job.mr.config.override.mapreduce.reduce.java.opts-Xmx512000m



and increase set env :

export KYLIN_JVM_SETTINGS="-Xms8g -Xmx300g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"



still getting errors as if i have not changed the RAM. 
and again the machine has 1.5 TB RAM. 


Please advice, any help will be appreciated.


Thanks
Omid




On Fri, Aug 14, 2020 at 4:50 AM Yaqian Zhang <Ya...@126.com> wrote:

Hi:
Do you use this method to configure the yarn parameter in Kylin?




在 2020年8月13日，19:55，Omid Vahdaty <om...@gmail.com> 写道：


HI,


Just installed kyling using docker.


I performed a job and got a reduce error


i have tried increasing the amount of RAM configuration for yarn get same error


i twould seem that regardless of what i configure i keep getting yarn total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB of ram and i have not limited the container to any limit.


what is the correct way to allocate RAM for a kylin job or generally speaking?

Re: RAM alloaction

Posted by Omid Vahdaty <om...@gmail.com>.
Hi,

Yes, I tried to overwrite the following:
kylin.job.mr.config.override.mapreduce.map.java.opts -Xmx256000m
kylin.job.mr.config.override.mapreduce.map.memory.mb 320000
kylin.job.mr.config.override.mapreduce.reduce.memory.mb 640000
kylin.job.mr.config.override.mapreduce.reduce.java.opts-Xmx512000m

and increase set env :

export KYLIN_JVM_SETTINGS="-Xms8g -Xmx300g -XX:MaxPermSize=512m
-XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4
-XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled
-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
-XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M
-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"

still getting errors as if i have not changed the RAM.
and again the machine has 1.5 TB RAM.

Please advice, any help will be appreciated.

Thanks
Omid


On Fri, Aug 14, 2020 at 4:50 AM Yaqian Zhang <Ya...@126.com> wrote:

> Hi:
> Do you use this method to configure the yarn parameter in Kylin?
>
>
> 在 2020年8月13日，19:55，Omid Vahdaty <om...@gmail.com> 写道：
>
> HI,
>
> Just installed kyling using docker.
>
> I performed a job and got a reduce error
>
> i have tried increasing the amount of RAM configuration for yarn get same
> error
>
> i twould seem that regardless of what i configure i keep getting yarn
> total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB
> of ram and i have not limited the container to any limit.
>
> what is the correct way to allocate RAM for a kylin job or generally
> speaking?
>
>
>
>

回复： RAM alloaction

Posted by 恩爸 <44...@qq.com>.
Hi:
&nbsp; You need to increase the ram of docker first, use below command to run docker:
&nbsp; &nbsp; &nbsp;docker run -m&nbsp;16G&nbsp;-p 7070:7070 -p 8088:8088 -p 50070:50070 -p 8032:8032 -p 8042:8042 -p 16010:16010 -p 8998:8998 -p 2181:2181&nbsp; apachekylin/apache-kylin-standalone:3.1.0


&nbsp; and then, increase the total memory of yarn' nm, for example:&nbsp;
&nbsp; &nbsp; <property&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <name&gt;yarn.nodemanager.resource.memory-mb</name&gt;
&nbsp; &nbsp; &nbsp; &nbsp; <value&gt;10240</value&gt;
&nbsp; &nbsp; </property&gt;


&nbsp; &nbsp;restart the yarn or docker.


&nbsp; &nbsp;At last, use below method&nbsp;Yaqian_Zhang&nbsp;mentioned to configure the yarn parameters in kylin and try again:



&nbsp; &nbsp;







Best regards,
Zhichao Zhang




&nbsp;




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "user"                                                                                    <Yaqian_Zhang@126.com&gt;;
发送时间:&nbsp;2020年8月14日(星期五) 上午9:49
收件人:&nbsp;"user"<user@kylin.apache.org&gt;;

主题:&nbsp;Re: RAM alloaction



Hi:Do you use this method to configure the yarn parameter in Kylin?


在 2020年8月13日，19:55，Omid Vahdaty <omidvd@gmail.com&gt; 写道：

HI,

Just installed kyling using docker.


I performed a job and got a reduce error


i have tried increasing the amount of RAM configuration for yarn get same error


i twould seem that regardless of what i configure i keep getting yarn total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB of ram and i have not limited the container to any limit.


what is the correct way to allocate RAM for a kylin job or generally speaking?

Re: RAM alloaction

Posted by Yaqian Zhang <Ya...@126.com>.
Hi:
Do you use this method to configure the yarn parameter in Kylin?


> 在 2020年8月13日，19:55，Omid Vahdaty <om...@gmail.com> 写道：
> 
> HI,
> 
> Just installed kyling using docker.
> 
> I performed a job and got a reduce error
> 
> i have tried increasing the amount of RAM configuration for yarn get same error
> 
> i twould seem that regardless of what i configure i keep getting yarn total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB of ram and i have not limited the container to any limit.
> 
> what is the correct way to allocate RAM for a kylin job or generally speaking?
> 
>