You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2014/05/23 07:58:19 UTC

should i need tune mapred.child.java.opts option if i use yarn and MRV2?

hi,mailist:
             i want to know if this option still cause limitation in YARN?

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Tsuyoshi OZAWA <oz...@gmail.com>.

Hi,
You need to set mapreduce.map.java.opts and mapreduce.reduce.java.opts
instead of mapred.child.java.opts to tune JVMs when you use MRv2.
Please note that you also need to tune
mapreduce.map.memory.mb/mapreduce.reduce.memory.mb to notify
ResourceManager of the amounts of resource usage.

This doc is also useful for you.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html

Thanks,
- Tsuyoshi

On Thu, May 22, 2014 at 10:58 PM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
>              i want to know if this option still cause limitation in YARN?

-- 
- Tsuyoshi

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Tsuyoshi OZAWA <oz...@gmail.com>.

Hi,
You need to set mapreduce.map.java.opts and mapreduce.reduce.java.opts
instead of mapred.child.java.opts to tune JVMs when you use MRv2.
Please note that you also need to tune
mapreduce.map.memory.mb/mapreduce.reduce.memory.mb to notify
ResourceManager of the amounts of resource usage.

This doc is also useful for you.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html

Thanks,
- Tsuyoshi

On Thu, May 22, 2014 at 10:58 PM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
>              i want to know if this option still cause limitation in YARN?

-- 
- Tsuyoshi

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Bing Jiang <ji...@gmail.com>.

I remember there is a maximum container memory config, It is
ResourceManage-side config.



2014-05-23 13:58 GMT+08:00 ch huang <ju...@gmail.com>:

> hi,mailist:
>              i want to know if this option still cause limitation in YARN?
>



-- 
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Bing Jiang <ji...@gmail.com>.

I remember there is a maximum container memory config, It is
ResourceManage-side config.



2014-05-23 13:58 GMT+08:00 ch huang <ju...@gmail.com>:

> hi,mailist:
>              i want to know if this option still cause limitation in YARN?
>



-- 
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Tsuyoshi OZAWA <oz...@gmail.com>.

Hi,
You need to set mapreduce.map.java.opts and mapreduce.reduce.java.opts
instead of mapred.child.java.opts to tune JVMs when you use MRv2.
Please note that you also need to tune
mapreduce.map.memory.mb/mapreduce.reduce.memory.mb to notify
ResourceManager of the amounts of resource usage.

This doc is also useful for you.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html

Thanks,
- Tsuyoshi

On Thu, May 22, 2014 at 10:58 PM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
>              i want to know if this option still cause limitation in YARN?

-- 
- Tsuyoshi

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Bing Jiang <ji...@gmail.com>.

I remember there is a maximum container memory config, It is
ResourceManage-side config.



2014-05-23 13:58 GMT+08:00 ch huang <ju...@gmail.com>:

> hi,mailist:
>              i want to know if this option still cause limitation in YARN?
>



-- 
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Tsuyoshi OZAWA <oz...@gmail.com>.

Hi,
You need to set mapreduce.map.java.opts and mapreduce.reduce.java.opts
instead of mapred.child.java.opts to tune JVMs when you use MRv2.
Please note that you also need to tune
mapreduce.map.memory.mb/mapreduce.reduce.memory.mb to notify
ResourceManager of the amounts of resource usage.

This doc is also useful for you.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html

Thanks,
- Tsuyoshi

On Thu, May 22, 2014 at 10:58 PM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
>              i want to know if this option still cause limitation in YARN?

-- 
- Tsuyoshi

Re: should i need tune mapred.child.java.opts option if i use yarn and MRV2?

Posted by Bing Jiang <ji...@gmail.com>.

I remember there is a maximum container memory config, It is
ResourceManage-side config.



2014-05-23 13:58 GMT+08:00 ch huang <ju...@gmail.com>:

> hi,mailist:
>              i want to know if this option still cause limitation in YARN?
>



-- 
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase