You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Gang Luo <lg...@yahoo.com.cn> on 2010/01/13 20:47:57 UTC

memory for map task

Hi all,
the parameter mapred.child.java.opts defines the memory for each task (e.g. 200mb), while io.sort.mb defines the memory size for sorting (e.g. 100mb). So, for each map task, the task memory minus io memory is the maximum memory the map function can use. Is it correct?

-Gang



      ___________________________________________________________ 
  好玩贺卡等你发，邮箱贺卡全新上线！ 
http://card.mail.cn.yahoo.com/

Re: memory for map task

Posted by Todd Lipcon <to...@cloudera.com>.

More or less correct, minus some wiggle room for JVM, accounting structures,
thread stacks, etc.

-Todd

2010/1/13 Gang Luo <lg...@yahoo.com.cn>

> Hi all,
> the parameter mapred.child.java.opts defines the memory for each task (e.g.
> 200mb), while io.sort.mb defines the memory size for sorting (e.g. 100mb).
> So, for each map task, the task memory minus io memory is the maximum memory
> the map function can use. Is it correct?
>
> -Gang
>
>
>
>      ___________________________________________________________
>  好玩贺卡等你发，邮箱贺卡全新上线！
> http://card.mail.cn.yahoo.com/
>