Container memory computing

[牛兆捷 <nz...@gmail.com>]

Hi, all:
This is the log of my nodemanager, I set 4GB memory for memory task and
virtual memory ratio to 2.5, so the total virtual memory is 10GB.

*log*:
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 725 for container-id
container_1367846809555_0003_01_000002: 943.9 MB of 4 GB physical memory
used; 2.4 GB of 10 GB virtual memory used

1)Do you know how yarn calculate the actual physical and virtual memory
usage? How can I add log to the container and print the usage for each
container?

2)BTW, I set task input to 1024mb and io.sort.mb to 512mb for experiment,
however, I find the total actual physical memory waste for about 900mb. Why
it looks like this, doesn't it will spill when the memory consumption is
more than 512(io.sort.mb)?
3)How can I set memory and cpu for hadoop sort example to make full use of
resource?
I have 24GB and 24core per machine, the input size is 256mb, so can I set
the vcores to
less than *1*?

-- 
*Sincerely,*
*Zhaojie*
*
*