memory usage tasks

[Koert Kuipers <ko...@tresata.com>]

silly question, but i have our hadoop slave boxes configured with 7 mappers
each, yet i see java 14 process for user mapred on each box. and each
process takes up about 2GB, which is equals to my memory allocation
(mapred.child.java.opts=-Xmx2048m). so it is using twice as much memory as
i expected! why is that?

Re: memory usage tasks

[Harsh J <ha...@cloudera.com>]

Hey Koert,

Do you have JVM reuse enabled? It may be a bug related to that.

If its still around, can you jstack the idle JVMs (i.e. the JVMs that
lie around when no jobs are running) and attach them somewhere to look
at their state? You'll need to run jstack on their PID as the mapred
user (sudo -u mapred jstack <PID> > jstack.PID.txt usually does it
nicely).

On Sat, Jun 9, 2012 at 6:19 AM, Koert Kuipers <ko...@tresata.com> wrote:
> silly question, but i have our hadoop slave boxes configured with 7 mappers
> each, yet i see java 14 process for user mapred on each box. and each
> process takes up about 2GB, which is equals to my memory allocation
> (mapred.child.java.opts=-Xmx2048m). so it is using twice as much memory as
> i expected! why is that?


Thanks,
-- 
Harsh J