You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tez.apache.org by Ranjan Banerjee <ra...@microsoft.com> on 2016/12/17 19:23:14 UTC

Difference between COMMITTED_HEAP_BYTES and PHYSICAL_MEMORY_BYTES

Hi,
   Could someone please explain the difference between PHYSICAL_MEMORY_BYTES and COMMITTED_HEAP_BYTES.
    From my understanding COMMITTED_HEAP_BYTES is the xmx value in the tez container and PHYSICAL_MEMORY_BYTES is the actual memory used by the task.

But in my sample queries I see both of them to be the exact value, does this mean that my task is using 100% of the memory assigned? Some clarification would be greatly appreciated!

Thanks
Ranjan

Re: Difference between COMMITTED_HEAP_BYTES and PHYSICAL_MEMORY_BYTES

Posted by Rajesh Balamohan <rb...@apache.org>.

Hi,

PHYSICAL_MEMORY_BYTES represents the cumulative RSS memory of the process
tree and COMMITTED_HEAP_BYTES is the current total committed heap space
usage of this JVM. COMMITTED_HEAP_BYTES is computed via
Runtime.getRuntime.totalMemory().

But when TezMxBeanResourceCalculator is used which is the default in recent
tez versions, PHYSICAL_MEMORY_BYTES is also computed via
getRuntime().totalMemory(). Hence in your queries, you see both the values
representing the same value.

https://issues.apache.org/jira/browse/TEZ-1698 has more details on this.

~Rajesh.B

On Sun, Dec 18, 2016 at 12:53 AM, Ranjan Banerjee <ra...@microsoft.com>
wrote:

> Hi,
>    Could someone please explain the difference between
> PHYSICAL_MEMORY_BYTES and COMMITTED_HEAP_BYTES.
>     From my understanding COMMITTED_HEAP_BYTES is the xmx value in the tez
> container and PHYSICAL_MEMORY_BYTES is the actual memory used by the task.
>
> But in my sample queries I see both of them to be the exact value, does
> this mean that my task is using 100% of the memory assigned? Some
> clarification would be greatly appreciated!
>
> Thanks
> Ranjan
>