Role of AM in TEZ as execution engine

[Raghuraman Murugaiyan <ra...@gmail.com>]

Hi All,

 I am new to TEZ and could you please help to understand how TEZ works and
how we can control the Number of mappers and can we execute instead of
increasing the AM memory (tez.am.resource.memory.mb) and TEZ container
size. We are dealing with huge volume of Data of size 4.5 TB per partition.
Is TEZ advisable for these much high volume of data?

In our Cluster We are running a Hive query with TEZ as execution engine and
we faced an issue like the below :

Container *XYZ* is running beyond physical memory limits. Current usage:
1.1 GB of 1 GB physical memory used; 2.7 GB of 4 GB virtual memory used.
Killing container

Initially, the  AM memory was 1 GB and TEZ container size was 1 GB. I
increased the AM memory to 6 GB and TEZ container size to 2 GB, then the
query ran fine. My question is , Why we need to increase the AM memory
here? what is the exact role AM is doing here in tez and different from MR.

Regards,
Raghu M