[jira] [Commented] (TEZ-704) Tez MapReduce job hangs if setting mapreduce.reduce.cpu.vcores to be 2

["Jian Fang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14082969#comment-14082969 ] 

Jian Fang commented on TEZ-704:
-------------------------------

After debugging tez and hadoop code, it turned out to be a problem in hadoop 2.4.x code, where the vcore was reset to 1, which caused the mismatch. I linked the JIRA YARN-2380 here.

> Tez MapReduce job hangs if setting mapreduce.reduce.cpu.vcores to be 2
> ----------------------------------------------------------------------
>
>                 Key: TEZ-704
>                 URL: https://issues.apache.org/jira/browse/TEZ-704
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>
> When running Hive on Tez in EMR hadoop districution, which set mapreduce.reduce.cpu.vcores to be 2 in mapred-site.xml, all MapReduce jobs get hang, without any progress.
> The problem seems like,
> Hive On Tez sets mapreduce.reduce.cpu.vcores to 2, so its application
> manager asks for a 2 cpu container, but the resource manager could not
> allocate a node for it, since all it has are 1 cpu containers.
> I saw the same problem when running MapReduce Job on Tez when setting mapreduce.reduce.cpu.vcores to be 2.
> This could be a bug in Tez MapReduce code.



--
This message was sent by Atlassian JIRA
(v6.2#6252)