[jira] [Created] (HADOOP-19835) Make MapReduce Application Master class configurable in YARNRunner

["Lewis John McGibbney (Jira)" <ji...@apache.org>]

Lewis John McGibbney created HADOOP-19835:
---------------------------------------------

             Summary: Make MapReduce Application Master class configurable in YARNRunner
                 Key: HADOOP-19835
                 URL: https://issues.apache.org/jira/browse/HADOOP-19835
             Project: Hadoop Common
          Issue Type: Bug
          Components: mapreduce
    Affects Versions: 3.4.3
            Reporter: Lewis John McGibbney
             Fix For: 3.4.4


YARNRunner currently always uses {{MRJobConfig.APPLICATION_MASTER_CLASS}} when building the command to start the Application Master, and does not read {{{}yarn.app.mapreduce.am{}}}. As a result, custom AM classes (e.g. Celeborn’s {{{}MRAppMasterWithCeleborn{}}}) are never used when submitting jobs via the standard job client.

This patch changes YARNRunner to take the AM class from configuration: it uses {{jobConf.get("yarn.app.mapreduce.am", MRJobConfig.APPLICATION_MASTER_CLASS)}} and passes that value into the launch arguments. If {{yarn.app.mapreduce.am}} is set, that class is used; otherwise behavior is unchanged (default AM class). This allows remote shuffle and other custom AMs to be used without forking the job client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org