[jira] [Created] (HDFS-15475) -D mapreduce.framework.name CLI parameter for miniCluster not working

["Xiang Zhang (Jira)" <ji...@apache.org>]

Xiang Zhang created HDFS-15475:
----------------------------------

             Summary: -D mapreduce.framework.name CLI parameter for miniCluster not working
                 Key: HDFS-15475
                 URL: https://issues.apache.org/jira/browse/HDFS-15475
             Project: Hadoop HDFS
          Issue Type: Improvement
    Affects Versions: 2.7.0
            Reporter: Xiang Zhang


I am running miniCluster using doc here: [https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/CLIMiniCluster.html.|https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/CLIMiniCluster.html]

I notice that by default mapreduce jobs do not run on YARN and I understand that setting mapreduce.framework.name to yarn can make this work.
{code:java}
<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>{code}
So I tried to add this in etc/hadoop/mapreduces-site.xml and managed to run a wordcount example through this:
{code:java}
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount hdfs://localhost:8020/user/iamabug/input hdfs://localhost:8020/user/iamabug/output
{code}
However, according to the doc, this parameter should also be available to be set through -D parameter, i.e.,
{code:java}
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar minicluster  -format -D mapreduce.framework.name=yarn -writeConfig 2.txt
{code}
Notice that I write config to 2.txt and the parameter can be found in this file:
{code:java}
<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    <source>programatically</source>
</property>
{code}
I submitted a wordcount example again and it didn't run on YARN according to the logs and YARN Web UI ([http://localhost:8088).|http://localhost:8088)./]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org