[jira] [Created] (MAPREDUCE-5897) Provide a utility to be able inspect the config as seen by a hadoop client daemon

["Gera Shegalov (JIRA)" <ji...@apache.org>]

Gera Shegalov created MAPREDUCE-5897:
----------------------------------------

             Summary: Provide a utility to be able inspect the config as seen by a hadoop client daemon 
                 Key: MAPREDUCE-5897
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5897
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
            Reporter: Gera Shegalov
            Assignee: Gera Shegalov


To ease debugging of config issues it is convenient to be able to generate a config as seen by the job client or a hadoop daemon

{noformat}
]$ hadoop org.apache.hadoop.util.ConfigTool -help 
Usage: ConfigTool [ -xml | -json ] [ -loadDefaults ] [ resource1... ]
      if resource contains '/', load from local filesystem
      otherwise, load from the classpath

Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.

The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
{noformat}

{noformat}
$ hadoop org.apache.hadoop.util.ConfigTool -Dmy.test.conf=val mapred-site.xml ./hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/etc/hadoop/core-site.xml | python -mjson.tool
{
    "properties": [
        {
            "isFinal": false,
            "key": "mapreduce.framework.name",
            "resource": "mapred-site.xml",
            "value": "yarn"
        },
        {
            "isFinal": false,
            "key": "mapreduce.client.genericoptionsparser.used",
            "resource": "programatically",
            "value": "true"
        },
        {
            "isFinal": false,
            "key": "my.test.conf",
            "resource": "from command line",
            "value": "val"
        },
        {
            "isFinal": false,
            "key": "from.file.key",
            "resource": "hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/etc/hadoop/core-site.xml",
            "value": "from.file.val"
        },
        {
            "isFinal": false,
            "key": "mapreduce.shuffle.port",
            "resource": "mapred-site.xml",
            "value": "${my.mapreduce.shuffle.port}"
        }
    ]
}
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)