You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Michel Tourn (JIRA)" <ji...@apache.org> on 2006/02/21 23:26:44 UTC
[jira] Created: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
JobClient cannot use a non-default server (unlike DFSShell)
-----------------------------------------------------------
Key: HADOOP-49
URL: http://issues.apache.org/jira/browse/HADOOP-49
Project: Hadoop
Type: Improvement
Reporter: Michel Tourn
JobClient cannot use a non-default Job Tracker server:
It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
Other hadoop command-line tools like DFSShell already have:
>bin/hadoop dfs
Usage: java DFSShell [-local | -dfs <namenode:port>] ...
Similarly I propose to add a -jt parameter:
>bin/hadoop job
JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
And: -jt <config> will load as a final resource: hadoop-<config>.xml
The latter syntax is discoverable by users because on failure the tool will say:
>bin/hadoop job -kill m7n6pi -jt unknown
Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
Or in case of success:
>bin/hadoop job -kill job_m7n6pi -jt myconfig
060221 221911 parsing file:/trunk/conf/hadoop-default.xml
060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
060221 221911 parsing file:/trunk/conf/hadoop-site.xml
060221 221911 Client connection to 66.196.91.10:7020: starting
And with a machine:port spec:
>bin/hadoop job -kill job_m7n6pi -jt machine:8020
060221 222109 parsing file:/trunk/conf/hadoop-default.xml
060221 222109 parsing file:/trunk/conf/hadoop-site.xml
060221 222109 Client connection to 66.196.91.10:8020: starting
Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Doug Cutting resolved HADOOP-49:
--------------------------------
Resolution: Fixed
Assign To: Michel Tourn
I just committed this.
Longer-term, perhaps we should have some standard command line options that all tools accept. This could be implemented by having bin/hadoop always invoke a single class, e.g., org.apache.hadoop.ToolRunner. This might looks something like
public class ToolRunner {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
... initialize configuration from command line options..
... get tool class from command line options ...
Tool tool = toolClass.newInstance();
tool.configure(conf);
tool.run(args);
}
}
public interface Tool extends Configurable {
run(String[] args);
}
If folks like this, we file this it as a separate issue..
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Reporter: Michel Tourn
> Assignee: Michel Tourn
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Doug Cutting updated HADOOP-49:
-------------------------------
Fix Version: 0.1.0
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Reporter: Michel Tourn
> Assignee: Michel Tourn
> Fix For: 0.1.0
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Michel Tourn (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Michel Tourn updated HADOOP-49:
-------------------------------
Attachment: jcjt.patch
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Reporter: Michel Tourn
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Michel Tourn (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Michel Tourn updated HADOOP-49:
-------------------------------
Attachment: (was: jcjt.patch)
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Reporter: Michel Tourn
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Doug Cutting updated HADOOP-49:
-------------------------------
Component: mapred
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Reporter: Michel Tourn
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Michel Tourn (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Michel Tourn updated HADOOP-49:
-------------------------------
Attachment: jcjt.patch
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Reporter: Michel Tourn
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (HADOOP-49) JobClient cannot use a non-default
server (unlike DFSShell)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-49?page=all ]
Doug Cutting closed HADOOP-49:
------------------------------
> JobClient cannot use a non-default server (unlike DFSShell)
> -----------------------------------------------------------
>
> Key: HADOOP-49
> URL: http://issues.apache.org/jira/browse/HADOOP-49
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Reporter: Michel Tourn
> Assignee: Michel Tourn
> Fix For: 0.1.0
> Attachments: jcjt.patch
>
> JobClient cannot use a non-default Job Tracker server:
> It will use the Job Tracker specified in conf/hadoop-default.xml or conf/hadoop-site.xml
> For users with multiple Hadoop systems, it is useful to be able to specify the Job Tracker.
> Other hadoop command-line tools like DFSShell already have:
> >bin/hadoop dfs
> Usage: java DFSShell [-local | -dfs <namenode:port>] ...
> Similarly I propose to add a -jt parameter:
> >bin/hadoop job
> JobClient -submit <job> | -status <id> | -kill <id> [-jt <jobtracker:port>|<config>]
> Where: -jt <jobtracker:port> is similar to -dfs <namenode:port>
> And: -jt <config> will load as a final resource: hadoop-<config>.xml
> The latter syntax is discoverable by users because on failure the tool will say:
> >bin/hadoop job -kill m7n6pi -jt unknown
> Exception in thread "main" java.lang.RuntimeException: hadoop-unknown.xml not found on CLASSPATH
> Or in case of success:
> >bin/hadoop job -kill job_m7n6pi -jt myconfig
> 060221 221911 parsing file:/trunk/conf/hadoop-default.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-myconfig.xml
> 060221 221911 parsing file:/trunk/conf/hadoop-site.xml
> 060221 221911 Client connection to 66.196.91.10:7020: starting
> And with a machine:port spec:
> >bin/hadoop job -kill job_m7n6pi -jt machine:8020
> 060221 222109 parsing file:/trunk/conf/hadoop-default.xml
> 060221 222109 parsing file:/trunk/conf/hadoop-site.xml
> 060221 222109 Client connection to 66.196.91.10:8020: starting
> Patch attached.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira