You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "Ruiz, Pierre" <Pi...@analog.com> on 2018/02/27 11:04:09 UTC
passing conf instance around
Hi all,
I'm quite new to Hadopp and only worked with a single node setup so far. I wrote a local driver that submits Jobs to my cluster. I instantiate a single Configuration instance right at the start of my process, and pass it around like that:
public static void main(String[] args) {
int exitCode = ERR_INVALID_ARGS.get();
Configuration conf = new Configuration(true);
try {
if (- == args[0].charAt(0)) {
exitCode = runSelectedTool(conf, args);
} else if (args.length >= 2 && args.length <= 3) {
exitCode = crunchFullDataset(conf, args);
}
} catch (Exception e){
e.printStackTrace();
exitCode = ERR_FATAL.get();
}
System.exit(exitCode);
}
private static int runSelectedTool(Configuration conf, String[] args) throws Exception {
int exitCode;
String toolSwitch = args[0];
args = Arrays.copyOfRange(args,1,args.length);
if (SWITCH_FORMATTER_COUNTER.equals(toolSwitch)) {
exitCode = ToolRunner.run(conf, new FormatterCounter(), args);
} else if (SWITCH_CANDIDATES_FILTER.equals(toolSwitch)) {
exitCode = ToolRunner.run(conf, new CandidatesFilter(), args);
}
}
Prior to this, I was instantiating a new conf object each time I called ToolRunner.run(), but now I use conf.set() & get() to pass values between jobs. Is it a bad idea (and why), or this the right way to proceed?
Many thanks,
Pierre