You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Matt Tanquary <ma...@gmail.com> on 2010/11/22 17:43:02 UTC
Cannot find MultipleOutputs files
I am trying to implement MultipleOutputs (the new API). I have done the
following:
Added the following to my Tool run() method:
MultipleOutputs.addNamedOutput(sortJob, "l1n1", TextOutputFormat.class,
NullWritable.class, Text.class);
MultipleOutputs.addNamedOutput(sortJob, "l1n2", TextOutputFormat.class,
NullWritable.class, Text.class);
Here's my reducer:
static class FinalSortReducer extends Reducer<TextTriplet, IntWritable,
NullWritable, Text>
{
private MultipleOutputs<NullWritable, Text> mos;
public void setup(Context context) {
mos = new MultipleOutputs<NullWritable, Text>(context);
}
public void reduce(TextTriplet key, Iterable<IntWritable> values,
Context context) throws IOException, InterruptedException {
Text out = new Text();
for (IntWritable val : values) {
context.getCounter(TotalCounter.ITEM).increment(1);
out.set(key.getThird() + "\t" + key.getSecond());
if (key.getFirst().toString().equals("l1n1"))
{
//mos.write(NullWritable.get(), out,
key.getFirst().toString());
mos.write("l1n1", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
//logger.info("Wrote: " + key.toString());
}else if (key.getFirst().toString().equals("l1n2"))
{
//mos.write(NullWritable.get(), out,
key.getFirst().toString());
mos.write("l1n2", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
//logger.info("Wrote: " + key.toString());
} else
{
context.write(NullWritable.get(), out);
}
}
}
public void cleanup(Context context) throws IOException,
InterruptedException {
mos.close();
}
}
Once the job is finished, I end up with nothing in my output path although
statistics show that my reducer output 83 records.
The only thing found in my output path is '_logs' folder.
I have searched up and down the folder tree to see if maybe the outputs are
somewhere else, but have not been able to find the output.
Any idea about what I might be doing wrong?
Thanks!
-M@
--
Have you thanked a teacher today? ---> http://www.liftateacher.org