You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Matt Tanquary <ma...@gmail.com> on 2010/11/22 17:43:02 UTC

Cannot find MultipleOutputs files

I am trying to implement MultipleOutputs (the new API). I have done the
following:

Added the following to my Tool run() method:

MultipleOutputs.addNamedOutput(sortJob, "l1n1", TextOutputFormat.class,
NullWritable.class, Text.class);
MultipleOutputs.addNamedOutput(sortJob, "l1n2", TextOutputFormat.class,
NullWritable.class, Text.class);


Here's my reducer:

static class FinalSortReducer extends Reducer<TextTriplet, IntWritable,
NullWritable, Text>
    {
        private MultipleOutputs<NullWritable, Text> mos;

        public void setup(Context context) {
          mos = new MultipleOutputs<NullWritable, Text>(context);
       }
        public void reduce(TextTriplet key, Iterable<IntWritable> values,
                Context context) throws IOException, InterruptedException {

            Text out = new Text();

            for (IntWritable val : values) {
                context.getCounter(TotalCounter.ITEM).increment(1);
                out.set(key.getThird() + "\t" + key.getSecond());
                if (key.getFirst().toString().equals("l1n1"))
                {
                    //mos.write(NullWritable.get(), out,
key.getFirst().toString());
                    mos.write("l1n1", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
                    //logger.info("Wrote: " + key.toString());
                }else if (key.getFirst().toString().equals("l1n2"))
                {
                    //mos.write(NullWritable.get(), out,
key.getFirst().toString());
                    mos.write("l1n2", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
                    //logger.info("Wrote: " + key.toString());
                } else
                {
                    context.write(NullWritable.get(), out);
                }
            }
        }

        public void cleanup(Context context) throws IOException,
InterruptedException {
            mos.close();
        }
    }

Once the job is finished, I end up with nothing in my output path although
statistics show that my reducer output 83 records.
The only thing found in my output path is '_logs' folder.

I have searched up and down the folder tree to see if maybe the outputs are
somewhere else, but have not been able to find the output.

Any idea about what I might be doing wrong?

Thanks!
-M@

-- 
Have you thanked a teacher today? ---> http://www.liftateacher.org