Re: AVRO MultipleOutput

[Harsh J <ha...@cloudera.com>]

Please use AvroMultipleOutputs everywhere you need to reference it. Do
not use the regular MR MultipleOutputs, as the Avro implementation is
standalone and has nothing in common with the MR API variant.

On Tue, Jan 13, 2015 at 12:18 PM, Ankur Jain <an...@yash.com> wrote:
> Please ignore last one…
>
>
>
> Thanks Deepak,
>
>
>
> I tried using below but that also not helping in changing output file name…
>
>
>
> In Job file
>
> MultipleOutputs.addNamedOutput(job, "clean", AvroKeyOutputFormat.class,
> Measurement.class , NullWritable.class);
>
>
>
> In Reducer
>
> Setup:
>
> MultipleOutputs<Measurement, NullWritable> mos;
>
>
>
>        @Override
>
>        protected void setup(Context context) throws IOException,
> InterruptedException {
>
>               mos = new MultipleOutputs<Measurement, NullWritable>(context);
>
>        }
>
>
>
> And in reducer method
>
> if (isClean) {
>
>               mos.write("clean", value.datum(), "clean");
>
>        } else {
>
>               mos.write("discarded", value.datum(), "discarded");
>
>        }
>
>
>
> Can you please point on the what I am doing wrong?
>
>
>
> Thank you
>
> Ankur
>
>
>
>
>
>
>
> From: Deepak Sharma [mailto:deepakmca05@gmail.com]
> Sent: Tuesday, January 13, 2015 11:38 AM
> To: user; Ankur Jain
> Subject: Re: AVRO MultipleOutput
>
>
>
> Hi Ankur
>
> Not sure on the avro part , but MR provides addNamedOutputs() that can be
> set in Driver class to rename the output files from part-*** to something
> else.
>
> HTH.
>
> Thanks
> Deepak
>
>
>
> On Tue, Jan 13, 2015 at 11:29 AM, Ankur Jain <an...@yash.com> wrote:
>
> Hi Team,
>
>
>
> My usecase is to create files with different names.
>
> I tried using AVRO mapreduce api for MultipleOutputs using
> org.apache.avro.mapreduce.AvroMultipleOutputs.
>
> I followed Javadoc mentioned for this class and found below issues…
>
>
>
> 1.       We cannot assign AvroMultipleOutputs object to MultipleOutputs.
>
> 2.       Even if I followed all steps mentioned in Javadoc I am still
> getting file with name part-0000
>
>
>
> Can someone please guide me on how I can create files with different name
> using AVRO mapreduce api?
>
> Any working example will be great help….
>
>
>
> I am using HDP 2.2 , avro (1.7.4), JDK 1.7, reading and writing in avro
> files.
>
>
>
> Thanks and Best Regards,
>
> Ankur
>
> Information transmitted by this e-mail is proprietary to YASH Technologies
> and/ or its Customers and is intended for use only by the individual or
> entity to which it is addressed, and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient or it appears that this mail has been
> forwarded to you without proper authority, you are notified that any use or
> dissemination of this information in any manner is strictly prohibited. In
> such cases, please notify us immediately at info@yash.com and delete this
> mail from your records.
>
>
>
>
> --
>
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
> Information transmitted by this e-mail is proprietary to YASH Technologies
> and/ or its Customers and is intended for use only by the individual or
> entity to which it is addressed, and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient or it appears that this mail has been
> forwarded to you without proper authority, you are notified that any use or
> dissemination of this information in any manner is strictly prohibited. In
> such cases, please notify us immediately at info@yash.com and delete this
> mail from your records.



-- 
Harsh J