Getting ClassCastException while using GenericRecord in Mapper

[Pratyush Chandra <ch...@gmail.com>]

Hi,

I have a simple schema say:

{
    "namespace": "example.proto",
    "name": "Message", "type": "record",
    "fields": [
          {"name": "user",   "type": ["string","null"]}
    ]
}

When I am running map reduce with following mapper :
  public static class IdentityAvroMapper extends  AvroMapper<GenericRecord,
Pair<GenericRecord, GenericRecord>> {
        public void map(GenericRecord text,
                AvroCollector<Pair<GenericRecord, GenericRecord>>
collector, Reporter reporter) throws IOException {
            collector.collect(new Pair<GenericRecord, GenericRecord>(text,
text));
        }
    }

It throws me error "java.lang.ClassCastException: example.proto.Message
cannot be cast to org.apache.avro.generic.GenericRecord"
It works fine if I use Message (generated class) in place of GenericRecord.

So, if I do not wish to use generated code, why am I not able to cast it to
GenericRecord. I want to be able to use map reduce with generic apis. Any
pointers please ?


Regards
Pratyush Chandra

Re: Getting ClassCastException while using GenericRecord in Mapper

[Terry Healy <th...@bnl.gov>]

Pratysuh-

First a HUGE caution that I am an AVRO M/R beginner, but I faced a
similar issue. I think that you are not extending AvroMapper as intended.

Here is a part of my similar code wich runs:

    public static class AvroFlowMapper<K> extends AvroMapper<K,
Pair<Long, GenericRecord>> {


        private GenericRecord genRec = new
GenericData.Record(NETFLOW_V5_SCHEMA);


        @Override
        public void map(K datum, AvroCollector<Pair<Long,
GenericRecord>> collector,
                Reporter reporter) throws IOException {

            genRec = (GenericRecord) datum;
            AvroFlowWritable afw = new AvroFlowWritable(genRec);
	    //....
Where AvroFlowWritable is my class as the content of which is described
by NETFLOW_V5_SCHEMA.

Again, others will certainly have better explanation / examples. I hope
this helps you get started.

-Terry

	

On 01/14/2013 08:33 AM, Pratyush Chandra wrote:
> When I am running map reduce with following mapper :
>   public static class IdentityAvroMapper extends 
> AvroMapper<GenericRecord, Pair<GenericRecord, GenericRecord>> {
>         public void map(GenericRecord text,
>                 AvroCollector<Pair<GenericRecord, GenericRecord>>
> collector, Reporter reporter) throws IOException {
>             collector.collect(new Pair<GenericRecord,
> GenericRecord>(text, text));
>         }
>     }

Re: Getting ClassCastException while using GenericRecord in Mapper

[Pratyush Chandra <ch...@gmail.com>]

Can somebody help me with this ?

Thanks
Pratyush

On Thu, Jan 10, 2013 at 1:16 AM, Pratyush Chandra <
chandra.pratyush@gmail.com> wrote:

> Hi,
>
> I have a simple schema say:
>
> {
>     "namespace": "example.proto",
>     "name": "Message", "type": "record",
>     "fields": [
>           {"name": "user",   "type": ["string","null"]}
>     ]
> }
>
> When I am running map reduce with following mapper :
>   public static class IdentityAvroMapper extends
> AvroMapper<GenericRecord, Pair<GenericRecord, GenericRecord>> {
>         public void map(GenericRecord text,
>                 AvroCollector<Pair<GenericRecord, GenericRecord>>
> collector, Reporter reporter) throws IOException {
>             collector.collect(new Pair<GenericRecord, GenericRecord>(text,
> text));
>         }
>     }
>
> It throws me error "java.lang.ClassCastException: example.proto.Message
> cannot be cast to org.apache.avro.generic.GenericRecord"
> It works fine if I use Message (generated class) in place of GenericRecord.
>
> So, if I do not wish to use generated code, why am I not able to cast it
> to GenericRecord. I want to be able to use map reduce with generic apis.
> Any pointers please ?
>
>
> Regards
> Pratyush Chandra
>



-- 
Pratyush Chandra