You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by anup ahire <ah...@gmail.com> on 2013/10/15 19:29:33 UTC

AvroStorage issue

Hello ,

I am trying to store data into avro using AvroStorage() with following
schema. I have pig 0.11.
{"type":"record","name":"TUPLE_0","fields":[{"name":"Header","type":["null","string"],"doc":"autogenerated
from Pig Field Schema"}]}

I am getting following errors when I run the job.


Caused by: java.lang.RuntimeException: Unsupported type in record:class
java.lang.String
    at
org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.getField(PigAvroDatumWriter.java:388)
    at
org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.writeRecord(PigAvroDatumWriter.java:362)
    at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
    at
org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:103)
    at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
    at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257)

Any idea whats going wrong ?

Thanks,
Anup

Re: AvroStorage issue

Posted by Bertrand Dechoux <de...@gmail.com>.
The "doc" field should be at the level of the record, not the field. Maybe
that's the issue even though the exception is not clear.

For the first version, you can let Pig generate the schema and then evolve
it.

Bertrand


On Tue, Oct 15, 2013 at 7:29 PM, anup ahire <ah...@gmail.com> wrote:

> Hello ,
>
> I am trying to store data into avro using AvroStorage() with following
> schema. I have pig 0.11.
>
> {"type":"record","name":"TUPLE_0","fields":[{"name":"Header","type":["null","string"],"doc":"autogenerated
> from Pig Field Schema"}]}
>
> I am getting following errors when I run the job.
>
>
> Caused by: java.lang.RuntimeException: Unsupported type in record:class
> java.lang.String
>     at
>
> org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.getField(PigAvroDatumWriter.java:388)
>     at
>
> org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.writeRecord(PigAvroDatumWriter.java:362)
>     at
>
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
>     at
>
> org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:103)
>     at
>
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
>     at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257)
>
> Any idea whats going wrong ?
>
> Thanks,
> Anup
>



-- 
Bertrand Dechoux