Multiple Input Schemas in AvroStorage() fails

["Mangtani, Kushal" <Ku...@viasat.com>]

Hi,

I'm one of the several Pig Developer/User community.I have a question regarding Avro1.6.1 and Pig0.11 compatibility. In ref to https://issues.apache.org/jira/browse/PIG-2579

Issue:
Data = LOAD 'test/avro-data' using org.apache.pig.piggybank.storage.avro.AvroStorage();
Store Data using PigStorage(',')

This simple PigScript fails with an AvroTypeException :No match for a field

Here,

*         avro-data folder contains multiple files each having a different avro schema. PS: The schemas are not arbitrary different rather they have some fields added and they can be unioned.

*         PigVersion is pig-0.11.0-cdh4.4.0.jar

*         Piggybank is compiled using pig-0.11.0-cdh4.4.0

PS:I even tried the avro_test_jar attached with the Jira Ticket PIG-2579; even it fails when you have a folder with multiple schemas.

Looking forward in hearing back from you.


Regards
Kushal Mangtani
Software Engineer
VIasat Inc

Re: Multiple Input Schemas in AvroStorage() fails

[Ruslan Al-Fakikh <me...@gmail.com>]

I guess you need to specify 'multiple_schema' in AvroStorage


On Thu, Nov 28, 2013 at 4:07 AM, Mangtani, Kushal <
Kushal.Mangtani@viasat.com> wrote:

> Hi,
>
> I'm one of the several Pig Developer/User community.I have a question
> regarding Avro1.6.1 and Pig0.11 compatibility. In ref to
> https://issues.apache.org/jira/browse/PIG-2579
>
> Issue:
> Data = LOAD 'test/avro-data' using
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> Store Data using PigStorage(',')
>
> This simple PigScript fails with an AvroTypeException :No match for a field
>
> Here,
>
> *         avro-data folder contains multiple files each having a different
> avro schema. PS: The schemas are not arbitrary different rather they have
> some fields added and they can be unioned.
>
> *         PigVersion is pig-0.11.0-cdh4.4.0.jar
>
> *         Piggybank is compiled using pig-0.11.0-cdh4.4.0
>
> PS:I even tried the avro_test_jar attached with the Jira Ticket PIG-2579;
> even it fails when you have a folder with multiple schemas.
>
> Looking forward in hearing back from you.
>
>
> Regards
> Kushal Mangtani
> Software Engineer
> VIasat Inc
>
>