You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Viraj Bhat (JIRA)" <ji...@apache.org> on 2013/05/20 23:51:15 UTC
[jira] [Updated] (PIG-3331) Default values not stored in avro file
when using specific schemas during store in AvroStorage
[ https://issues.apache.org/jira/browse/PIG-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Viraj Bhat updated PIG-3331:
----------------------------
Description:
Script which stores Avro using a predefined schema does not store the default values in the file
{code}
a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
float,doublenum: double);
b2 = foreach a generate id, intnum5, intnum100;
c2 = filter b2 by 110 <= id and id < 120;
STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
{ "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
"type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
"default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
');
{code}
Opening the file shows the following schema
{noformat}
avro.schema
{"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
{noformat}
There seems to be a problem storing the schema.
Viraj
was:
Script which stores Avro using a predefined schema does not store the default values in the file
{code}
a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
float,doublenum: double);
b2 = foreach a generate id, intnum5, intnum100;
c2 = filter b2 by 110 <= id and id < 120;
STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
{ "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
"type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
"default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
');
{code}
Opening the file shows the following schema
{quote}
avro.schema
{"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
{quote}
There seems to be a problem storing the schema.
Viraj
> Default values not stored in avro file when using specific schemas during store in AvroStorage
> ----------------------------------------------------------------------------------------------
>
> Key: PIG-3331
> URL: https://issues.apache.org/jira/browse/PIG-3331
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Affects Versions: 0.11.1
> Reporter: Viraj Bhat
> Assignee: Viraj Bhat
> Fix For: 0.11.2
>
>
> Script which stores Avro using a predefined schema does not store the default values in the file
> {code}
> a = LOAD 'numbers.txt' USING PigStorage (':') as (intnum1000: int,id:
> int,intnum5: int,intnum100: int,intnum: int,longnum: long,floatnum:
> float,doublenum: double);
> b2 = foreach a generate id, intnum5, intnum100;
> c2 = filter b2 by 110 <= id and id < 120;
> STORE c2 INTO '/tmp/TestAvroStorage/testDefaultValueWrite' USING
> org.apache.pig.piggybank.storage.avro.AvroStorage (' { "debug" : 5, "schema" :
> { "name" : "rmyrecord", "type" : "record", "fields" : [ { "name" : "id",
> "type" : "int" , "default" : 0 }, { "name" : "intnum5", "type" : "int",
> "default" : 0 }, { "name" : "intnum100", "type" : "int", "default" : 0 } ] } }
> ');
> {code}
> Opening the file shows the following schema
> {noformat}
> avro.schema
> {"type":"record","name":"rmyrecord","fields":[{"name":"id","type":"int"},{"name":"intnum5","type":"int"},{"name":"intnum100","type":"int"}]}
> {noformat}
> There seems to be a problem storing the schema.
> Viraj
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira