You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Bill Graham (JIRA)" <ji...@apache.org> on 2011/08/03 02:50:26 UTC
[jira] [Created] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
AvroStorage doesn't work with Avro 1.5.1
----------------------------------------
Key: PIG-2202
URL: https://issues.apache.org/jira/browse/PIG-2202
Project: Pig
Issue Type: Bug
Reporter: Bill Graham
See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
{noformat}
java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084344#comment-13084344 ]
Alan Gates commented on PIG-2202:
---------------------------------
I'll be reviewing this patch.
> AvroStorage doesn't work with Avro 1.5.1
> ----------------------------------------
>
> Key: PIG-2202
> URL: https://issues.apache.org/jira/browse/PIG-2202
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2202_1.patch, PIG-2202_2.patch
>
>
> See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
> {noformat}
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2202:
-----------------------------
Assignee: Bill Graham
Release Note: AvroStorage upgraded to work with Avro 1.5.1.
Status: Patch Available (was: Open)
> AvroStorage doesn't work with Avro 1.5.1
> ----------------------------------------
>
> Key: PIG-2202
> URL: https://issues.apache.org/jira/browse/PIG-2202
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2202_1.patch, PIG-2202_2.patch
>
>
> See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
> {noformat}
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2202:
-----------------------------
Attachment: PIG-2202_1.patch
Attached is patch 1, which allows you to compile against Avro 1.5.1. This patch still results in 2 unit test failures with exceptions as shown below.
Is it acceptable for {{AvroStorage}} to be upgraded to work with Avro 1.5.1 and not support earlier Avro releases?
{noformat}
11/08/02 17:40:35 WARN mapred.LocalJobRunner: job_local_0005
java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
at org.apache.avro.generic.GenericData.getField(GenericData.java:470)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:102)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:99)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:244)
at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.write(PigAvroRecordWriter.java:49)
at org.apache.pig.piggybank.storage.avro.AvroStorage.putNext(AvroStorage.java:580)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getNext(POStore.java:143)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.runPipeline(POSplit.java:254)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.processPlan(POSplit.java:236)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.getNext(POSplit.java:228)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:456)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:424)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:404)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:258)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
{noformat}
> AvroStorage doesn't work with Avro 1.5.1
> ----------------------------------------
>
> Key: PIG-2202
> URL: https://issues.apache.org/jira/browse/PIG-2202
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: PIG-2202_1.patch
>
>
> See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
> {noformat}
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2202:
-----------------------------
Attachment: PIG-2202_2.patch
Attached is patch 2 that upgrades {{PigAvroDatumWriter}} to work with changes to it's Avro superclass, {{GenericDatumWriter}}.
This patch is backward compatible with Avro 1.4.1.
> AvroStorage doesn't work with Avro 1.5.1
> ----------------------------------------
>
> Key: PIG-2202
> URL: https://issues.apache.org/jira/browse/PIG-2202
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: PIG-2202_1.patch, PIG-2202_2.patch
>
>
> See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
> {noformat}
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2202) AvroStorage doesn't work with Avro
1.5.1
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2202:
----------------------------
Resolution: Fixed
Fix Version/s: 0.10
Status: Resolved (was: Patch Available)
Patch checked in. Thanks Bill.
> AvroStorage doesn't work with Avro 1.5.1
> ----------------------------------------
>
> Key: PIG-2202
> URL: https://issues.apache.org/jira/browse/PIG-2202
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.10
>
> Attachments: PIG-2202_1.patch, PIG-2202_2.patch
>
>
> See PIG-2195. Pig jobs that use AvroStorage to write records using the 'schema' or 'data' options fail when using Avro 1.5.1 with the following exception:
> {noformat}
> java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira