[jira] [Commented] (PIG-3621) Python Avro library can't read Avros made with builtin AvroStorage

["Russell Jurney (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852191#comment-13852191 ] 

Russell Jurney commented on PIG-3621:
-------------------------------------

I discovered the bug. Line 106 of AvroStorage.java uses the word "record" as the schema name. This is a reserved word in the Python implementation. Changing this name makes records from AvroStorage read fine by Python Avro.

Patch coming.

> Python Avro library can't read Avros made with builtin AvroStorage
> ------------------------------------------------------------------
>
>                 Key: PIG-3621
>                 URL: https://issues.apache.org/jira/browse/PIG-3621
>             Project: Pig
>          Issue Type: Bug
>          Components: internal-udfs
>    Affects Versions: 0.12.0
>            Reporter: Russell Jurney
>            Assignee: Joseph Adler
>
> Using this script:
> from avro import schema, datafile, io
> import pprint
> import sys
> import json
> field_id = None
> # Optional key to print
> if (len(sys.argv) > 2):
>   field_id = sys.argv[2]
> # Test reading avros
> rec_reader = io.DatumReader()
> # Create a 'data file' (avro file) reader
> df_reader = datafile.DataFileReader(
>   open(sys.argv[1]),
>   rec_reader
> )
> the last line fails with:
> Traceback (most recent call last):
>   File "/Users/rjurney/bin/cat_avro", line 22, in <module>
>     rec_reader
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/datafile.py", line 247, in __init__
>     self.datum_reader.writers_schema = schema.parse(self.get_meta(SCHEMA_KEY))
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/schema.py", line 784, in parse
>     return make_avsc_object(json_data, names)
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/schema.py", line 740, in make_avsc_object
>     return RecordSchema(name, namespace, fields, names, type, doc, other_props)
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/schema.py", line 653, in __init__
>     other_props)
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/schema.py", line 294, in __init__
>     new_name = names.add_name(name, namespace, self)
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/avro/schema.py", line 268, in add_name
>     raise SchemaParseException(fail_msg)
> avro.schema.SchemaParseException: record is a reserved type name.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)