You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Josh Spiegel (JIRA)" <ji...@apache.org> on 2013/04/01 19:33:15 UTC

[jira] [Commented] (AVRO-1262) Provide access to the writer schema from the mapper

    [ https://issues.apache.org/jira/browse/AVRO-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618951#comment-13618951 ] 

Josh Spiegel commented on AVRO-1262:
------------------------------------

Harsh - thanks for confirming!  The way I patched it locally was to expose the schema on AvroKey.  I think your configuration suggestion would work too but it requires serializing the schema and reparsing it in the mapper.  My solution does not require a reparse but it is not perfect either because it might imply to a user that the schema can change per datum (which of course is not true).  

In any case, I am optimistic that Doug will know what to do :)

Thanks,
Josh
                
> Provide access to the writer schema from the mapper
> ---------------------------------------------------
>
>                 Key: AVRO-1262
>                 URL: https://issues.apache.org/jira/browse/AVRO-1262
>             Project: Avro
>          Issue Type: New Feature
>          Components: java
>    Affects Versions: 1.7.4
>            Reporter: Josh Spiegel
>            Assignee: Doug Cutting
>             Fix For: 1.7.5
>
>         Attachments: AVRO-1262.patch
>
>
> When using an Avro InputFormat like AvroKeyInputFormat, the writer schema of the container file should be accessible from the mapper.  This is useful in cases where a reader schema is not specified.  
> A workaround is to use FileSplit#getPath() to access the container file and manually pull out the schema.  This workaround is not ideal because internally the writer schema has already been read (see AvroRecordReaderBase#createAvroFileReader(...)) - it is awkward and inefficient for the user to repeat this work.
> See also:
> http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCAOF3b61nFw4ztOo9Q5pHHtoUDFZ3sRrvEdRGbXGV_cscTqd5LA%40mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira