You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Josh Spiegel (JIRA)" <ji...@apache.org> on 2013/04/01 19:33:15 UTC
[jira] [Commented] (AVRO-1262) Provide access to the writer schema
from the mapper
[ https://issues.apache.org/jira/browse/AVRO-1262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618951#comment-13618951 ]
Josh Spiegel commented on AVRO-1262:
------------------------------------
Harsh - thanks for confirming! The way I patched it locally was to expose the schema on AvroKey. I think your configuration suggestion would work too but it requires serializing the schema and reparsing it in the mapper. My solution does not require a reparse but it is not perfect either because it might imply to a user that the schema can change per datum (which of course is not true).
In any case, I am optimistic that Doug will know what to do :)
Thanks,
Josh
> Provide access to the writer schema from the mapper
> ---------------------------------------------------
>
> Key: AVRO-1262
> URL: https://issues.apache.org/jira/browse/AVRO-1262
> Project: Avro
> Issue Type: New Feature
> Components: java
> Affects Versions: 1.7.4
> Reporter: Josh Spiegel
> Assignee: Doug Cutting
> Fix For: 1.7.5
>
> Attachments: AVRO-1262.patch
>
>
> When using an Avro InputFormat like AvroKeyInputFormat, the writer schema of the container file should be accessible from the mapper. This is useful in cases where a reader schema is not specified.
> A workaround is to use FileSplit#getPath() to access the container file and manually pull out the schema. This workaround is not ideal because internally the writer schema has already been read (see AvroRecordReaderBase#createAvroFileReader(...)) - it is awkward and inefficient for the user to repeat this work.
> See also:
> http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCAOF3b61nFw4ztOo9Q5pHHtoUDFZ3sRrvEdRGbXGV_cscTqd5LA%40mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira