You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Isabel Drost (JIRA)" <ji...@apache.org> on 2011/07/24 22:25:10 UTC
[jira] [Created] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
NoSuchFieldException when emitting record with array of union of records in mapred
----------------------------------------------------------------------------------
Key: AVRO-864
URL: https://issues.apache.org/jira/browse/AVRO-864
Project: Avro
Issue Type: Bug
Reporter: Isabel Drost
I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-864:
------------------------------
Fix Version/s: (was: 1.6.0)
1.5.2
> NoSuchFieldException when emitting record with array of union of records in mapred
> ----------------------------------------------------------------------------------
>
> Key: AVRO-864
> URL: https://issues.apache.org/jira/browse/AVRO-864
> Project: Avro
> Issue Type: Bug
> Reporter: Isabel Drost
> Assignee: Isabel Drost
> Fix For: 1.5.2
>
> Attachments: AVRO-864.diff, AVRO-864.diff
>
>
> I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
> When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
Posted by "Isabel Drost (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Isabel Drost updated AVRO-864:
------------------------------
Attachment: AVRO-864.diff
Patch illustrating the issue - contains a unit test to reproduce the problem as well as two small changes that so far have fixed the problem for me.
Big Thanks+kiss to Thilo Fromm for helping me debug this.
> NoSuchFieldException when emitting record with array of union of records in mapred
> ----------------------------------------------------------------------------------
>
> Key: AVRO-864
> URL: https://issues.apache.org/jira/browse/AVRO-864
> Project: Avro
> Issue Type: Bug
> Reporter: Isabel Drost
> Attachments: AVRO-864.diff
>
>
> I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
> When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-864:
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I committed this. Thanks, Isabel!
> NoSuchFieldException when emitting record with array of union of records in mapred
> ----------------------------------------------------------------------------------
>
> Key: AVRO-864
> URL: https://issues.apache.org/jira/browse/AVRO-864
> Project: Avro
> Issue Type: Bug
> Reporter: Isabel Drost
> Assignee: Isabel Drost
> Fix For: 1.6.0
>
> Attachments: AVRO-864.diff, AVRO-864.diff
>
>
> I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
> When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-864:
------------------------------
Attachment: AVRO-864.diff
Here's a slightly different version of the patch. Instead of catching exceptions it calls super directly for generic and specific records. The provided unit test still passes.
> NoSuchFieldException when emitting record with array of union of records in mapred
> ----------------------------------------------------------------------------------
>
> Key: AVRO-864
> URL: https://issues.apache.org/jira/browse/AVRO-864
> Project: Avro
> Issue Type: Bug
> Reporter: Isabel Drost
> Fix For: 1.6.0
>
> Attachments: AVRO-864.diff, AVRO-864.diff
>
>
> I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
> When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-864) NoSuchFieldException when emitting
record with array of union of records in mapred
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-864:
------------------------------
Fix Version/s: 1.6.0
Assignee: Isabel Drost
Status: Patch Available (was: Open)
I will commit this soon unless there are objections.
> NoSuchFieldException when emitting record with array of union of records in mapred
> ----------------------------------------------------------------------------------
>
> Key: AVRO-864
> URL: https://issues.apache.org/jira/browse/AVRO-864
> Project: Avro
> Issue Type: Bug
> Reporter: Isabel Drost
> Assignee: Isabel Drost
> Fix For: 1.6.0
>
> Attachments: AVRO-864.diff, AVRO-864.diff
>
>
> I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.
> When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira