AvroDeepCopier with private constructors in avro objects

["Durfey,Stephen" <St...@Cerner.com>]

In the AvroDeepCopier class, the createNewInstance method uses targetClass.newInstance() to instantiate  a new instance of an Avro object when AvroSpecificDeepCopier is used. This works fine when the default constructor on the Avro object is public, but causes an IllegalAccessException to be thrown when the default constructor is private. This occurs whenever a join is performed between PCollections (when the IntermediateEmitter requires a detached value).

In my particular use case we wanted to force downstream consumers to use the Builders on our Avro objects.

In the Avros class, the specifics method uses the Hadoop provided ReflectionUtils, which forces the access modifier to public before object instantiation. Is this inconsistency a bug, or is this the desired behavior? If it is, I would be glad to log the JIRA and work on a patch to use ReflectionUtils inside AvroDeepCopier.

Stephen Durfey
Software Engineer|Cerner

CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.

Re: AvroDeepCopier with private constructors in avro objects

[Josh Wills <jo...@gmail.com>]

Hey Stephen,

It's a limitation that is tied to the fact that we go out of our way to
stay backwards compatible w/the older Avro versions (e.g., 1.5.4) that are
still used in Hadoop 2.x and that didn't have the Builder APIs. That said,
we definitely have places in the code where we do some magic to figure out
what the version of Avro we're using does/does not support and change
behavior accordingly, and I'd love a patch for it. This specific sort of
issue comes up for protocol buffers as well, and there's a check for it in
o.a.c.types.Protos that should come in handy for your patch.

Thanks!
Josh


On Wed, Mar 20, 2013 at 9:28 AM, Durfey,Stephen
<St...@cerner.com>wrote:

>  In the AvroDeepCopier class, the createNewInstance method uses
> targetClass.newInstance() to instantiate  a new instance of an Avro object
> when AvroSpecificDeepCopier is used. This works fine when the default
> constructor on the Avro object is public, but causes an
> IllegalAccessException to be thrown when the default constructor is
> private. This occurs whenever a join is performed between PCollections
> (when the IntermediateEmitter requires a detached value).
>
>  In my particular use case we wanted to force downstream consumers to use
> the Builders on our Avro objects.
>
>  In the Avros class, the specifics method uses the Hadoop provided
> ReflectionUtils, which forces the access modifier to public before object
> instantiation. Is this inconsistency a bug, or is this the desired
> behavior? If it is, I would be glad to log the JIRA and work on a patch to
> use ReflectionUtils inside AvroDeepCopier.
>
>  Stephen Durfey
> Software Engineer|Cerner
>  CONFIDENTIALITY NOTICE This message and any included attachments are
> from Cerner Corporation and are intended only for the addressee. The
> information contained in this message is confidential and may constitute
> inside or non-public information under international, federal, or state
> securities laws. Unauthorized forwarding, printing, copying, distribution,
> or use of such information is strictly prohibited and may be unlawful. If
> you are not the addressee, please promptly delete this message and notify
> the sender of the delivery error by e-mail or you may call Cerner's
> corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>