You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Michael Armbrust (JIRA)" <ji...@apache.org> on 2011/08/10 04:57:27 UTC
[jira] [Created] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
SpecificDatumReader should allow users to specify the ClassLoader used.
-----------------------------------------------------------------------
Key: AVRO-873
URL: https://issues.apache.org/jira/browse/AVRO-873
Project: Avro
Issue Type: Improvement
Components: java
Reporter: Michael Armbrust
Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
One relatively simple way to fix this limitations involves:
* Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
* Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
* Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Michael Armbrust (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated AVRO-873:
----------------------------------
Attachment: (was: specificDataClassLoader.patch)
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
> Assignee: Michael Armbrust
> Attachments: specificDataClassLoader2.patch
>
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Michael Armbrust (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated AVRO-873:
----------------------------------
Attachment: specificDataClassLoader.patch
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
> Attachments: specificDataClassLoader.patch
>
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Michael Armbrust (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated AVRO-873:
----------------------------------
Attachment: specificDataClassLoader2.patch
Thanks for the feedback. Here is an update patch.
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
> Assignee: Michael Armbrust
> Attachments: specificDataClassLoader2.patch
>
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Michael Armbrust (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated AVRO-873:
----------------------------------
Status: Patch Available (was: Open)
Here is a patch the implements the improvement. Comments or suggestions appreciated.
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting reassigned AVRO-873:
---------------------------------
Assignee: Michael Armbrust
Looks like a useful addition.
Perhaps instead of adding a new field to SpecificDatumReader we can add a new method to SpecificData, since the base GenericData already has a field containing the desired value:
{code}
public SpecificData getSpecificData() { return (SpecificData)getData(); }
{code}
Then this can be used in SpecificDatumReader, as getSpecificData().getClass(...).
Also, I'd prefer if the classLoader field were defined nearer the top of the class, since it's used by both a constructor and the getClass() implementation. I'd place it just before the constructors.
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
> Assignee: Michael Armbrust
> Attachments: specificDataClassLoader.patch
>
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-873) SpecificDatumReader should allow users
to specify the ClassLoader used.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-873:
------------------------------
Resolution: Fixed
Fix Version/s: 1.6.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I committed this. Thanks, Michael.
I made a few minor changes, javadoc mostly...
> SpecificDatumReader should allow users to specify the ClassLoader used.
> -----------------------------------------------------------------------
>
> Key: AVRO-873
> URL: https://issues.apache.org/jira/browse/AVRO-873
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Michael Armbrust
> Assignee: Michael Armbrust
> Fix For: 1.6.0
>
> Attachments: specificDataClassLoader2.patch
>
>
> Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.
> One relatively simple way to fix this limitations involves:
> * Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
> * Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
> * Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira