You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Justin SB (JIRA)" <ji...@apache.org> on 2009/11/20 07:29:39 UTC
[jira] Created: (AVRO-228) Refactor to separate concerns:
(de)serialization vs marshalling to data model
Refactor to separate concerns: (de)serialization vs marshalling to data model
-----------------------------------------------------------------------------
Key: AVRO-228
URL: https://issues.apache.org/jira/browse/AVRO-228
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.3.0
Reporter: Justin SB
I've attached a patch that separates out the code that deals with marshalling to java data models (Generic, Specific, Reflect) from the code that deals with (de)serializtion (DatumReader, DatumWriter). This means that e.g GenericDatumReader, SpecificDatumReader and ReflectDatumReader are unnecessary. Instead, a single AvroDatumReader is parameterized with a ClassMapping. The class mapping interface is currently implemented by the existing GenericData, SpecificData, ReflectData mappers.
The patch is large, so might be quite hard to follow. Essentially I renamed GenericDatumReader to AvroDatumReader, and moved any model-specific functionality to the ClassMapping interface. I did the same for DatumWriter & Requestor & Responder.
There remains some oddities for the ProxyingResponder / Requestor, but I wanted to keep the patch as a straight re-organization without significant code changes.
I believe this patch will set us up for some refactors down the road - ClassMapping is really 3 interfaces in one - the mapping of Java types to Avro types, some utility functions (like hashCode), and field accessors. Splitting this interface into 3 interfaces would allow reuse of the parts separately. Then it would be easy to use avro with arbitrary data models (by implementing the accessor functions) without needing to implement DatumWriters etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-228) Refactor to separate concerns:
(de)serialization vs marshalling to data model
Posted by "Justin SB (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Justin SB updated AVRO-228:
---------------------------
Attachment: 0001-Separated-serialization-code-from-marshalling-code.patch
Patch
> Refactor to separate concerns: (de)serialization vs marshalling to data model
> -----------------------------------------------------------------------------
>
> Key: AVRO-228
> URL: https://issues.apache.org/jira/browse/AVRO-228
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.3.0
> Reporter: Justin SB
> Attachments: 0001-Separated-serialization-code-from-marshalling-code.patch
>
>
> I've attached a patch that separates out the code that deals with marshalling to java data models (Generic, Specific, Reflect) from the code that deals with (de)serializtion (DatumReader, DatumWriter). This means that e.g GenericDatumReader, SpecificDatumReader and ReflectDatumReader are unnecessary. Instead, a single AvroDatumReader is parameterized with a ClassMapping. The class mapping interface is currently implemented by the existing GenericData, SpecificData, ReflectData mappers.
> The patch is large, so might be quite hard to follow. Essentially I renamed GenericDatumReader to AvroDatumReader, and moved any model-specific functionality to the ClassMapping interface. I did the same for DatumWriter & Requestor & Responder.
> There remains some oddities for the ProxyingResponder / Requestor, but I wanted to keep the patch as a straight re-organization without significant code changes.
> I believe this patch will set us up for some refactors down the road - ClassMapping is really 3 interfaces in one - the mapping of Java types to Avro types, some utility functions (like hashCode), and field accessors. Splitting this interface into 3 interfaces would allow reuse of the parts separately. Then it would be easy to use avro with arbitrary data models (by implementing the accessor functions) without needing to implement DatumWriters etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-228) Refactor to separate concerns:
(de)serialization vs marshalling to data model
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786997#action_12786997 ]
Doug Cutting commented on AVRO-228:
-----------------------------------
I like this approach. Instead of having three classes per in-memory representation, FooDatumReader, FooDatumWriter and FooData, we have just a single class for each. Unfortunately this is a big patch that's already out of date, mostly due to other changes I've made. Sorry! I'll take a closer look at this soon and give you more detailed feedback, so that, if you can then update it, we can hopefully commit it quickly, before it gets out of date again. Does that sound reasonable?
> Refactor to separate concerns: (de)serialization vs marshalling to data model
> -----------------------------------------------------------------------------
>
> Key: AVRO-228
> URL: https://issues.apache.org/jira/browse/AVRO-228
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.3.0
> Reporter: Justin SB
> Attachments: 0001-Separated-serialization-code-from-marshalling-code.patch
>
>
> I've attached a patch that separates out the code that deals with marshalling to java data models (Generic, Specific, Reflect) from the code that deals with (de)serializtion (DatumReader, DatumWriter). This means that e.g GenericDatumReader, SpecificDatumReader and ReflectDatumReader are unnecessary. Instead, a single AvroDatumReader is parameterized with a ClassMapping. The class mapping interface is currently implemented by the existing GenericData, SpecificData, ReflectData mappers.
> The patch is large, so might be quite hard to follow. Essentially I renamed GenericDatumReader to AvroDatumReader, and moved any model-specific functionality to the ClassMapping interface. I did the same for DatumWriter & Requestor & Responder.
> There remains some oddities for the ProxyingResponder / Requestor, but I wanted to keep the patch as a straight re-organization without significant code changes.
> I believe this patch will set us up for some refactors down the road - ClassMapping is really 3 interfaces in one - the mapping of Java types to Avro types, some utility functions (like hashCode), and field accessors. Splitting this interface into 3 interfaces would allow reuse of the parts separately. Then it would be easy to use avro with arbitrary data models (by implementing the accessor functions) without needing to implement DatumWriters etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.