Serialization/deserialization questions

[Emmanuel Lecharny <el...@gmail.com>]

Hi guys,

I'm reworking all the serializations for the base objects we are 
manipulating. I have a few questions and I'd like to get you opinion.

The base idea is to get rid of all the Externalizable/Serializable 
interfaces, and to provide our own helper classes to serialize the 
objects we have to write down the disk. The rationnal is that we avoid 
the cost of writing the useless java class informations, keeping the 
written objects as small as possible, and the process as fast as possible.

Q1 : Do we have to store a version number into the written data ?
Q2 : I'm going to inject the SchemaManager in all the deserialized data 
(it can be null, if the SchemaManager has not been created). Do you 
think it's a problem ?
Q3 : In order to be able to create the object as it was when it was 
serialized, I'll create package visible only constructors, with all the 
needed parameters. Does it sounds the right approach ?

Thanks !

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Serialization/deserialization questions

[Kiran Ayyagari <ka...@apache.org>]

On Fri, Feb 25, 2011 at 5:52 PM, Emmanuel Lecharny <el...@gmail.com> wrote:
> Hi guys,
>
> I'm reworking all the serializations for the base objects we are
> manipulating. I have a few questions and I'd like to get you opinion.
>
> The base idea is to get rid of all the Externalizable/Serializable
> interfaces, and to provide our own helper classes to serialize the objects
> we have to write down the disk. The rationnal is that we avoid the cost of
> writing the useless java class informations, keeping the written objects as
> small as possible, and the process as fast as possible.
>
> Q1 : Do we have to store a version number into the written data ?
I would say we 'Should' , cause I have seen enough cases where server
fails to start just because a class definition was
slightly changed which is unrelated to the storage format
> Q2 : I'm going to inject the SchemaManager in all the deserialized data (it
> can be null, if the SchemaManager has not been created). Do you think it's a
> problem ?
IMO it is a unnecessary field, we better set the SchemaManager in the
de/serializer class
> Q3 : In order to be able to create the object as it was when it was
> serialized, I'll create package visible only constructors, with all the
> needed parameters. Does it sounds the right approach ?
+1 perfectly valid, construction cost will cheaper in few cases (e.x DN, RDN)
>
> Thanks !
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>

thanks Emmanuel

-- 
Kiran Ayyagari

Re: Serialization/deserialization questions

[Emmanuel Lécharny <el...@apache.org>]

On 2/25/11 2:57 PM, Alex Karasulu wrote:
> On Fri, Feb 25, 2011 at 2:22 PM, Emmanuel Lecharny<el...@gmail.com>  wrote:
>> Hi guys,
>>
>> I'm reworking all the serializations for the base objects we are
>> manipulating. I have a few questions and I'd like to get you opinion.
>>
>> The base idea is to get rid of all the Externalizable/Serializable
>> interfaces, and to provide our own helper classes to serialize the objects
>> we have to write down the disk. The rationnal is that we avoid the cost of
>> writing the useless java class informations, keeping the written objects as
>> small as possible, and the process as fast as possible.
> Could you provide some of the problems that you encountered due to the
> use of the Externalizable/Serializable intefaces in some of these
> classes?

The main issue is the injection of the SchemaManager when we create the 
base object. readExternal() does not take any parameter, when using a 
helper class to do the same thing allows you to do so.
>> Q1 : Do we have to store a version number into the written data ?
> +1
>
>> Q2 : I'm going to inject the SchemaManager in all the deserialized data (it
>> can be null, if the SchemaManager has not been created). Do you think it's a
>> problem ?
> We can have separate serialization services that have a handle on the
> SchemaManager. These services can use schema information to handle
> their task of [de]serializing. However there are some entities that
> need access to schema information in order to conduct some operations
> like Entry.add( attribute ). How we handle access to schema
> information while they perform schema checks is another matter which
> we need to discuss.
>
> Perhaps a schema aware Entry wrapper can access this information
> leaving the simple Entry free to do as it likes while it intercepts
> calls and conducts schema live checks before delegating the call.
> Things like this can also be handled automatically with byte code
> manipulation. We can discuss this separately which would be involved.
What we need here is mainly to store the schemaManager, not to process 
the read data against the schemamanager. Doing so is a costly operation, 
and if we can avoid it, this is a net gain.

Also keep in mind that the serializations are mainly used inside the 
server, not anywhere else.
>> Q3 : In order to be able to create the object as it was when it was
>> serialized, I'll create package visible only constructors, with all the
>> needed parameters. Does it sounds the right approach ?
> This will require the [de]-serializer to be in the same package. That
> could present an issue with split packages if we want to make these
> entities into services down the line. I'd go ahead and make the
> constructors public for now.  Off the top of my head I cannot think of
> a detriment to exposing these constructors but perhaps you have some
> in mind?
Well, making those constructors public make the user think he can use 
them, when it's just intended to be used by the deserialization process. 
I don't think, OTOH, that having the deserializers along the class it 
works on is a big problem.

One other option would be to use a static method in the class to do the 
serialization/deserialization, and make the constructors private.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: Serialization/deserialization questions

[Alex Karasulu <ak...@apache.org>]

On Fri, Feb 25, 2011 at 2:22 PM, Emmanuel Lecharny <el...@gmail.com> wrote:
> Hi guys,
>
> I'm reworking all the serializations for the base objects we are
> manipulating. I have a few questions and I'd like to get you opinion.
>
> The base idea is to get rid of all the Externalizable/Serializable
> interfaces, and to provide our own helper classes to serialize the objects
> we have to write down the disk. The rationnal is that we avoid the cost of
> writing the useless java class informations, keeping the written objects as
> small as possible, and the process as fast as possible.

Could you provide some of the problems that you encountered due to the
use of the Externalizable/Serializable intefaces in some of these
classes?

> Q1 : Do we have to store a version number into the written data ?

+1

> Q2 : I'm going to inject the SchemaManager in all the deserialized data (it
> can be null, if the SchemaManager has not been created). Do you think it's a
> problem ?

We can have separate serialization services that have a handle on the
SchemaManager. These services can use schema information to handle
their task of [de]serializing. However there are some entities that
need access to schema information in order to conduct some operations
like Entry.add( attribute ). How we handle access to schema
information while they perform schema checks is another matter which
we need to discuss.

Perhaps a schema aware Entry wrapper can access this information
leaving the simple Entry free to do as it likes while it intercepts
calls and conducts schema live checks before delegating the call.
Things like this can also be handled automatically with byte code
manipulation. We can discuss this separately which would be involved.

> Q3 : In order to be able to create the object as it was when it was
> serialized, I'll create package visible only constructors, with all the
> needed parameters. Does it sounds the right approach ?

This will require the [de]-serializer to be in the same package. That
could present an issue with split packages if we want to make these
entities into services down the line. I'd go ahead and make the
constructors public for now.  Off the top of my head I cannot think of
a detriment to exposing these constructors but perhaps you have some
in mind?

Regards,
Alex