You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Stepan Mishura <st...@gmail.com> on 2006/05/02 08:08:49 UTC
[classlib] How to test serialization? (was: [classlib] Tests for serialization – which package, name?)
Mikhail,
You are right - I think I asked not proper question. The base issue is: how
we are going to test serialization and what support from testing framework
is required? So far we have two serialization frameworks. Roughly, the first
framework provides utility class with set of static methods and the second
one provides super class to be extended.
Let's understand what kind of serialization tests are required.
1) Compatibility: testing scenario - serialize a number of a class' objects
on RI to get a set of golden files, create test to verify that
deserialization of golden files produces expected objects on Harmony
implementation. I think that everybody agree that this scenario is required
for every serializable class.
2) Self testing: testing scenario is to verify that an object can be
serialized/deserialized smoothly on Harmony implementation. Again this
scenario is also required for every serializable class.
3) Error: testing scenario is to check that deserialization of corrupted
serial form results in IOException, for example, if a class has fields
startDate and endDate then class implementation should verify that startDate
< endDate after deserialization. This scenario is not required for every
class.
Are there any others testing scenarios?
I think that this is obvious that 1 and 2 scenarios need support from
testing framework. Let's review what support we currently have:
--- compatibility testing scenario:
If you use the first framework then you have explicitly add to each test for
serializable class a separate method and use utility methods to perform
testing, for example,
public void serializationCompatibility() {
SerializationTester.assertCompabilityEquals(new MyClass(), "
MyClassFile.ser");
}
In case of the second framework a test inherits method testGolden() that
does standart testing routine: for each object returned by getData() method
it finds corresponding golden file, deserializes it and compares with the
object.
--- self testing scenario:
Similar to compatibility scenario: using the first framework you have
explicitly add a separate testing method:
public void testSerialization() {
assertTrue(SerializationTester.assertEquals(new SomeSerializable()));
}
In case of the second framework a test inherits testSelf() does the
following testing routine: for each object returned by getData() method it
serializes/deserializes the object and compares with the initial object.
To summarize: for the first framework it is required to add explicitly at
least two testing methods to each test. The second framework uses
inheritance mechanism so two standart testing methods are inherited from
super class and you only have to implement getData() abstract method.
IMHO, as far as it is possible to unify serialization testing I think that
inheritance appoach makes sense - two standart testing methods, less
copied/pasted code that leads to more maintainable tests.
Thoughts?
Thanks,
Stepan.
On 4/28/06, Mikhail Loenko wrote:
>
> I suggest that we first decide how we will test serialization,
> otherwise we may make a low-level decision that will
> make some high-level designs impossible
>
> Thanks,
> Mikhail
>
> <SNIP>
>
--
Thanks,
Stepan Mishura
Intel Middleware Products Division
------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib] How to test serialization? (was: [classlib] Tests for serialization – which package, name?)
Posted by Stepan Mishura <st...@gmail.com>.
Anton, thank you for pointing out to awt/swing classes - I've looked through
a number of randomly selected classes and I've realized that many of them
haven't specified serial form.
I'll try to estimate percentage of such classes.
Thanks,
Stepan.
On 5/2/06, Anton Avtamonov wrote:
>
> Stepan.
> Let me answer you shortly: yes, my main concern is about an assetion step.
> Just because I saw too many classes without specified serialization
> form, without equals() and so on. Actually, I suppose that is the
> entire graphical framework (awt + Swing) which is definitely more then
> 5% :-). I didn't deep investigations of course, but suppose that the
> simple assertion approach would not work there. Just because it will
> require to compare specific fields rather than call assertEquals().
>
> About third scenario with IOException.
> If IOException is the only possible reaction for incorrect form then
> you are right: we don't need any specific checks.
>
> --
> Anton Avtamonov,
> Intel Middleware Products Division
>
> <SNIP>
>
--
Thanks,
Stepan Mishura
Intel Middleware Products Division
------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib] How to test serialization? (was: [classlib] Tests for serialization – which package, name?)
Posted by Anton Avtamonov <an...@gmail.com>.
Stepan.
Let me answer you shortly: yes, my main concern is about an assetion step.
Just because I saw too many classes without specified serialization
form, without equals() and so on. Actually, I suppose that is the
entire graphical framework (awt + Swing) which is definitely more then
5% :-). I didn't deep investigations of course, but suppose that the
simple assertion approach would not work there. Just because it will
require to compare specific fields rather than call assertEquals().
About third scenario with IOException.
If IOException is the only possible reaction for incorrect form then
you are right: we don't need any specific checks.
--
Anton Avtamonov,
Intel Middleware Products Division
On 5/2/06, Stepan Mishura <st...@gmail.com> wrote:
> On 5/2/06, Anton Avtamonov wrote:
> >
> > On 5/2/06, Stepan Mishura wrote:
> > > Mikhail,
> > >
> > > You are right - I think I asked not proper question. The base issue is:
> > how
> > > we are going to test serialization and what support from testing
> > framework
> > > is required? So far we have two serialization frameworks. Roughly, the
> > first
> > > framework provides utility class with set of static methods and the
> > second
> > > one provides super class to be extended.
> > >
> > > Let's understand what kind of serialization tests are required.
> > > 1) Compatibility: testing scenario - serialize a number of a class'
> > objects
> > > on RI to get a set of golden files, create test to verify that
> > > deserialization of golden files produces expected objects on Harmony
> > > implementation. I think that everybody agree that this scenario is
> > required
> > > for every serializable class.
> >
> > Agree. The only exception is those cases when serialization form is
> > not documented and cannot be implemented as in RI.
> >
> > >
> > > 2) Self testing: testing scenario is to verify that an object can be
> > > serialized/deserialized smoothly on Harmony implementation. Again this
> > > scenario is also required for every serializable class.
> >
> > Agree.
> >
> > >
> > > 3) Error: testing scenario is to check that deserialization of corrupted
> > > serial form results in IOException, for example, if a class has fields
> > > startDate and endDate then class implementation should verify that
> > startDate
> > > < endDate after deserialization. This scenario is not required for every
> > > class.
> >
> > Yes. This requires both corrupted ser forms and assertion customization.
>
>
> Sorry, I didn't catch - what assertion customization is required in case of
> IOException?
>
> >
> > > Are there any others testing scenarios?
> > >
> > > I think that this is obvious that 1 and 2 scenarios need support from
> > > testing framework. Let's review what support we currently have:
> > > --- compatibility testing scenario:
> > > If you use the first framework then you have explicitly add to each test
> > for
> > > serializable class a separate method and use utility methods to perform
> > > testing, for example,
> > > public void serializationCompatibility() {
> > > SerializationTester.assertCompabilityEquals(new MyClass(), "
> > > MyClassFile.ser");
> > > }
> > >
> > > In case of the second framework a test inherits method testGolden() that
> > > does standart testing routine: for each object returned by getData()
> > method
> > > it finds corresponding golden file, deserializes it and compares with
> > the
> > > object.
> >
> > You forgot to mention the comparison method which may have to be
> > overriden as well in case:
> > - RI serialzaition differ
>
>
> How many such cases? I expect that there should be no more then 3-5% of
> overall amount of serializable classes. Should we base serialization
> testing framework design on such rare cases? I think no. Also IMHO that such
> cases may be resolved by overriding testGolden() method and putting all
> specific to this method.
>
> - you need to check the state of transient fields
>
>
> Most of them are private, right? So you will check them via public API?
>
> - equals() method is not defined for the type or not covers all the
> > fields. In this case you have to provide 'manual' comparison (very
> > often case, IMHO)
>
>
> IMHO this is the right approach to have separate comparision method that is
> differ from other assert methods to compare object in case of
> serialization/deserialization testing. So if you have to perform 'manual'
> comparison in any case then it would be better to do it in separate
> and standartized assert method. IMHO, assertDeserialized is a proper name
> for such method. Also this is very often case for Exception classes so we
> can generalize this case in super class too.
>
>
>
> > >
> > > --- self testing scenario:
> > > Similar to compatibility scenario: using the first framework you have
> > > explicitly add a separate testing method:
> > > public void testSerialization() {
> > > assertTrue(SerializationTester.assertEquals(new SomeSerializable()));
> > > }
> > >
> > > In case of the second framework a test inherits testSelf() does the
> > > following testing routine: for each object returned by getData() method
> > it
> > > serializes/deserializes the object and compares with the initial object.
> >
> > Again, you have to provide comparison customization. Besides, it may
> > differ from comparison for the golden form. So, to cover everything
> > you have to define two comparison methods in your framework: for
> > golden comparison and for self-comparison.
>
>
> Is this theoretical thought or you have concrete example?
>
> >
> > > To summarize: for the first framework it is required to add explicitly
> > at
> > > least two testing methods to each test.
> >
> > Yes. And allows to do all the particular asserions (fields comparison)
> > right in the test. At the other hand, if the comparison is 'default',
> > you just need assertEquals().
> >
> > The second framework uses
> > > inheritance mechanism so two standart testing methods are inherited from
> > > super class and you only have to implement getData() abstract method.
> >
> > This is really convenient for the simple cases when you don't have to
> > customize the assertion step.
> >
> > >
> > > IMHO, as far as it is possible to unify serialization testing I think
> > that
> > > inheritance appoach makes sense - two standart testing methods, less
> > > copied/pasted code that leads to more maintainable tests.
> >
> > As I said before, second approach can result in less code since some
> > TestCases will just override one-two methods. However I disagree about
> > maintanability - the inheritance approach is much less flexible (see
> > above).
>
>
> So the main disadvantage for you is assertion step, right?
>
> --
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
> ------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib] How to test serialization? (was: [classlib] Tests for serialization – which package, name?)
Posted by Stepan Mishura <st...@gmail.com>.
On 5/2/06, Anton Avtamonov wrote:
>
> On 5/2/06, Stepan Mishura wrote:
> > Mikhail,
> >
> > You are right - I think I asked not proper question. The base issue is:
> how
> > we are going to test serialization and what support from testing
> framework
> > is required? So far we have two serialization frameworks. Roughly, the
> first
> > framework provides utility class with set of static methods and the
> second
> > one provides super class to be extended.
> >
> > Let's understand what kind of serialization tests are required.
> > 1) Compatibility: testing scenario - serialize a number of a class'
> objects
> > on RI to get a set of golden files, create test to verify that
> > deserialization of golden files produces expected objects on Harmony
> > implementation. I think that everybody agree that this scenario is
> required
> > for every serializable class.
>
> Agree. The only exception is those cases when serialization form is
> not documented and cannot be implemented as in RI.
>
> >
> > 2) Self testing: testing scenario is to verify that an object can be
> > serialized/deserialized smoothly on Harmony implementation. Again this
> > scenario is also required for every serializable class.
>
> Agree.
>
> >
> > 3) Error: testing scenario is to check that deserialization of corrupted
> > serial form results in IOException, for example, if a class has fields
> > startDate and endDate then class implementation should verify that
> startDate
> > < endDate after deserialization. This scenario is not required for every
> > class.
>
> Yes. This requires both corrupted ser forms and assertion customization.
Sorry, I didn't catch - what assertion customization is required in case of
IOException?
>
> > Are there any others testing scenarios?
> >
> > I think that this is obvious that 1 and 2 scenarios need support from
> > testing framework. Let's review what support we currently have:
> > --- compatibility testing scenario:
> > If you use the first framework then you have explicitly add to each test
> for
> > serializable class a separate method and use utility methods to perform
> > testing, for example,
> > public void serializationCompatibility() {
> > SerializationTester.assertCompabilityEquals(new MyClass(), "
> > MyClassFile.ser");
> > }
> >
> > In case of the second framework a test inherits method testGolden() that
> > does standart testing routine: for each object returned by getData()
> method
> > it finds corresponding golden file, deserializes it and compares with
> the
> > object.
>
> You forgot to mention the comparison method which may have to be
> overriden as well in case:
> - RI serialzaition differ
How many such cases? I expect that there should be no more then 3-5% of
overall amount of serializable classes. Should we base serialization
testing framework design on such rare cases? I think no. Also IMHO that such
cases may be resolved by overriding testGolden() method and putting all
specific to this method.
- you need to check the state of transient fields
Most of them are private, right? So you will check them via public API?
- equals() method is not defined for the type or not covers all the
> fields. In this case you have to provide 'manual' comparison (very
> often case, IMHO)
IMHO this is the right approach to have separate comparision method that is
differ from other assert methods to compare object in case of
serialization/deserialization testing. So if you have to perform 'manual'
comparison in any case then it would be better to do it in separate
and standartized assert method. IMHO, assertDeserialized is a proper name
for such method. Also this is very often case for Exception classes so we
can generalize this case in super class too.
> >
> > --- self testing scenario:
> > Similar to compatibility scenario: using the first framework you have
> > explicitly add a separate testing method:
> > public void testSerialization() {
> > assertTrue(SerializationTester.assertEquals(new SomeSerializable()));
> > }
> >
> > In case of the second framework a test inherits testSelf() does the
> > following testing routine: for each object returned by getData() method
> it
> > serializes/deserializes the object and compares with the initial object.
>
> Again, you have to provide comparison customization. Besides, it may
> differ from comparison for the golden form. So, to cover everything
> you have to define two comparison methods in your framework: for
> golden comparison and for self-comparison.
Is this theoretical thought or you have concrete example?
>
> > To summarize: for the first framework it is required to add explicitly
> at
> > least two testing methods to each test.
>
> Yes. And allows to do all the particular asserions (fields comparison)
> right in the test. At the other hand, if the comparison is 'default',
> you just need assertEquals().
>
> The second framework uses
> > inheritance mechanism so two standart testing methods are inherited from
> > super class and you only have to implement getData() abstract method.
>
> This is really convenient for the simple cases when you don't have to
> customize the assertion step.
>
> >
> > IMHO, as far as it is possible to unify serialization testing I think
> that
> > inheritance appoach makes sense - two standart testing methods, less
> > copied/pasted code that leads to more maintainable tests.
>
> As I said before, second approach can result in less code since some
> TestCases will just override one-two methods. However I disagree about
> maintanability - the inheritance approach is much less flexible (see
> above).
So the main disadvantage for you is assertion step, right?
--
Thanks,
Stepan Mishura
Intel Middleware Products Division
------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [classlib] How to test serialization? (was: [classlib] Tests for serialization – which package, name?)
Posted by Anton Avtamonov <an...@gmail.com>.
On 5/2/06, Stepan Mishura <st...@gmail.com> wrote:
> Mikhail,
>
> You are right - I think I asked not proper question. The base issue is: how
> we are going to test serialization and what support from testing framework
> is required? So far we have two serialization frameworks. Roughly, the first
> framework provides utility class with set of static methods and the second
> one provides super class to be extended.
>
> Let's understand what kind of serialization tests are required.
> 1) Compatibility: testing scenario - serialize a number of a class' objects
> on RI to get a set of golden files, create test to verify that
> deserialization of golden files produces expected objects on Harmony
> implementation. I think that everybody agree that this scenario is required
> for every serializable class.
Agree. The only exception is those cases when serialization form is
not documented and cannot be implemented as in RI.
>
> 2) Self testing: testing scenario is to verify that an object can be
> serialized/deserialized smoothly on Harmony implementation. Again this
> scenario is also required for every serializable class.
Agree.
>
> 3) Error: testing scenario is to check that deserialization of corrupted
> serial form results in IOException, for example, if a class has fields
> startDate and endDate then class implementation should verify that startDate
> < endDate after deserialization. This scenario is not required for every
> class.
Yes. This requires both corrupted ser forms and assertion customization.
>
> Are there any others testing scenarios?
>
> I think that this is obvious that 1 and 2 scenarios need support from
> testing framework. Let's review what support we currently have:
> --- compatibility testing scenario:
> If you use the first framework then you have explicitly add to each test for
> serializable class a separate method and use utility methods to perform
> testing, for example,
> public void serializationCompatibility() {
> SerializationTester.assertCompabilityEquals(new MyClass(), "
> MyClassFile.ser");
> }
>
> In case of the second framework a test inherits method testGolden() that
> does standart testing routine: for each object returned by getData() method
> it finds corresponding golden file, deserializes it and compares with the
> object.
You forgot to mention the comparison method which may have to be
overriden as well in case:
- RI serialzaition differ
- you need to check the state of transient fields
- equals() method is not defined for the type or not covers all the
fields. In this case you have to provide 'manual' comparison (very
often case, IMHO)
>
> --- self testing scenario:
> Similar to compatibility scenario: using the first framework you have
> explicitly add a separate testing method:
> public void testSerialization() {
> assertTrue(SerializationTester.assertEquals(new SomeSerializable()));
> }
>
> In case of the second framework a test inherits testSelf() does the
> following testing routine: for each object returned by getData() method it
> serializes/deserializes the object and compares with the initial object.
Again, you have to provide comparison customization. Besides, it may
differ from comparison for the golden form. So, to cover everything
you have to define two comparison methods in your framework: for
golden comparison and for self-comparison.
>
> To summarize: for the first framework it is required to add explicitly at
> least two testing methods to each test.
Yes. And allows to do all the particular asserions (fields comparison)
right in the test. At the other hand, if the comparison is 'default',
you just need assertEquals().
The second framework uses
> inheritance mechanism so two standart testing methods are inherited from
> super class and you only have to implement getData() abstract method.
This is really convenient for the simple cases when you don't have to
customize the assertion step.
>
> IMHO, as far as it is possible to unify serialization testing I think that
> inheritance appoach makes sense - two standart testing methods, less
> copied/pasted code that leads to more maintainable tests.
As I said before, second approach can result in less code since some
TestCases will just override one-two methods. However I disagree about
maintanability - the inheritance approach is much less flexible (see
above).
--
Anton Avtamonov,
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org