[OT] Re: Unit testing revisited

[Leo Simons <ma...@leosimons.com>]

On Wed, Mar 22, 2006 at 07:34:16AM -0500, Geir Magnusson Jr wrote:
> >Heh. You find *those* by running the app server tests :-). I suspect that
> >running the J2EE TCK against geronimo running on harmony and comparing it
> >with running the J2EE TCK against geronimo running on the sun jdk is
> >going to be pretty insightful...
> 
> Like a mortar attack is insightful. :)

LOL. There might be a building or two left standing, and we can give the
people that built those a standing ovation :-)

> It will be an interesting test of "The Algebra of TCK-ness"
> 
> If A = Sun JDK passes Java SE TCK
> If B(A) = Geronimo passes Java EE TCK on compliant Sun JDK
> If C = Harmony JDK passes on Java SE TCK
> 
> then it should be true that B(C).   No need to test!
> 
> :)

LOL. I think you've just proved the whole discussion is fruitless.

> >>Please point me to it!  I always want to see new ways of doing this. 
> >>Challenge away!
> >
> >Okay :-), top-of-head,
> >
> >http://svn.apache.org/repos/asf/excalibur/trunk/framework/impl/src/test/org/apache/avalon/framework/context/test/ContextTestCase.java
> >
> >(one of the last remaining bits of code that can be traced back to apache
> >jserv which was tested using testlet which was around before JUnit). In
> >general, the parts of jakarta and what grew out of it that are derivants of
> >the JServ branch of working (including avalon, now excalibur, cocoon) often
> >do thingsl ike this.
> >
> >The fact I typed that URL from memory and was right is kinda scary, isn't
> >it? I've not worked on that code for *years* and its moved a few dozen
> >times...
> 
> That is scary.  It's also scary that you proposed Avalon as an example :)

LOL. That project has some of the best frigging brilliant ideas in its
codebase, some of which most of the java world *still* hasn't figured out.
Ever heard of classloader hell? We /solved/ that, in 1999, before the J2EE
world knew it would become a problem (all the people at JBoss still haven't
figured it out and I hear Geronimo is having trouble too). I still view OSGi
as a broken committee-designed version of half of avalon. The core of the
code that makes apache cocoon go as fast as it goes lived in avalon and now
lives in excalibur. No-one hardly touches it but no-one needs to because its
as rock solid as, well, the JDK :-)

Quality of the code was never a problem with avalon. Another one of my
assertions is that as code quality approaches infinity the number of
people that can work on it together approaches 0. Lisp hackers don't even
try (you might spot a rephrased version of the "all code continuously evolves
to become like a broken version of Common Lisp", above), in avalon, we did...

...doesn't mean reading the code is a bad idea. Go read :-)

> >>So the problem boils down to the fact that we are implicitly doing 
> >>integration testing.  That's why I've been suggesting the framework - 
> >>let us test the code in isolation first, using "implementation tests". 
> >>Then, if our isolation framework is sexy enough, lets try to reproduce 
> >>the same classloader/security model we would experience in a VM, and do 
> >>spec/API testing.  *Then* we can do integration testing by running the 
> >>code in the VM ("in situ") and do the proper (aka (*.test.*) ) 
> >>spec/API/tck testing.
> >>
> >>I'll post this as a separate message because this one is way too woolly 
> >>at this point.
> >
> >Okay, this does sound like "the core" of the matter. There you go.
> >
> >I'll point out that every time you restrict to an ordered sequence of
> >taking care of things in an open souce community you do slow them down just
> >a little (hey, that's an interesting assertion. Hmm. Should write a book
> >about it I guess) so make sure its what you want :-).
> 
> Huh?

You didn't say "let us test the code in isolation [using a smart framework]",
you said "let us test the code in isolation *first* [using a smart
framework]". I need to write a book about why I think the difference
matters, and it needs to be a book because I'll need many many words...

LSD

Re: [OT] Re: Unit testing revisited

[Stepan Mishura <st...@gmail.com>]

On 3/22/06, Geir Magnusson Jr wrote:
>
>
>
> Leo Simons wrote:
> > On Wed, Mar 22, 2006 at 07:34:16AM -0500, Geir Magnusson Jr wrote:
> >>> LEO :
> >>> I'll point out that every time you restrict to an ordered sequence of
> >>> taking care of things in an open souce community you do slow them down
> just
> >>> a little (hey, that's an interesting assertion. Hmm. Should write a
> book
> >>> about it I guess) so make sure its what you want :-).
> >> Huh?
> >
> > You didn't say "let us test the code in isolation [using a smart
> framework]",
> > you said "let us test the code in isolation *first* [using a smart
> > framework]". I need to write a book about why I think the difference
> > matters, and it needs to be a book because I'll need many many words...
>
> Oh - no, I didn't mean that. Sorry. All three are independent.  You can
> do them in parallel.  We can build our mechanism to do the
> implementation tests correctly while we continue to do everything else.
>
> I just wanted to see if I could get through the fog and be clear what
> the issues are and stop confusing #1 and #2, both of which are important.
>
> To test java.util.Foo, I believe it's important to have BOTH
>
>     java.util.FooTest
>
> AND
>
>     org.apache.harmony.test.java.util.FooTest
>
> as they are intended to test different things (the first as a
> 'un-integrated' implementation test and the second as an 'in-situ'
> API/spec test).


Yes, indeed. We should admit that we need both tests rather then arguing
which test is the right.

Thanks,
Stepan.

If we agree on that and recognize that, I suspect the test debates will
> come to rapid closure, and we'll have a mini-roadmap of what we want to
> do in the testing area that is parallelizable and doesn't hold anyone up.
>
> geir
>
>
>


--
Thanks,
Stepan Mishura
Intel Middleware Products Division

Re: [OT] Re: Unit testing revisited

[Geir Magnusson Jr <ge...@pobox.com>]

Leo Simons wrote:
> On Wed, Mar 22, 2006 at 07:34:16AM -0500, Geir Magnusson Jr wrote:
>>> LEO : 
>>> I'll point out that every time you restrict to an ordered sequence of
>>> taking care of things in an open souce community you do slow them down just
>>> a little (hey, that's an interesting assertion. Hmm. Should write a book
>>> about it I guess) so make sure its what you want :-).
>> Huh?
> 
> You didn't say "let us test the code in isolation [using a smart framework]",
> you said "let us test the code in isolation *first* [using a smart
> framework]". I need to write a book about why I think the difference
> matters, and it needs to be a book because I'll need many many words...

Oh - no, I didn't mean that. Sorry. All three are independent.  You can 
do them in parallel.  We can build our mechanism to do the 
implementation tests correctly while we continue to do everything else.

I just wanted to see if I could get through the fog and be clear what 
the issues are and stop confusing #1 and #2, both of which are important.

To test java.util.Foo, I believe it's important to have BOTH

     java.util.FooTest

AND

     org.apache.harmony.test.java.util.FooTest

as they are intended to test different things (the first as a 
'un-integrated' implementation test and the second as an 'in-situ' 
API/spec test).

If we agree on that and recognize that, I suspect the test debates will 
come to rapid closure, and we'll have a mini-roadmap of what we want to 
do in the testing area that is parallelizable and doesn't hold anyone up.

geir