Testable specifications

[Steve Loughran <st...@apache.org>]

Andrew C. Oliver wrote:

> I kind of like the concept of a test kit personally.  Spec's can be 
> shitty and have 12 logical interpretations.  Sure there can be multiple 
> ways to pass a unit test but boy I wish clients and servers had to pass 
> an IMAP TCK so that we didn't have 12 strange divergent versions of 
> IMAP.  I think if the TCKs were open, you'd find they'd be better as a 
> side effect.

As I have said myself : 
http://people.apache.org/~stevel/slides/distributed_testing_with_smartfrog_slides.pdf

Any specification without tests shouldn't exist.

I'd view WS-Addressing as a worst-in-case example; the fact that the TCK 
for Java6+ has to incorporate the JAX-WS API, with its own tests for 
WS-Addressing, is semi-irrelevant given there is 0 guarantee of interop 
between other SOAP stacks with their own misinterpretation of WS-A, a 
spec that got to the 1.0 status at the W3C without a single test case 
(*). If you dont think of testing when the spec is written, it will be 
untestable, and interop impossible.


Our Grid-related spec, has a test kit hosted on sourceforge: 
http://sourceforge.net/projects/deployment
and it builds under Gump. Every night. We still suffer interop problems 
between WS-A and WS-RF implementations, but those things we cannot take 
blame for.

Putting the JCP aside for a moment (they do at least have tests; its 
access that is the issue), perhaps we could consider a more ruthless 
policy towards W3C and OASIS specs in the future. Vendors are not above 
donating large bodies of code to actually legitimise a WS-* 
specification or two, and if we pushed back on specifications without 
normative test suites, they'd find greater barriers to adoption up 
front, but longer term things would actually interoperate, which can 
only be a good thing.

-steve


* to be precise, final draft, before they started discussing testability.

-- 
Steve Loughran                  http://www.1060.org/blogxter/publish/5
Author: Ant in Action           http://antbook.org/

Re: Testable specifications

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

Ralph Goers wrote:
> Sam Ruby said:
>> Let me caution everybody here.  We (the ASF) aren't here to bolster or
>> destroy monopolies[1].  Nor do we exist as a referendum of sorts on
>> various standards bodies.  The ASF is a place where like minded people
>> can join together to build sustainable communities anchored by code
>> offered under the Apache License.
> 
> This is true because the ASF membership has defined itself this way. If
> the ASF members decided they wanted to emulate the FSF or be a referendum
> on standards bodies, then they would be free to do so.  I'm not trying to
> argue with the point you are making. I just like to point out that the way
> things are at the ASF is because that is the way we choose them to be.

This thread is entirely off topic.  The definition of "what is the ASF" is
for the members to collectively decide, which is why anything more on this
topic belongs on members@.  I'm pretty sure an "activist" ASF would be seen
by many (including me) as exclusionary, as opposed to inclusive.  Sam's
point above is that we have built the ASF as an inclusive community of coders.

There is a huge diversity of opinion between the members on technology, the
politics of coding, IP law and so forth.  And this diversity is part of what
makes the ASF unique.  But the *members* are free to collectively decide to
go another direction, take it up with them :)

Re: Testable specifications

[Ralph Goers <Ra...@dslextreme.com>]

Sam Ruby said:
>
> Let me caution everybody here.  We (the ASF) aren't here to bolster or
> destroy monopolies[1].  Nor do we exist as a referendum of sorts on
> various standards bodies.  The ASF is a place where like minded people
> can join together to build sustainable communities anchored by code
> offered under the Apache License.
>

This is true because the ASF membership has defined itself this way. If
the ASF members decided they wanted to emulate the FSF or be a referendum
on standards bodies, then they would be free to do so.  I'm not trying to
argue with the point you are making. I just like to point out that the way
things are at the ASF is because that is the way we choose them to be.

Ralph

Re: Testable specifications

[Sam Ruby <ru...@apache.org>]

On 7/10/07, Steve Loughran <st...@apache.org> wrote:
>
> Putting the JCP aside for a moment (they do at least have tests; its
> access that is the issue), perhaps we could consider a more ruthless
> policy towards W3C and OASIS specs in the future. Vendors are not above
> donating large bodies of code to actually legitimise a WS-*
> specification or two, and if we pushed back on specifications without
> normative test suites, they'd find greater barriers to adoption up
> front, but longer term things would actually interoperate, which can
> only be a good thing.

First, if you want to put aside the JCP for the moment, perhaps this
isn't the list to do that.

Let me caution everybody here.  We (the ASF) aren't here to bolster or
destroy monopolies[1].  Nor do we exist as a referendum of sorts on
various standards bodies.  The ASF is a place where like minded people
can join together to build sustainable communities anchored by code
offered under the Apache License.

- Sam Ruby

[1] http://refspace.com/quotes/The_Way_We_Live_Now:_Questions_for_Linus_Torvalds