You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jcp-open@apache.org by Steve Loughran <st...@apache.org> on 2008/03/17 12:17:43 UTC
Re: Handling Innovation and Producing an RI
Endre Stølsvik wrote:
>
> Can this be easily copied in other projects? Do one have the dedication
> from the dumpsters to pull this through, as one has _had_ with Sun on
> Tomcat? .. or the dedication from other vendors, having full-time
> employees basically owning the implementation, as JBoss on Tomcat?
>
> It most definitely didn't work with IBM on Pluto, IMO.
>
I think the issue is not so much RI as general code quality. Sometimes
companies, to great PR, donate truckloads of source to the OSS world,
including to apache, but neglect to include a single line of comment on
it. Because in the closed source team, there was always fred who
understood the database module, and sarah who knew the networking side
of things. Whenever you had a problem, you knew who to walk over and ask.
In the OSS ecosystem, any project that is only understood by a group of
developers who work for a single vendor, is at risk of the vendor
changing their plans on a whim. I actually think JBoss is different
-they are committed to Tomcat and their stack- but again, in Axis1.x, a
lot of the original IBM axis team suddenly got repurposed to deal with
internal crises, and others had to (eventually) step into the gap, but
even there there whole swathes of code we were scared of.
At least with a TCK-driven project, you have a test kit that is the
effective specification.
-steve
Re: Handling Innovation and Producing an RI
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Mar 17, 2008, at 7:13 PM, Sam Ruby wrote:
> On Mon, Mar 17, 2008 at 7:00 PM, Geir Magnusson Jr. <ge...@pobox.com>
> wrote:
>>
>> On Mar 17, 2008, at 6:51 PM, Sam Ruby wrote:
>>
>>> On Mon, Mar 17, 2008 at 5:26 PM, Geir Magnusson Jr. <ge...@pobox.com>
>>> wrote:
>>>>
>>>> On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
>>>>
>>>>>
>>>>> On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
>>>>>
>>>>>>
>>>>>> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
>>>>>>
>>>>>>> Sam Ruby wrote:
>>>>>>>
>>>>>>>> Given how hard it was to challenge the TCK, in some cases we
>>>>>>>> simply
>>>>>>>> complied instead.
>>>>>>>
>>>>>>> which is, in a way, an argument against JCP standards,
>>>>>>> especially
>>>>>>> those whose TCK is meant to be a secret
>>>>>>
>>>>>> I don't think I've seen a binary TCK yet.
>>>>>
>>>>> If I may comment, you have to sign a NDA to see the sources.
>>>>
>>>> The point is that you can see the source to figure out what's going
>>>> on. Clearly open source TCKs would be far better.
>>>
>>> Steve's point is valid too. In some ways, the TCK effectively
>>> defines
>>> additional standards, ones that are not agreed to by the expert
>>> group,
>>> and not publicly documented.
>>
>> I think that depends mostly on the spec lead, doesn't it?
>
> I guess that would depend on what your definition of 'mostly' is.
>
> As it stands now, you can't claim compliance until you pass the test.
> If you spot an issue with a test, you must report it, make your case,
> wait for a fix to get implemented, get a new version of the test, and
> retest. If all goes well, that's generally a few days worth of work.
No - a spec lead can simply agree, invalidate the test, and that's
that - you don't have to get it fixed. Some TCKs, like the Java EE
TCK, distribute a regularly updated machine readbale exclusion list of
invalidated tests that the test suite uses at test time.
>
>
> Sometimes it is easier to simply comply instead.
>
> If there weren't NDA requirements, we could simply report the full
> status to our licensees so that they could make an informed choice as
> to whether they want to pick up a given alpha, beta, or release
> candidate. Reporting and fixing would go on in parallel. And the
> tests themselves would improve in the process.
>
> - Sam Ruby
Re: Handling Innovation and Producing an RI
Posted by Sam Ruby <ru...@intertwingly.net>.
On Mon, Mar 17, 2008 at 7:00 PM, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Mar 17, 2008, at 6:51 PM, Sam Ruby wrote:
>
> > On Mon, Mar 17, 2008 at 5:26 PM, Geir Magnusson Jr. <ge...@pobox.com>
> > wrote:
> >>
> >> On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
> >>
> >>>
> >>> On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
> >>>
> >>>>
> >>>> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
> >>>>
> >>>>> Sam Ruby wrote:
> >>>>>
> >>>>>> Given how hard it was to challenge the TCK, in some cases we
> >>>>>> simply
> >>>>>> complied instead.
> >>>>>
> >>>>> which is, in a way, an argument against JCP standards, especially
> >>>>> those whose TCK is meant to be a secret
> >>>>
> >>>> I don't think I've seen a binary TCK yet.
> >>>
> >>> If I may comment, you have to sign a NDA to see the sources.
> >>
> >> The point is that you can see the source to figure out what's going
> >> on. Clearly open source TCKs would be far better.
> >
> > Steve's point is valid too. In some ways, the TCK effectively defines
> > additional standards, ones that are not agreed to by the expert group,
> > and not publicly documented.
>
> I think that depends mostly on the spec lead, doesn't it?
I guess that would depend on what your definition of 'mostly' is.
As it stands now, you can't claim compliance until you pass the test.
If you spot an issue with a test, you must report it, make your case,
wait for a fix to get implemented, get a new version of the test, and
retest. If all goes well, that's generally a few days worth of work.
Sometimes it is easier to simply comply instead.
If there weren't NDA requirements, we could simply report the full
status to our licensees so that they could make an informed choice as
to whether they want to pick up a given alpha, beta, or release
candidate. Reporting and fixing would go on in parallel. And the
tests themselves would improve in the process.
- Sam Ruby
Re: Handling Innovation and Producing an RI
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Mar 17, 2008, at 6:51 PM, Sam Ruby wrote:
> On Mon, Mar 17, 2008 at 5:26 PM, Geir Magnusson Jr. <ge...@pobox.com>
> wrote:
>>
>> On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
>>
>>>
>>> On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
>>>
>>>>
>>>> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
>>>>
>>>>> Sam Ruby wrote:
>>>>>
>>>>>> Given how hard it was to challenge the TCK, in some cases we
>>>>>> simply
>>>>>> complied instead.
>>>>>
>>>>> which is, in a way, an argument against JCP standards, especially
>>>>> those whose TCK is meant to be a secret
>>>>
>>>> I don't think I've seen a binary TCK yet.
>>>
>>> If I may comment, you have to sign a NDA to see the sources.
>>
>> The point is that you can see the source to figure out what's going
>> on. Clearly open source TCKs would be far better.
>
> Steve's point is valid too. In some ways, the TCK effectively defines
> additional standards, ones that are not agreed to by the expert group,
> and not publicly documented.
I think that depends mostly on the spec lead, doesn't it?
>
> Yes, open source would be ideal. But we are not asking for that.
> What we have stated as a goal is the complete elimination of the NDA
> for project software by 2010.
Yep. That's a good point to emphasize.
geir
>
>
> - Sam Ruby
Re: Handling Innovation and Producing an RI
Posted by Sam Ruby <ru...@intertwingly.net>.
On Mon, Mar 17, 2008 at 5:26 PM, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
>
> >
> > On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
> >
> >>
> >> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
> >>
> >>> Sam Ruby wrote:
> >>>
> >>>> Given how hard it was to challenge the TCK, in some cases we simply
> >>>> complied instead.
> >>>
> >>> which is, in a way, an argument against JCP standards, especially
> >>> those whose TCK is meant to be a secret
> >>
> >> I don't think I've seen a binary TCK yet.
> >
> > If I may comment, you have to sign a NDA to see the sources.
>
> The point is that you can see the source to figure out what's going
> on. Clearly open source TCKs would be far better.
Steve's point is valid too. In some ways, the TCK effectively defines
additional standards, ones that are not agreed to by the expert group,
and not publicly documented.
Yes, open source would be ideal. But we are not asking for that.
What we have stated as a goal is the complete elimination of the NDA
for project software by 2010.
- Sam Ruby
Re: Handling Innovation and Producing an RI
Posted by Steve Loughran <st...@apache.org>.
Geir Magnusson Jr. wrote:
>
> On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
>
>>
>> On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
>>
>>>
>>> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
>>>
>>>> Sam Ruby wrote:
>>>>
>>>>> Given how hard it was to challenge the TCK, in some cases we simply
>>>>> complied instead.
>>>>
>>>> which is, in a way, an argument against JCP standards, especially
>>>> those whose TCK is meant to be a secret
>>>
>>> I don't think I've seen a binary TCK yet.
>>
>> If I may comment, you have to sign a NDA to see the sources.
>
> The point is that you can see the source to figure out what's going on.
> Clearly open source TCKs would be far better.
you arent allowed to talk about the test with people that havent signed
the NDA, can't include code snippets in a bug rep, cant ask people who
havent signed the NDA to run the TCK over their tests before submitting
a patch. That == secret to me.
To be fair, Sun do make the TCK available. Whereas for OOXML, MS have
published truck loads of XSD and english, and no test suites for the
various XML formats that make up the system.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Re: Handling Innovation and Producing an RI
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Mar 17, 2008, at 3:37 PM, Craig L Russell wrote:
>
> On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
>
>>
>> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
>>
>>> Sam Ruby wrote:
>>>
>>>> Given how hard it was to challenge the TCK, in some cases we simply
>>>> complied instead.
>>>
>>> which is, in a way, an argument against JCP standards, especially
>>> those whose TCK is meant to be a secret
>>
>> I don't think I've seen a binary TCK yet.
>
> If I may comment, you have to sign a NDA to see the sources.
The point is that you can see the source to figure out what's going
on. Clearly open source TCKs would be far better.
geir
>
>
> But considering the alternative (specs without compliance tests),
> TCK works pretty well.
>
> Craig
>>
>>
>> geir
>>
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
Re: Handling Innovation and Producing an RI
Posted by Craig L Russell <Cr...@Sun.COM>.
On Mar 17, 2008, at 10:36 AM, Geir Magnusson Jr. wrote:
>
> On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
>
>> Sam Ruby wrote:
>>
>>> Given how hard it was to challenge the TCK, in some cases we simply
>>> complied instead.
>>
>> which is, in a way, an argument against JCP standards, especially
>> those whose TCK is meant to be a secret
>
> I don't think I've seen a binary TCK yet.
If I may comment, you have to sign a NDA to see the sources.
But considering the alternative (specs without compliance tests), TCK
works pretty well.
Craig
>
>
> geir
>
Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: Handling Innovation and Producing an RI
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Mar 17, 2008, at 12:36 PM, Steve Loughran wrote:
> Sam Ruby wrote:
>
>> Given how hard it was to challenge the TCK, in some cases we simply
>> complied instead.
>
> which is, in a way, an argument against JCP standards, especially
> those whose TCK is meant to be a secret
I don't think I've seen a binary TCK yet.
geir
Re: Handling Innovation and Producing an RI
Posted by Steve Loughran <st...@apache.org>.
Sam Ruby wrote:
> On Mon, Mar 17, 2008 at 7:23 AM, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>> On Mar 17, 2008, at 7:17 AM, Steve Loughran wrote:
>>
>> > Endre Stølsvik wrote:
>> >
>> >> Can this be easily copied in other projects? Do one have the
>> >> dedication from the dumpsters to pull this through, as one has
>> >> _had_ with Sun on Tomcat? .. or the dedication from other vendors,
>> >> having full-time employees basically owning the implementation, as
>> >> JBoss on Tomcat?
>> >> It most definitely didn't work with IBM on Pluto, IMO.
>> >
>> > I think the issue is not so much RI as general code quality.
>> > Sometimes companies, to great PR, donate truckloads of source to the
>> > OSS world, including to apache, but neglect to include a single line
>> > of comment on it. Because in the closed source team, there was
>> > always fred who understood the database module, and sarah who knew
>> > the networking side of things. Whenever you had a problem, you knew
>> > who to walk over and ask.
>> >
>> > In the OSS ecosystem, any project that is only understood by a group
>> > of developers who work for a single vendor, is at risk of the vendor
>> > changing their plans on a whim. I actually think JBoss is different -
>> > they are committed to Tomcat and their stack- but again, in Axis1.x,
>> > a lot of the original IBM axis team suddenly got repurposed to deal
>> > with internal crises, and others had to (eventually) step into the
>> > gap, but even there there whole swathes of code we were scared of.
>> >
>> > At least with a TCK-driven project, you have a test kit that is the
>> > effective specification.
>>
>> That's not always the case - there are many instances where tests in
>> TCKs are challenged because there are implementation assumptions built
>> in that can conflict with other spec-complaint implementations.
I'm a firm believer in test-driven specifications: your specification is
your test suite and vice versa. If you build the test suite after the
spec, there are always inconsistencies.
That said, I dont think junit or testng are good languages for
specifying the behaviour of distributed systems.
>
> Glen Daniels would likely find statements that Axis 1.x was dominated
> by IBM to be amusing.
I dont think they dominated it, but they dominated whole swathes of the
WSDL- to java code that, when the team got reassigned to focus on
websphere, got pretty much abandonded without a single comment.
>
> And to back up Geir's point: Axis 1.x did generate a number of
> successful TCK challenges.
>
> The TCK was effectively a second test suite, an adjunct to the one we
> already had at the time. It uncovered a number of bugs. In the
> initial runs, if memory servers, the largest set was simple Axis bugs.
> Not JAX RPC compliance issues, but bugs of other sorts. These second
> largest set was JAX RPC issues. The remainder (and smallest set of
> the three) were TCK issues. An example of the latter was that SOAP
> does involve HTTP headers, and HTTP headers have a number of options
> in the way that they are expressed (e.g. quoting). The TCK at the
> time made a few simplifying assumptions in areas such as these in ways
> that weren't warranted.
heh, java.net.HttpURL makes a few simplifying assumptions about HTTP
headers that aren't warranted. Like its cookie handling. or its habit of
patching the content-length header if the response was too short.
>
> Given how hard it was to challenge the TCK, in some cases we simply
> complied instead.
which is, in a way, an argument against JCP standards, especially those
whose TCK is meant to be a secret
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Re: Handling Innovation and Producing an RI
Posted by Sam Ruby <ru...@intertwingly.net>.
On Mon, Mar 17, 2008 at 7:23 AM, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Mar 17, 2008, at 7:17 AM, Steve Loughran wrote:
>
> > Endre Stølsvik wrote:
> >
> >> Can this be easily copied in other projects? Do one have the
> >> dedication from the dumpsters to pull this through, as one has
> >> _had_ with Sun on Tomcat? .. or the dedication from other vendors,
> >> having full-time employees basically owning the implementation, as
> >> JBoss on Tomcat?
> >> It most definitely didn't work with IBM on Pluto, IMO.
> >
> > I think the issue is not so much RI as general code quality.
> > Sometimes companies, to great PR, donate truckloads of source to the
> > OSS world, including to apache, but neglect to include a single line
> > of comment on it. Because in the closed source team, there was
> > always fred who understood the database module, and sarah who knew
> > the networking side of things. Whenever you had a problem, you knew
> > who to walk over and ask.
> >
> > In the OSS ecosystem, any project that is only understood by a group
> > of developers who work for a single vendor, is at risk of the vendor
> > changing their plans on a whim. I actually think JBoss is different -
> > they are committed to Tomcat and their stack- but again, in Axis1.x,
> > a lot of the original IBM axis team suddenly got repurposed to deal
> > with internal crises, and others had to (eventually) step into the
> > gap, but even there there whole swathes of code we were scared of.
> >
> > At least with a TCK-driven project, you have a test kit that is the
> > effective specification.
>
> That's not always the case - there are many instances where tests in
> TCKs are challenged because there are implementation assumptions built
> in that can conflict with other spec-complaint implementations.
Glen Daniels would likely find statements that Axis 1.x was dominated
by IBM to be amusing.
And to back up Geir's point: Axis 1.x did generate a number of
successful TCK challenges.
The TCK was effectively a second test suite, an adjunct to the one we
already had at the time. It uncovered a number of bugs. In the
initial runs, if memory servers, the largest set was simple Axis bugs.
Not JAX RPC compliance issues, but bugs of other sorts. These second
largest set was JAX RPC issues. The remainder (and smallest set of
the three) were TCK issues. An example of the latter was that SOAP
does involve HTTP headers, and HTTP headers have a number of options
in the way that they are expressed (e.g. quoting). The TCK at the
time made a few simplifying assumptions in areas such as these in ways
that weren't warranted.
Given how hard it was to challenge the TCK, in some cases we simply
complied instead.
- Sam Ruby
Re: Handling Innovation and Producing an RI
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Mar 17, 2008, at 7:17 AM, Steve Loughran wrote:
> Endre Stølsvik wrote:
>
>> Can this be easily copied in other projects? Do one have the
>> dedication from the dumpsters to pull this through, as one has
>> _had_ with Sun on Tomcat? .. or the dedication from other vendors,
>> having full-time employees basically owning the implementation, as
>> JBoss on Tomcat?
>> It most definitely didn't work with IBM on Pluto, IMO.
>
> I think the issue is not so much RI as general code quality.
> Sometimes companies, to great PR, donate truckloads of source to the
> OSS world, including to apache, but neglect to include a single line
> of comment on it. Because in the closed source team, there was
> always fred who understood the database module, and sarah who knew
> the networking side of things. Whenever you had a problem, you knew
> who to walk over and ask.
>
> In the OSS ecosystem, any project that is only understood by a group
> of developers who work for a single vendor, is at risk of the vendor
> changing their plans on a whim. I actually think JBoss is different -
> they are committed to Tomcat and their stack- but again, in Axis1.x,
> a lot of the original IBM axis team suddenly got repurposed to deal
> with internal crises, and others had to (eventually) step into the
> gap, but even there there whole swathes of code we were scared of.
>
> At least with a TCK-driven project, you have a test kit that is the
> effective specification.
That's not always the case - there are many instances where tests in
TCKs are challenged because there are implementation assumptions built
in that can conflict with other spec-complaint implementations.
geir