You are viewing a plain text version of this content. The canonical link for it is here.
Posted to woden-dev@ws.apache.org by John Kaputin <KA...@uk.ibm.com> on 2006/08/01 12:49:32 UTC
Contributing WSDL 2.0 test cases to W3C test suite
Key feedback from the recent WSDL 2.0 Interop event is that more WSDL 2.0
test cases are needed to expand the coverage of the W3C test suite, so that
the spec can move to Proposed Recommendation and Woden can move to a v0.9
or v1.0 release. We (Woden) really need to help with this as there's not
much contribution to the test suite from the WSDL 2.0 working group.
Lawrence has already contributed a lot of test cases.
There are now report pages on the WSDL 2.0 Working Group's site (
http://www.w3.org/2002/ws/desc/) that indicate test case coverage for WSDL
elements/attributes and for WSDL assertions. See the 'Test Coverage Report'
and 'Test Assertion Report' links in the Test Suite section. From these we
can work out what types of test cases are required. Creating test case WSDL
documents is generally a case of copying an existing WSDL from the test
suite and modifying it. There's a fairly lightweight process for
contributing test cases as patches to the ws-desc mailing list which we can
document on the Woden wiki, if it's not already documented by the WSDL
working group.
Perhaps we can discuss this further under A.O.B. on today's Woden call.
regards,
John Kaputin
---------------------------------------------------------------------
To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: woden-dev-help@ws.apache.org
Re: Contributing WSDL 2.0 test cases to W3C test suite
Posted by Arthur Ryman <ar...@gmail.com>.
Guys,
I've been using the Ant task to run Woden against the W3C test suite. I just
check out the W3C test suite from CVS and run Woden against that directory.
The Ant task dumps the component model as an XML file that is used for
comparision with a baseline. The Ant task also has an option to write a
report that gives the assertions violated. I need to write an XSLT that
compares the report against the W3C test metadata files.
On 8/1/06, Lawrence Mandel <lm...@ca.ibm.com> wrote:
>
>
> The W3C WSDL 2.0 working group has already defined a test metadata format
> that contains a listing of the files included in the test and the expected
> results including assertion violations. For an example see [1].
>
> [1]
> http://dev.w3.org/cvsweb/2002/ws/desc/test-suite/documents/bad/IRI-1B/TestMetadata.xml?rev=1.1&content-type=text/x-cvsweb-markup
>
> Lawrence
>
>
>
> *"Jeremy Hughes" <hu...@apache.org>*
> Sent by: jpjhughes@gmail.com
>
> 08/01/2006 10:41 AM
> Please respond to
> woden-dev@ws.apache.org
>
> To
> woden-dev@ws.apache.org cc
> Subject
> Re: Contributing WSDL 2.0 test cases to W3C test suite
>
>
>
>
> This would be very cool. Have you looked at how the XML Schema spec does
> this? The XmlSchema project downloads a tarball of all the tests. Each of
> the contributors of test cases contributed .testSet files which describes
> each of the sets of tests they contributed. Until recently XmlSchema was
> just testing against one of the test sets and I contributed some changes to
> introspect the .testSet files and test against all the schemas described in
> them.
>
> The metadata stored in the .testSet about each schema test is basically:
> schema filename
> is the schema valid
> is it agreed by W3C
>
> For WSDL I think we need to know what errors should be reported by a
> validator if the WSDL is not valid - currently I believe they're just in a
> 'bad' or 'good' directory.
>
> Also, a tarball would be good - rather than downloading each testcase
> individually - as it might just speed testing up.
>
> Perhaps we should move this discussion over to the W3C list to determine
> how the test files should be described.
>
> Cheers,
> Jeremy
>
> On 8/1/06, *Lawrence Mandel* <*lmandel@ca.ibm.com* <lm...@ca.ibm.com>>
> wrote:
>
> Hi Jeremy,
>
> Currently the Woden test suite hard codes the W3C tests. We need to update
> our test suite to automatically pull down the latest W3C test suite and use
> it to test Woden including testing that invalid documents are flagged by
> Woden with the correct assertions. I'll open a Jira for this if one isn't
> open already.
>
> Lawrence
>
>
> *"Jeremy Hughes" <**hughesj@apache.org* <hu...@apache.org>*>*
> Sent by: *jpjhughes@gmail.com* <jp...@gmail.com>
>
> 08/01/2006 07:14 AM
> Please respond to*
> **woden-dev@ws.apache.org* <wo...@ws.apache.org>
>
>
> To
> *woden-dev@ws.apache.org* <wo...@ws.apache.org> cc
> Subject
> Re: Contributing WSDL 2.0 test cases to W3C test suite
>
>
>
>
>
>
> Hi, how about a coverage report for which of these WSDLs we test Woden
> against ?
>
> Cheers,
> Jeremy
>
> On 8/1/06, John Kaputin <*KAPUTIN@uk.ibm.com* <KA...@uk.ibm.com>> wrote:
> >
> > Key feedback from the recent WSDL 2.0 Interop event is that more WSDL
> 2.0
> > test cases are needed to expand the coverage of the W3C test suite, so
> that
> > the spec can move to Proposed Recommendation and Woden can move to a
> v0.9
> > or v1.0 release. We (Woden) really need to help with this as there's
> not
> > much contribution to the test suite from the WSDL 2.0 working group.
> > Lawrence has already contributed a lot of test cases.
> >
> > There are now report pages on the WSDL 2.0 Working Group's site (
> > *http://www.w3.org/2002/ws/desc/* <http://www.w3.org/2002/ws/desc/>)
> that indicate test case coverage for WSDL
> > elements/attributes and for WSDL assertions. See the 'Test Coverage
> Report'
> > and 'Test Assertion Report' links in the Test Suite section. From these
> we
> > can work out what types of test cases are required. Creating test case
> WSDL
> > documents is generally a case of copying an existing WSDL from the test
> > suite and modifying it. There's a fairly lightweight process for
> > contributing test cases as patches to the ws-desc mailing list which we
> can
> > document on the Woden wiki, if it's not already documented by the WSDL
> > working group.
> >
> > Perhaps we can discuss this further under A.O.B. on today's Woden call.
> >
> > regards,
> > John Kaputin
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: *woden-dev-unsubscribe@ws.apache.org*<wo...@ws.apache.org>
> > For additional commands, e-mail: *woden-dev-help@ws.apache.org*<wo...@ws.apache.org>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: *woden-dev-unsubscribe@ws.apache.org*<wo...@ws.apache.org>
> For additional commands, e-mail: *woden-dev-help@ws.apache.org*<wo...@ws.apache.org>
>
>
>
>
Re: Contributing WSDL 2.0 test cases to W3C test suite
Posted by Lawrence Mandel <lm...@ca.ibm.com>.
The W3C WSDL 2.0 working group has already defined a test metadata format
that contains a listing of the files included in the test and the expected
results including assertion violations. For an example see [1].
[1]
http://dev.w3.org/cvsweb/2002/ws/desc/test-suite/documents/bad/IRI-1B/TestMetadata.xml?rev=1.1&content-type=text/x-cvsweb-markup
Lawrence
"Jeremy Hughes" <hu...@apache.org>
Sent by: jpjhughes@gmail.com
08/01/2006 10:41 AM
Please respond to
woden-dev@ws.apache.org
To
woden-dev@ws.apache.org
cc
Subject
Re: Contributing WSDL 2.0 test cases to W3C test suite
This would be very cool. Have you looked at how the XML Schema spec does
this? The XmlSchema project downloads a tarball of all the tests. Each of
the contributors of test cases contributed .testSet files which describes
each of the sets of tests they contributed. Until recently XmlSchema was
just testing against one of the test sets and I contributed some changes
to introspect the .testSet files and test against all the schemas
described in them.
The metadata stored in the .testSet about each schema test is basically:
schema filename
is the schema valid
is it agreed by W3C
For WSDL I think we need to know what errors should be reported by a
validator if the WSDL is not valid - currently I believe they're just in a
'bad' or 'good' directory.
Also, a tarball would be good - rather than downloading each testcase
individually - as it might just speed testing up.
Perhaps we should move this discussion over to the W3C list to determine
how the test files should be described.
Cheers,
Jeremy
On 8/1/06, Lawrence Mandel <lm...@ca.ibm.com> wrote:
Hi Jeremy,
Currently the Woden test suite hard codes the W3C tests. We need to update
our test suite to automatically pull down the latest W3C test suite and
use it to test Woden including testing that invalid documents are flagged
by Woden with the correct assertions. I'll open a Jira for this if one
isn't open already.
Lawrence
"Jeremy Hughes" <hu...@apache.org>
Sent by: jpjhughes@gmail.com
08/01/2006 07:14 AM
Please respond to
woden-dev@ws.apache.org
To
woden-dev@ws.apache.org
cc
Subject
Re: Contributing WSDL 2.0 test cases to W3C test suite
Hi, how about a coverage report for which of these WSDLs we test Woden
against ?
Cheers,
Jeremy
On 8/1/06, John Kaputin <KA...@uk.ibm.com> wrote:
>
> Key feedback from the recent WSDL 2.0 Interop event is that more WSDL
2.0
> test cases are needed to expand the coverage of the W3C test suite, so
that
> the spec can move to Proposed Recommendation and Woden can move to a
v0.9
> or v1.0 release. We (Woden) really need to help with this as there's
not
> much contribution to the test suite from the WSDL 2.0 working group.
> Lawrence has already contributed a lot of test cases.
>
> There are now report pages on the WSDL 2.0 Working Group's site (
> http://www.w3.org/2002/ws/desc/) that indicate test case coverage for
WSDL
> elements/attributes and for WSDL assertions. See the 'Test Coverage
Report'
> and 'Test Assertion Report' links in the Test Suite section. From these
we
> can work out what types of test cases are required. Creating test case
WSDL
> documents is generally a case of copying an existing WSDL from the test
> suite and modifying it. There's a fairly lightweight process for
> contributing test cases as patches to the ws-desc mailing list which we
can
> document on the Woden wiki, if it's not already documented by the WSDL
> working group.
>
> Perhaps we can discuss this further under A.O.B. on today's Woden call.
>
> regards,
> John Kaputin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: woden-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: woden-dev-help@ws.apache.org
Re: Contributing WSDL 2.0 test cases to W3C test suite
Posted by Jeremy Hughes <hu...@apache.org>.
This would be very cool. Have you looked at how the XML Schema spec does
this? The XmlSchema project downloads a tarball of all the tests. Each of
the contributors of test cases contributed .testSet files which describes
each of the sets of tests they contributed. Until recently XmlSchema was
just testing against one of the test sets and I contributed some changes to
introspect the .testSet files and test against all the schemas described in
them.
The metadata stored in the .testSet about each schema test is basically:
schema filename
is the schema valid
is it agreed by W3C
For WSDL I think we need to know what errors should be reported by a
validator if the WSDL is not valid - currently I believe they're just in a
'bad' or 'good' directory.
Also, a tarball would be good - rather than downloading each testcase
individually - as it might just speed testing up.
Perhaps we should move this discussion over to the W3C list to determine how
the test files should be described.
Cheers,
Jeremy
On 8/1/06, Lawrence Mandel <lm...@ca.ibm.com> wrote:
>
>
> Hi Jeremy,
>
> Currently the Woden test suite hard codes the W3C tests. We need to update
> our test suite to automatically pull down the latest W3C test suite and use
> it to test Woden including testing that invalid documents are flagged by
> Woden with the correct assertions. I'll open a Jira for this if one isn't
> open already.
>
> Lawrence
>
>
>
> *"Jeremy Hughes" <hu...@apache.org>*
> Sent by: jpjhughes@gmail.com
>
> 08/01/2006 07:14 AM Please respond to
> woden-dev@ws.apache.org
>
> To
> woden-dev@ws.apache.org cc
>
> Subject
> Re: Contributing WSDL 2.0 test cases to W3C test suite
>
>
>
>
>
>
> Hi, how about a coverage report for which of these WSDLs we test Woden
> against ?
>
> Cheers,
> Jeremy
>
> On 8/1/06, John Kaputin <KA...@uk.ibm.com> wrote:
> >
> > Key feedback from the recent WSDL 2.0 Interop event is that more WSDL
> 2.0
> > test cases are needed to expand the coverage of the W3C test suite, so
> that
> > the spec can move to Proposed Recommendation and Woden can move to a
> v0.9
> > or v1.0 release. We (Woden) really need to help with this as there's
> not
> > much contribution to the test suite from the WSDL 2.0 working group.
> > Lawrence has already contributed a lot of test cases.
> >
> > There are now report pages on the WSDL 2.0 Working Group's site (
> > http://www.w3.org/2002/ws/desc/) that indicate test case coverage for
> WSDL
> > elements/attributes and for WSDL assertions. See the 'Test Coverage
> Report'
> > and 'Test Assertion Report' links in the Test Suite section. From these
> we
> > can work out what types of test cases are required. Creating test case
> WSDL
> > documents is generally a case of copying an existing WSDL from the test
> > suite and modifying it. There's a fairly lightweight process for
> > contributing test cases as patches to the ws-desc mailing list which we
> can
> > document on the Woden wiki, if it's not already documented by the WSDL
> > working group.
> >
> > Perhaps we can discuss this further under A.O.B. on today's Woden call.
> >
> > regards,
> > John Kaputin
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: woden-dev-help@ws.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: woden-dev-help@ws.apache.org
>
>
>
Re: Contributing WSDL 2.0 test cases to W3C test suite
Posted by Lawrence Mandel <lm...@ca.ibm.com>.
Hi Jeremy,
Currently the Woden test suite hard codes the W3C tests. We need to update
our test suite to automatically pull down the latest W3C test suite and
use it to test Woden including testing that invalid documents are flagged
by Woden with the correct assertions. I'll open a Jira for this if one
isn't open already.
Lawrence
"Jeremy Hughes" <hu...@apache.org>
Sent by: jpjhughes@gmail.com
08/01/2006 07:14 AM
Please respond to
woden-dev@ws.apache.org
To
woden-dev@ws.apache.org
cc
Subject
Re: Contributing WSDL 2.0 test cases to W3C test suite
Hi, how about a coverage report for which of these WSDLs we test Woden
against ?
Cheers,
Jeremy
On 8/1/06, John Kaputin <KA...@uk.ibm.com> wrote:
>
> Key feedback from the recent WSDL 2.0 Interop event is that more WSDL
2.0
> test cases are needed to expand the coverage of the W3C test suite, so
that
> the spec can move to Proposed Recommendation and Woden can move to a
v0.9
> or v1.0 release. We (Woden) really need to help with this as there's
not
> much contribution to the test suite from the WSDL 2.0 working group.
> Lawrence has already contributed a lot of test cases.
>
> There are now report pages on the WSDL 2.0 Working Group's site (
> http://www.w3.org/2002/ws/desc/) that indicate test case coverage for
WSDL
> elements/attributes and for WSDL assertions. See the 'Test Coverage
Report'
> and 'Test Assertion Report' links in the Test Suite section. From these
we
> can work out what types of test cases are required. Creating test case
WSDL
> documents is generally a case of copying an existing WSDL from the test
> suite and modifying it. There's a fairly lightweight process for
> contributing test cases as patches to the ws-desc mailing list which we
can
> document on the Woden wiki, if it's not already documented by the WSDL
> working group.
>
> Perhaps we can discuss this further under A.O.B. on today's Woden call.
>
> regards,
> John Kaputin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: woden-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: woden-dev-help@ws.apache.org
Re: Contributing WSDL 2.0 test cases to W3C test suite
Posted by Jeremy Hughes <hu...@apache.org>.
Hi, how about a coverage report for which of these WSDLs we test Woden against ?
Cheers,
Jeremy
On 8/1/06, John Kaputin <KA...@uk.ibm.com> wrote:
>
> Key feedback from the recent WSDL 2.0 Interop event is that more WSDL 2.0
> test cases are needed to expand the coverage of the W3C test suite, so that
> the spec can move to Proposed Recommendation and Woden can move to a v0.9
> or v1.0 release. We (Woden) really need to help with this as there's not
> much contribution to the test suite from the WSDL 2.0 working group.
> Lawrence has already contributed a lot of test cases.
>
> There are now report pages on the WSDL 2.0 Working Group's site (
> http://www.w3.org/2002/ws/desc/) that indicate test case coverage for WSDL
> elements/attributes and for WSDL assertions. See the 'Test Coverage Report'
> and 'Test Assertion Report' links in the Test Suite section. From these we
> can work out what types of test cases are required. Creating test case WSDL
> documents is generally a case of copying an existing WSDL from the test
> suite and modifying it. There's a fairly lightweight process for
> contributing test cases as patches to the ws-desc mailing list which we can
> document on the Woden wiki, if it's not already documented by the WSDL
> working group.
>
> Perhaps we can discuss this further under A.O.B. on today's Woden call.
>
> regards,
> John Kaputin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: woden-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: woden-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: woden-dev-help@ws.apache.org