Re: running tests from OASIS

[kelvin goodson <ke...@gmail.com>]

The OASIS source repository now has a bridge interface to allow the
tests to invoke a selected runtime via runtime specific
implementations of the bridge interface. It also has a bridge
implementation for Tuscany there; the idea being that any implementor
of a runtime can have their bridge implementation hosted in the OASIS
source repo.  So all you need to do is set the test environment to
know which bridge implementation to use by setting a system property
or environment variable to the implementation class name.

Below are some instructions on how to run the current setup.  Once
this has settled down I'll add the instructions to the website.
I'm currently seeing 113 tests run, of which 40 fail.
I'm planning to start working through these to update the wiki
conformance page [1] to reflect our conformance status.

One thing I'm a little unsure of is whether there are any legal issues
with automatically sucking the source out of the OASIS repo into your
local build environment using the svn:externals property.  Does anyone
have any insights on this?

------------------------------------

Oasis Test Execution Documentation

To run the OASIS tests against Tuscany

- extract/update your 2.x source from svn, this will automatically
fetch the test code from the OASIS svn repository if not already
extracted
- change directory to otest/current
- ensure that your runtime environment has either an environment
variable or a Java System Property named
OASIS_TESTENV_RUNTIME_BRIDGE_CLASS set to client.TuscanyRuntimeBridge
- run mvn

Note that this will run the tests against the most recent version of
the OASIS tests,  and changes in the OASIS structure may cause
instability in this build process.  An alternative directory
otest/fixed_version is set to extract a fixed version of the OASIS svn
repository subtree into your extracted source,  the idea being that
the svn:externals property that causes this extraction to occur is
periodically updated to use a repository version id that is known to
work. Just follow the above steps in the otest/fixed_version directory
to run the tests in this way.

You can of course run mvn:eclipse:eclipse in either of these
directories and run the tests individually from the eclipse
environment. Note however that the test suite has the notion of source
as a resource.  To fix this in maven I have used the build helper
plugin to add a source directory.  To do this in eclipse you'll need
to add a source path of src/main resources/General to your eclipse
build properties for the project.



-----------------------------------------------

[1] http://cwiki.apache.org/confluence/display/TUSCANYWIKI/Conformance+Testing


===================================================

2009/4/24 Simon Laws <si...@googlemail.com>
>
> On Fri, Apr 24, 2009 at 10:51 AM, kelvin goodson <kg...@thegoodsons.org.uk> wrote:
> > I've just added the start of something that will make the otests available
> > to all. Under the new otest directory is a current and a fixed_version
> > directory. We can't rely on the foreign svn repository that holds these
> > tests not changing significantly (that wouldn't be a risk to the build since
> > I don't have any intention of hooking this into the build at the moment).
> >
> > The "current" directory has an svn:externals property set which does an
> > unqualified extract of the otests from the foreign svn repo.  The
> > "fixed_version" directory does the same svn:externals extract,  but
> > qualified with a revision number of the foreign repo. That way any one
> > wanting to have a play with these tests can go to an informally blessed
> > version. At the time of writing these work out to be identical.
> >
> > Without having dug very far at the moment I see 84 tests running, with 40
> > failures and 1 error.  I'll work to see how many of these are just down to
> > an imature tuscany infrastructure and how many are real failures/errors.
> > I'll then look at ensuring we have JIRAs to accompany the issues.
> >
> ...snip
>
> Hi Kelvin
>
> Sounds good to me. I haven't updated and tried this yet but am looking
> at svn. Can you say something about how the Tuscany framework code,
> that makes the generic OASIS tests work, will be arranged?
>
> Simon

Re: running tests from OASIS

[ant elder <an...@gmail.com>]

On Wed, May 6, 2009 at 11:45 AM, kelvin goodson <ke...@gmail.com> wrote:

> One thing I'm a little unsure of is whether there are any legal issues
> with automatically sucking the source out of the OASIS repo into your
> local build environment using the svn:externals property.  Does anyone
> have any insights on this?
>

I think we can assume its ok. AFAICT the intention is that they're
covered by the OASIS license and so qualify as read-only standards
documents which we can legally redistribute in both source and binary
so the svn external is ok. The license is still the one that assumes
its embedded in a spec document, it would be better if there was one
tailored for use with source files not included in the specs.

   ...ant