[RFC] Moving error/exception case conformance tests to new conferr directory

[Sh...@lotus.com]

I'd like to consider moving the 'err' tests currently checked in under
xml-xalan/test/conf to be under their own directory tree
xml-xalan/test/conferr, with golds in conferr-gold.  Comments, other
suggestions?

These tests are meant to test Xalan's conformance to the XSLT/XPath specs
under error conditions: generally when the processor cannot complete a
transformation, and throws an exception or error.  Currently they're
automated for the Java version of the processor.

- This will make it slightly easier to automate the error tests with my new
StylesheetErrorTestlet (to be checked in today).

- This will clearly separate the error tests from regular, positive,
conformance tests.  To start with, most people will likely care about the
positive conf tests anyway.

- This will ease maintenance, since even with the Java processor keeping
the error-related tests updated is a very different beast than the positive
tests (we validate part of the exception.toString() which can change for
various reasons).  Eventually we'll also need to instrument the error tests
for the C processor as well.

- This should reduce confusion about exactly which tests have been run;
since commonly now we skip the error tests anyway.

Another potential directory we could add might be called confnew, which is
for newly-proposed conformance tests that have not been 'certified' to pass
on the current build from CVS (or, we could simply put these in
contrib/conf, to avoid more top-level dirs?).

So, if you're still reading 8-), care about this issue, and have a comment
to add, let us know!

- Shane

Re: [RFC] Moving error/exception case conformance tests to new conferrdirectory

[Gary L Peskin <ga...@firstech.com>]

Tom Amiro wrote:
> 
> > PS Have you had a chance to test the Unix build (with the old build.xml)
> > since I made the CVS change to org.apache.xalan.xslt.Process?
> 
> I've had no problems building with the current build.xml on the Unix
> platform, since you checked in your fix. Thank you!
> 
> Tom

Thank YOU.  Sorry I didn't pick up on this being my fault sooner :(

Gary

Re: [RFC] Moving error/exception case conformance tests to new conferrdirectory

[Tom Amiro <To...@Sun.COM>]

> PS Have you had a chance to test the Unix build (with the old build.xml)
> since I made the CVS change to org.apache.xalan.xslt.Process?

I've had no problems building with the current build.xml on the Unix
platform, since you checked in your fix. Thank you!

Tom

Re: [RFC] Moving error/exception case conformance tests to new conferrdirectory

[Gary L Peskin <ga...@firstech.com>]

Shane_Curcuru@lotus.com wrote:
> ...
> So, if you're still reading 8-), 

Yes

> care about this issue, and have a comment to add, let us know!

My comment is: that I defer to your judgement on all such matters! :)

Gary

PS Have you had a chance to test the Unix build (with the old build.xml)
since I made the CVS change to org.apache.xalan.xslt.Process?

Re: [RFC] Moving error/exception case conformance tests to new conferrdirectory

[Tom Amiro <To...@Sun.COM>]

I like the idea of separating the negative tests from the positive tests.
While negative tests are very valuable, they are not as high a
priority as postive tests, so being able to distinguish the two is
very useful.

This may be trivial, but naming the directory "confneg" rather than
"conferr" might be better. At least to me, "neg" more intuitively
implies "negative test" than "err" does. One might think that errors
thrown by positive tests get put in this "conferr" directory.

Tom