You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by gr...@ca.ibm.com on 2002/12/03 23:09:16 UTC
Last call: conf test restructuring
Hi all,
After lots of discussion regarding the conformance test restructuring,
there were many people in favour of option (2), and/or proposed variants of
option (2). We propose to move forward with Shane Curcuru's proposal for
restructuring the test harness.
The goal of this discussion is to "agree on basic principles to get the job
done for existing committers to test Xalan releases, and let the committers
who will actually perform this work finish the details and implement a
working
system".
- create an accept tree, parallel to the conf tree
- have in the accept tree only a few categories (including idkey, output,
sort, whitespace)
- have in these categories only the tests for which processors do not agree
on the output (see considerations below)
- pass an option to the test harness (perhaps "qetest.processor")
indicating which processor was used
- update the test harness to choose the correct gold file for the test
- an example of the naming scheme: (sort21 case)
/accept/sort/sort21.xml
/accept/sort/sort21.xsl
XML and XSL pair for test
/accept-gold/sort/sort21.out
"default" output file that output will be checked against
/accept-gold/sort/sort21-XalanJ-C.out
should XalanJ-C ("XSLTC") have correct output that doesn't match
sort21.out, it would go here.
similar files may exist for XalanC or XalanJ-I ("Xalan").
Considerations made in the proposal:
- attempts will be made to reduce the number of test cases going into the
accept bucket. This will be done by making improvements to the test harness
in detecting these differences, and rewriting test cases to remain
processor independent where possible.
- attempts will be made to reduce the number of 'gold' files in the
accept-gold tree. By only creating processor-specific gold files when
necessary, duplication of files will be reduced.
- as the test files are tagged with every new Xalan release, it should be
easy to check out test files corresponding to a certain previous release.
- in the event of future multi-dimensionality, additional gold files can be
created, or directory structures can be duplicated.
Please respond with any comments, objections, opinions you may have --
Thanks!
Gordon
----
Gordon Chiu
grchiu@ca.ibm.com
Re: Last call: conf test restructuring
Posted by Santiago Pericas-Geertsen <Sa...@sun.com>.
+1
-- Santiago
On Tuesday 03 December 2002 19:03, scott_boag@us.ibm.com wrote:
> +1
>
> grchiu@ca.ibm.com wrote on 12/03/2002 05:09:16 PM:
> > Hi all,
> >
> > After lots of discussion regarding the conformance test restructuring,
> > there were many people in favour of option (2), and/or proposed variants
>
> of
>
> > option (2). We propose to move forward with Shane Curcuru's proposal for
> > restructuring the test harness.
> >
> > The goal of this discussion is to "agree on basic principles to get the
>
> job
>
> > done for existing committers to test Xalan releases, and let the
>
> committers
>
> > who will actually perform this work finish the details and implement a
> > working
> > system".
> >
> > - create an accept tree, parallel to the conf tree
> > - have in the accept tree only a few categories (including idkey, output,
> > sort, whitespace)
> > - have in these categories only the tests for which processors do not
>
> agree
>
> > on the output (see considerations below)
> > - pass an option to the test harness (perhaps "qetest.processor")
> > indicating which processor was used
> > - update the test harness to choose the correct gold file for the test
> > - an example of the naming scheme: (sort21 case)
> >
> > /accept/sort/sort21.xml
> > /accept/sort/sort21.xsl
> > XML and XSL pair for test
> >
> > /accept-gold/sort/sort21.out
> > "default" output file that output will be checked against
> >
> > /accept-gold/sort/sort21-XalanJ-C.out
> > should XalanJ-C ("XSLTC") have correct output that doesn't match
> > sort21.out, it would go here.
> > similar files may exist for XalanC or XalanJ-I ("Xalan").
> >
> > Considerations made in the proposal:
> >
> > - attempts will be made to reduce the number of test cases going into the
> > accept bucket. This will be done by making improvements to the test
>
> harness
>
> > in detecting these differences, and rewriting test cases to remain
> > processor independent where possible.
> > - attempts will be made to reduce the number of 'gold' files in the
> > accept-gold tree. By only creating processor-specific gold files when
> > necessary, duplication of files will be reduced.
> > - as the test files are tagged with every new Xalan release, it should be
> > easy to check out test files corresponding to a certain previous release.
> > - in the event of future multi-dimensionality, additional gold files can
>
> be
>
> > created, or directory structures can be duplicated.
> >
> > Please respond with any comments, objections, opinions you may have --
> >
> > Thanks!
> > Gordon
> >
> > ----
> > Gordon Chiu
> > grchiu@ca.ibm.com
Re: Last call: conf test restructuring
Posted by sc...@us.ibm.com.
+1
grchiu@ca.ibm.com wrote on 12/03/2002 05:09:16 PM:
> Hi all,
>
> After lots of discussion regarding the conformance test restructuring,
> there were many people in favour of option (2), and/or proposed variants
of
> option (2). We propose to move forward with Shane Curcuru's proposal for
> restructuring the test harness.
>
> The goal of this discussion is to "agree on basic principles to get the
job
> done for existing committers to test Xalan releases, and let the
committers
> who will actually perform this work finish the details and implement a
> working
> system".
>
> - create an accept tree, parallel to the conf tree
> - have in the accept tree only a few categories (including idkey, output,
> sort, whitespace)
> - have in these categories only the tests for which processors do not
agree
> on the output (see considerations below)
> - pass an option to the test harness (perhaps "qetest.processor")
> indicating which processor was used
> - update the test harness to choose the correct gold file for the test
> - an example of the naming scheme: (sort21 case)
>
> /accept/sort/sort21.xml
> /accept/sort/sort21.xsl
> XML and XSL pair for test
>
> /accept-gold/sort/sort21.out
> "default" output file that output will be checked against
>
> /accept-gold/sort/sort21-XalanJ-C.out
> should XalanJ-C ("XSLTC") have correct output that doesn't match
> sort21.out, it would go here.
> similar files may exist for XalanC or XalanJ-I ("Xalan").
>
> Considerations made in the proposal:
>
> - attempts will be made to reduce the number of test cases going into the
> accept bucket. This will be done by making improvements to the test
harness
> in detecting these differences, and rewriting test cases to remain
> processor independent where possible.
> - attempts will be made to reduce the number of 'gold' files in the
> accept-gold tree. By only creating processor-specific gold files when
> necessary, duplication of files will be reduced.
> - as the test files are tagged with every new Xalan release, it should be
> easy to check out test files corresponding to a certain previous release.
> - in the event of future multi-dimensionality, additional gold files can
be
> created, or directory structures can be duplicated.
>
> Please respond with any comments, objections, opinions you may have --
>
> Thanks!
> Gordon
>
> ----
> Gordon Chiu
> grchiu@ca.ibm.com
>