You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xalan.apache.org by cu...@apache.org on 2001/09/05 20:55:06 UTC
cvs commit: xml-xalan/test/java/xdocs/sources/tests getstarted.xml overview.xml run.xml submit.xml
curcuru 01/09/05 11:55:06
Modified: test/java/xdocs/sources/tests getstarted.xml overview.xml
run.xml submit.xml
Log:
Documentation updates; mostly caught up to new methodologies with better descriptions
Revision Changes Path
1.5 +14 -13 xml-xalan/test/java/xdocs/sources/tests/getstarted.xml
Index: getstarted.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/getstarted.xml,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- getstarted.xml 2001/08/21 19:58:31 1.4
+++ getstarted.xml 2001/09/05 18:55:06 1.5
@@ -57,7 +57,8 @@
the normal conf tests, but only on the axes subdir, and using the TraxSaxWrapper class.
<br/><code>build api -DtestClass=TransformStateTest -Dapi.loggingLevel=30 </code> This runs
the org.apache.qetest.xalanj2.TransformStateTest with a lower loggingLevel (so less is output).
- Note that testClass is one of the few properties that is not prefixed.
+ Note that testClass is one of the few properties that is not prefixed, since it is
+ not passed on to the test itself, but is only used by the Ant script to load the test.
</p>
</s2>
@@ -110,26 +111,26 @@
for a simple alternate way to set your classpath using JARDIR. This allows QE/QA/test
people to run the same set of tests quickly against different versions of the product.</p>
<note>(Aug-01 Section needs updating: JARDIR docs -sc)</note>
- <p>ALTERNATE: Set an environment variable JARDIR to point to directory where you have all applicable jars.
- Delete testxsl.jar if present there.
+ <p>ALTERNATE way using JARDIR:</p>
+ <p>1: Set an environment variable JARDIR to point to directory where you have all applicable jars.
+ <br/>Delete testxsl.jar if already present there (if you want to rebuild it).
<br/>Required: at least xalan.jar and bsf.jar from a Xalan-J build;
xerces.jar from the <ref>same</ref> Xalan-J build (or your parser's JAXP-compatible jar instead of xerces.jar);
- and js.jar (see <jump href="http://xml.apache.org/xalan-j/extensions.html#supported-lang">Xalan-J doc for information on js.jar</jump>)
+ and js.jar (see <jump href="http://xml.apache.org/xalan-j/extensions.html#supported-lang">Xalan-J doc for information on js.jar, only needed for extensions tests</jump>)
<br/><code>set JARDIR=e:\builds\myjars </code>
</p>
- <p>3: Copy all product jars to the JARDIR</p>
- <p>4: cd to the test directory and cleanup the tests.</p>
+ <p>2: Copy needed product jars to the JARDIR</p>
+ <p>3: cd to the test directory and cleanup the tests</p>
<p><code>cd e:\builds\xml-xalan\test </code>
- <br/><code>build.bat clean </code>
+ <br/><code>build clean </code>
</p>
- <p>5: Build the tests appropriate to your product.
- <br/>Xalan-J 1.x:
- <br/><code>build.bat package.xalan </code>
- <br/>Xalan-J 2.x:
- <br/><code>build.bat package.xalan2 </code> (<ref>or package.trax</ref>)
+ <p>4: Build the tests
+ <br/><code>build jar </code>
</p>
<p>This will create <code>build\testxsl.jar </code>, which you should manually copy
- into JARDIR before you <link idref="run" anchor="how-to-run">run any tests</link>.</p>
+ into JARDIR before you <link idref="run" anchor="how-to-run">run any tests</link>.
+ Note that you might instead want to <code>build all</code> if you want to also
+ run any of the bugzilla or extensions tests as well.</p>
<note>The use of JARDIR is not required; you're free to manage the classpath
yourself, or the test build file will assume it should use the Xalan-J 2.0
directories by default. I've found that simply putting all the needed jars in
1.6 +31 -7 xml-xalan/test/java/xdocs/sources/tests/overview.xml
Index: overview.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/overview.xml,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- overview.xml 2001/08/21 19:58:31 1.5
+++ overview.xml 2001/09/05 18:55:06 1.6
@@ -4,6 +4,7 @@
<s1 title="Overview">
<ul>
<li><link anchor="purpose">Purpose of these tests</link></li>
+<li><link anchor="brief-overview">Brief overview of testing methods</link></li>
<li><link anchor="dir-map">Directory Structure</link></li>
<li><link anchor="test-map">Listing of Java tests and drivers</link></li>
<li><link anchor="credits">Credits for the tests</link></li>
@@ -18,9 +19,23 @@
the tests before and after a code change. That way you can at least ensure that
your changes didn't cause any regressions to the code before you check your
changes in. In the future, we hope to provide a tool to compare test results from
- one run to another to make this process even simpler.</p>
+ one run to another (without necessarily having to re-run the test) to make this process even simpler.</p>
</s2>
+ <anchor name="brief-overview"/>
+ <s2 title="Brief overview of testing methods">
+ <p>The Xalan tests include a richly featured and fully automated testing framework
+ that the great majority of the tests use. The org.apache.qetest package provides
+ an independent testing and automation harness for Xalan, including logging and
+ reporting mechanisims.</p>
+ <p>Nearly all tests are automated, run without any user interaction and with a minimum amount of setup, and produce
+ a rolled-up report of their pass/fail/other status. Our existing testing library includes
+ a wide array of tests, from XSLT conformance to detailed API tests, and welcomes
+ user-submitted tests as well. The 'smoketest' target (a subset of the most important
+ tests) are also integrated into the <jump href="http://jakarta.apache.org/builds/gump">GUMP nightly build system</jump>, and thus ensure a
+ minimum baseline of functionality on a daily basis.</p>
+ </s2>
+
<anchor name="dir-map"/>
<s2 title="Directory Structure">
<gloss>
@@ -30,7 +45,8 @@
<label></label>
<label><code>xml-xalan/test/bin</code></label>
<item>Java test automation dependencies (includes
- <jump href="http://jakarta.apache.org/ant/index.html">Ant 1.2</jump>)</item>
+ <jump href="http://jakarta.apache.org/ant/index.html">Ant 1.2</jump>
+ so that you don't have to install it yourself)</item>
<label><code>xml-xalan/test/java/src</code></label>
<item>Java test automation source tree - this includes
a generic testing framework as well as specific API tests for parts of Xalan
@@ -69,13 +85,15 @@
<anchor name="test-map"/>
<s2 title="Listing of Java tests and drivers">
+ <note>This section is a sort of catalog of existing tests. Beginning users
+ will probably want to see <link idref="run" anchor="how-to-run">how to run tests</link> as well.</note>
<p>Java Test Drivers (data driven testing</p>
<p>A Java Test Driver executes a test for each xml/xsl file pair in
the specified directory tree or each pair in the specified fileList.
For each test, the driver iterates over the tree or list of files
and asks a Testlet to execute a test on each one. This is also similar to
data driven testing, where a common algorithim is defined for a test case, and
-then a large number of data points are run through the test case in order.
+then a large number of data points (in this case, the xml/xsl file pairs) are run through the test case in order.
The best example is <jump href="apidocs/org/apache/qetest/xsl/StylesheetTestletDriver.html">StylesheetTestletDriver</jump></p>
<p>The Test Drivers rely on various Testlet implementations
to define the actual testing algorithim to apply to each xml.xsl
@@ -92,11 +110,15 @@
using DOM trees, SAX events, or input/output streams.</p>
<p>The three levels of iteration, test algorithim, and
processor flavor are all independently changeable, so we can
-easily try out different kinds of tests.</p>
+easily try out different kinds of tests. This technique is used to
+run the full sets of 'conf' (conformance), 'perf' (performance),
+extensions, and 'contrib' (user contributed stylesheets) tests.</p>
<gloss>
<label>org.apache.qetest.xsl.<link idref="run" anchor="how-to-run-c">CConformanceTest</link></label>
-<item>essentially the same as ConformanceTest, but for Xalan-C. I plan to
-make this use the Testlet model soon to simplify this model, and
+<item>This is similar to the StylesheetTestlet, but for Xalan-C.
+It has a different method of discovering test files, and simply shells
+out to a command prompt to run each stylesheet through the TestXSLT.exe program.
+I plan to make this use the Testlet model soon to simplify this model, and
to provide better support for differing command lines.</item>
</gloss>
@@ -142,7 +164,9 @@
<label>TestThreads.java</label>
<item>MANUALLY executed test for running multiple threads
-and transforming multiple stylesheets simultaneously.</item>
+and transforming multiple stylesheets simultaneously. An updated and automated
+test is now available, org.apache.qetest.xsl.ThreadedTestletDriver, which
+should be used instead. </item>
</gloss>
<p>All in subpackages of: org.apache.qetest.trax</p>
1.6 +22 -25 xml-xalan/test/java/xdocs/sources/tests/run.xml
Index: run.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/run.xml,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- run.xml 2001/08/21 19:58:31 1.5
+++ run.xml 2001/09/05 18:55:06 1.6
@@ -13,8 +13,8 @@
<s2 title="How-to: Run tests">
<p>Nearly all tests for Xalan are independent Java classes built
into testxsl.jar that
- can be run either standalone on the command line, programmatically
- from your application, or from
+ can be run either individually on the command line, programmatically
+ from your application or our handy Ant build.xml file, or in batches from
<jump href="apidocs/org/apache/qetest/xsl/XSLTestHarness.html">XSLTestHarness</jump>.
There really isn't any magic to them: you can just set your classpath and
execute java.exe to run them; some Tests and Testlets currently provide defaults
@@ -35,21 +35,15 @@
<br/>(runs TRAX interface tests with Xalan-J 2.x<br/>
</p>
<p>Alternately: some convenience batch files are provided for the most common
- tests - these simply turn around and call build.bat/.sh for you.<br/></p>
+ tests - these simply turn around and call build.bat/.sh for you.<br/>
+ (Namely conf.bat, perf.bat, and the like)</p>
<p>Alternately: Run java.exe and the desired test class yourself:<br/></p>
- <p>Note: the above batch and shell files, and the JARDIR environment variable,
- are simply conveniences for running the tests. The Java API tests and test drivers can
- also be executed by hand, for those who wish to manage their own classpaths and/or
- simply pass all arguments to the tests on the command line, etc.</p>
- <p>Sorry! We don't have .sh equivalents for the convenience .bat files -
- submissions of ports of these files are welcomed! We do plan to add an Ant
- task that can run Xalan Tests and Testlets, so in the future both building
- and running tests will mostly be done from an Ant buildfile, which is
- cross-platform by default.</p>
<note>Running tests with alternate JAXP parsers: all org.apache.qetest.trax.*
tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser, like
crimson.jar. Be sure to manually set the appropriate system properties to use
- your parser instead of xerces.jar, which is the default for Xalan-J.</note>
+ your parser instead of xerces.jar, which is the default for Xalan-J. Tests will
+ also run on any JAXP 1.1 compatible xslt processor, namely either our default
+ xalan one or our new xsltc one.</note>
</s2>
<anchor name="how-to-view-results"/>
@@ -58,12 +52,13 @@
writing a full set of output results to their <code><link anchor="test-options-logfile">logFile</link></code>, as
set from a .properties file or the command line. Generally the
output results file is easier to deal with. The basic format is
- fairly simple and you can certainly read it as-is.</p>
+ fairly simple and you can certainly read it as-is. Also, many tests
+ send only summary results to the console, but full output to the results file.</p>
<p>To 'pretty-print' results or produce reports, please use the
viewResults.xsl stylesheet and associated batch/shell files, like so:<br/>
<code>cd \xml-xalan\test</code><br/><br/>
<code>build conf -DoptionName=optionValue <link anchor="test-options-logfile">-Dqetest.logFile results/MyResults.xml</link></code><br/><br/>
- <code>viewResults.bat results/MyResults.xml results\MyPrettyResults.html [options]</code><br/><br/>
+ <code>viewResults.bat results/MyResults.xml results/MyPrettyResults.html [options]</code><br/><br/>
These options are passed to Xalan's command line to transform the
xml formatted output results into a nicer-looking html file. The most
common option would be <code>-param loggingLevel 99</code> to get
@@ -84,14 +79,14 @@
<p>To see all options, call a test with an illegal argument to force it
to print out it's .usage(). You may mix setting options from a properties
file and from the command line; command line options will take precedence.</p>
- <p>For another description of options, see <br/><code>xml-xalan\test\conf.properties</code>,<br/>
- which describes most of them as used in the context of running the StylesheetTestletDriver
- over the xml-xalan\tests\conf tree of stylesheets. Simply change inputDir and goldDir
- to run over a different set of files (like a performance test suite, which we hope to
- add soon).</p>
- <note>Path-like options set in a properties file must use the local system's
- File.separator character, and backslashes \ must be escaped \\. The checked in
- copy of conf.properties is for a Windows platform.</note>
+ <p>For another description of options, see <br/><code>xml-xalan\test\test.properties</code>,<br/>
+ which describes most of them with comments. Remember that the prefixes
+ 'qetest.', 'conf.' etc. are used by the Ant build.xml file to manage which
+ properties are used for different kinds of tests, and are ripped off before
+ being passed to the Java test script itself. Thus qetest.loggingLevel=99 in
+ the test.properties file becomes just loggingLevel of 99 when passed to the test.</p>
+ <note>Path-like options set in a properties file generally should use
+ forward slashes (legal in URL's), even on Windows platforms.</note>
<p>Quick list of options</p>
<anchor name="test-options-logfile"/>
<gloss>
@@ -103,8 +98,6 @@
<item>determines how much information is sent just to the default ConsoleLogger:
since often you won't be watching the console as the test is running, you can set this
lower than your loggingLevel to speed the tests up a little</item>
- <label>-load <ref>file.properties</ref></label>
- <item>(read in a .properties file, that can set any/all of the other opts)</item>
<label>-inputDir <ref>path/to/tests</ref></label>
<item>path to a directory tree of input *.xml/*.xsl files, using your system's separator</item>
<label>-outputDir <ref>path/to/output/area</ref></label>
@@ -121,6 +114,10 @@
<jump href="apidocs/org/apache/qetest/xslwrapper/ProcessorWrapper.html">ProcessorWrapper.java</jump> </item>
<label>-testlet <ref>TestletClassname</ref></label>
<item>For StylesheetTestletDriver, use a different class for the testing algorithim</item>
+ <label>-load <ref>file.properties</ref></label>
+ <item>(read in a .properties file, that can set any/all of the other opts)
+ This option is automatically used by the Ant build.xml file, so you normally
+ would not specify it yourself.</item>
<label>-debug</label>
<item>prints extra test debugging info</item>
</gloss>
1.3 +5 -3 xml-xalan/test/java/xdocs/sources/tests/submit.xml
Index: submit.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/submit.xml,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- submit.xml 2000/12/06 20:51:58 1.2
+++ submit.xml 2001/09/05 18:55:06 1.3
@@ -35,13 +35,11 @@
changes to the package name, etc.</p>
<p>You can pretty-print the results by using the <link idref="run" anchor="how-to-view-results">viewResults.xsl stylesheet</link> to turn
the XML into an HTML format.</p>
-<p>The same file can be used with minor modifications for either Xalan-J 1.x
-or Xalan-J 2.x API tests, with proper package and import modifications.</p>
</s2>
<anchor name="write-xsl-tests"/>
<s2 title="How to Write Stylesheet Tests">
- <p>Test cases in the upcoming "conf" group will ultimately be submitted to OASIS as part of
+ <p>Test cases in the "conf" group will ultimately be submitted to OASIS as part of
their vendor-independent project on XSLT/XPath Conformance Testing. For more
information about this project, visit
<jump href="http://www.oasis-open.org/committees/xslt/index.html">http://www.oasis-open.org/committees/xslt/index.html</jump></p>
@@ -65,5 +63,9 @@
after the test is submitted. We hope to have a template for contributing
stylesheet tests available soon that will closely match the eventual OASIS format.</p>
+<p>The Xalan team will continue to provide test automation, like the
+StylesheetTestletDriver and various StylesheetTestlet classes, that
+enables a user to easily run large suites of tests with fully rolled-up
+reporting and a rich set of options.</p>
</s2>
</s1>
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-cvs-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-cvs-help@xml.apache.org