RE: Testing without reflection

[Vincent Massol <vm...@pivolis.com>]

Hi Thomas,

Thank you for your proposal to include JPillar in Cactus. I have read
the introduction to JPillar and I think it is a bit too far from Cactus
to be integrated at this point in time. I would suggest to put JPillar
on SourceForge, work on the Cactus integration if it makes sense and if
there's interest we can then see how we can bundle them together. To be
honest, I don't clearly see how it would benefit either Cactus or
JPillar to be integrated at this point in time. I'm also not working on
the J2ME platform and thus have zero knowledge on it...

Thanks a lot
-Vincent

> -----Original Message-----
> From: Thomas Hawtin [mailto:thawtin@tackline.demon.co.uk]
> Sent: 15 March 2003 17:04
> To: cactus-user@jakarta.apache.org
> Subject: Testing without reflection
> 
> A month ago I was looking to write some tests to run under MIDP. MIDP
> doesn't have reflection, and even if it did it wouldn't be a good idea
> to use it. Similarly reflection is not available for tests run in
> applets and servers with security set. To get around it is fiddly. So
> having nothing better to do, I wrote a little tool to generate static
> code to replace the reflection.
> 
> I'm not sure where to go with it, or even if it is really that useful.
I
> think probably it is best becoming part of something else, for
instance
> Cactus. Opinions?
> 
> I've put the text of the introduction below. The source is at:
> 
>     http://www.tackline.demon.co.uk/java/jpillar/jpillar.zip
> 
> Tom Hawtin
> 
> 
> 
> 
> JPillar
> 
> JPillar permits JUnit-style test cases to be run in environments with
> security enabled or where the reflection API is not available. It
> becomes possible to easily run tests on production configured web
> servers, browsers and mobiles. Note that this is very much a
> pre-pre-alpha and only something I'm doing for 'fun' - it even lacks
its
> own tests!
> 
> This product includes software developed by the Apache Software
> Foundation (http://www.apache.org/). Specifically the files
> ant/uk/co/demon/tackline/jpillar/JPillarTask.java and
> compiler/org/apache/bcel/ClassRepository.java are based upon files
from
> Ant 1.5.1 and BCEL 5.0 respectively. See those files for more
> information and full copyright.
> How it works
> 
> The JPillar tool compiles byte code to run test case methods without
> using reflection. Reflection can be a problem because some J2ME
> platforms do not support it and the security policies of some
> applications do not permit it. Using JPillar tests can be run in
> situations closer to their deployed environment and without coding
> overhead. Both J2MEUnit and JUnit are supported. Other styles should
be
> relatively easy to add (see the Framework interface).
> 
> For each class that matches the relevant frameworks prerequists for a
> test, a class is created to run the tests. For instance, the standard
> JUnit interpretation matches any class ending in "Test". These are
then
> treated as if they extend TestCase (there's no particular reason why
> they should extend or implement any particular type) and a test
instance
> for each test method is produced. The generated class has a suffix of
> "TestSuite". A new instance is a test that tests all of the test
methods.
> 
> Unfortunately the JUnit runners have code that throws secuirty
> exceptions. Worse the code is very fond of statics and wont load due
to
> a static initialiser failing. At the very least in order to use it,
> change junit.runner.BaseTestRunner.getPreferecnes such that the
secuirty
> exception from readPreferences does not propagate.
> 
>      try {
>          readPreferences();
>      } catch (java.lang.SecurityException exc) {
>          System.err.println("Not permitted to read preferences
file.");
>      }
> 
> Writing tests
> 
> Note: the exact configuration is subject to change. For instance I
might
> introduce marker types for test cases. See Framework implementations
to
> see the current specifications.
> JUnit test
> 
> Existing tests should run as usual. New JPillar test cases can be
> simplified.
> 
> Firstly, you do not need to extend TestCase - the generated class
> implements Test. Having said that extending Assert (TestCase's
> superclass) will usually be necessary. Typical exceptions to the rule
> are: using Java 1.5 (use: import static junit.framework.Assert;) or
have
> your own assert class.
> 
> Secondly there is no need for main or suite methods. These are handled
> by the generated class. In fact there is no need to go through hoops
to
> invoke the suite method as instantiating the class through the no-args
> constructor does the same.
> 
> There is a slight caveat in that setUp and tearDown if present need to
> be accessible at package level or empty (this is checked). Also you
will
> need a no-args constructor (probably the default).
> J2MEUnit test
> 
> Existing tests should run as usual. New JPillar test cases can be
> simplified. The simplification is more extreme than JUnit as the
> J2MEUnit tests start off more complicated.
> 
> As JUnit you do not need to extend TestCase, or have the main and
suite
> methods. In J2MEUnit the suite method is high maintenance. Even more
so
> is the runTest method which can also be dispensed with.
> 
> The same caveats setUp, tearDown and a no-arg constructor applies as
> JUnit.
> Running the JPillar compiler
> Command Line Interface
> 
> The compiler can be run from the command line with the syntax:
> java -d dir [-framework (junit | j2meunit)] -classpath path [-run]
> [-verify] classfiles ...
> 
> -d dir - Sets the directory to write files. Mandatory.
> 
> -framework (junit | j2meunit) - Determines which framework to generate
> code for. junit is the default.
> 
> -classpath path - Is the entire classpath necessary for the tests. It
> should include your classes, junit.jar and rt.jar (or j2meunit and
> midpapi.zip). Mandatory.
> 
> -run - Runs the generated code.
> 
> -verify - Verifies the generated code.
> ANT task
> 
> The task requires to be defined before it can be run
> 
>      <taskdef name="jpillar"
>           classname="uk.co.demon.tackline.jpillar.JPillarTask"
>           classpath="${jpillar}/jpillartask.jar"
>      />
> 
> The task is similar to javac, requiring srcdir, destdir and classpath.
> The framework can also be set to either "junit" or "j2meunit" with the
> former as the default. For instance:
> 
>       <jpillar framework="junit"
>           srcdir="${classes}"
>           destdir="${testclasses}"
>           classpath="${classes}:%{junit}:${rtjar}">
>        <patternset includes="**/*.class"/>
>      </jpillar>
> 
> As a servlet filter
> 
> To make it easier to run tests in applets a SuiteFilter generates
> classes on the fly. Not sure a filter is the best approach, as it
can't
> obviously be extended to generate a TestPackage class (I suppose a
> servlet for generating directory listings would plug the gap).
> Running JPillar classes
>  From the JPillar compiler
> 
> Just add -run to the command line. The tests are run in a class loader
> context separate from the compiler, but within the same process.
> 
> Command Line Interface
> 
> Simply run the generated class. There is no need for you to provide
your
> own main method in your test case. The runner used is textui, but can
> easily be changed (see the Framework class).
> java -classpath . pkg.MyTestSuite
> Through a JUnit runner
> 
> The generated class provides a suite method, so again can be used as
if
> it was a classic JUnit TestCase. For instance:
> java -classpath junit.jar junit.textui.TestRunner pkg.MyTestSuite
> As an applet
> 
> Two applets are provided. Both require the slightly modified JUnit
(see
> above). AppletRunner (see etc/webapp/appletrunner.html) runs the test
> with the textui and dumps the results to a text area. JAppletRunner
(see
> etc/webapp/jappletrunner.html) runs the test with the swingui. Note
the
> swingui has reduced functionality and because of its somewhat
excentric
> approach to multithreading, do not resize the appletviewer window
while
> it is starting. An applet parameter "class" determines which class is
run.
>  From a servlet
> 
> The WebRunner servlet runs the test through the (modified) JUnit
textui,
> returning the results in the page. Use a URL such as:
>
http://localhost:8080/jpillar/webrun?class=uk.co.demon.tackline.jpillar.
ru
> ntime.junit.SampleTestSuite
> Through a Midlet
> 
> The classes supplied with J2MEUnit make reference to a non-MIDP/CLDC
> class, so you'll need to recompile them (JPillar build.xml does that).
> Also the WTK 2.0 beta I have doesn't like J2MEUnit for some reason,
> although 1.04 is fine. Unfortunately MIDP 1.0 only supports per JAD
user
> attributes, not per Midlet as MIDP 2.0.
> uk.co.demon.tackline.jpillar.runtime.j2meunit.AttributeRunner,
although
> rough and ready, will run a comma separated list of TestCase class
> names. Note my build.xml breaks lines in the JAD which need to be
fixed
> (I shouldn't be using the manifest task).
> 
> Other ways
> 
> An EJB runner should be trivial. Applets and indeed midlets ought to
be
> able to POST their results back. Should only be a small change to
Cactus
> (or a filter), to make it run generated versions of the tests on the
> server side (whilst retaining reflection mechanism for beginXxx/endXxx
> methods on the client. Really ought to be using the Ant
> JUnit/JUnitReport tasks and Cactus mechanisms for creating reports.
> Easing TestCase main without JPillar
> 
> It is possible to avoid writing a custom main, even without the use of
> JPillar. You still need a main, but crucially it does not need to
refer
> to the name of the class itself. Use the main method public static
void
> main(String[] args) { main(new Error()); } and extend the class below.
> You can add this to whatever base test case class you use for your
> project. This code is 1.4 or later only, but you can see how JUnit
reads
> a stack trace in order to do it for earlier versions.
> 
> public abstract class TestCaseMain extends junit.framework.TestCase {
>      protected static void main(Throwable traceable) {
>         try {
>             junit.textui.TestRunner.run(Class.forName(
>                     traceable.getStackTrace()[0].getClassName()
>             ));
>             System.exit(0);
>         } catch (Throwable exc) {
>             exc.printStackTrace();
>             System.exit(1);
>         }
>      }
> }
> 
> And Bob becomes your uncle. For the perverts out there, use a static
> initialiser, temporarily redirect System.err, and start a thread that
> re-enables System.err and runs the test case (this will happen after
the
> rest of the static inialisation of your class).
> Building
> 
> In order to perform a full build JPillar, you require the following
> 
>      * Apache Ant - to perform the build and to build the Ant task. It
> is also used by the command line compiler (although I guess those
> classes will move across to Commons IO?).
>      * Jakarta BCEL (Byte Code Engineering Library) - basic
requirement.
> Note I have added a class (ClassRepository) in the supplied source.
>      * JUnit - for the JUnit specific runtime parts. To make the small
> modification to enable it to run with security, you will also need the
> source.
>      * J2ME - for the J2MEUnit specific runtime parts.
>      * The Servlet API - for instance tomcat.
> 
> I fully admit that the build.xml is a complete mess. But it's the sort
> of excrutiatingly dull task I hate.
> Stuff to do/think about
> Subclassing versus delegation
> 
> I have decided to use delegation rather than subclassing for the
> relationship between the generated class and the target test case.
There
> is a disadvantage in that protected methods in superclasses from
> different packages become inaccessible. To guard against this being a
> stealth issue, JPillar checks that if the test case setUp/tearDown
> methods are inaccessible, then they should only contain the single
> instruction RETURN.
> 
> My main reason for choosing delegation is that it just feels better
the
> subtyping. An instance of the generated class is used as a collection
of
> instances that map onto a single test method. This would be unpleasant
> if the generated class was a subclass of the test case class.
Delegation
> also allows a decoupling between the framework used by the test case,
> and that used by the runner. If you like, you can also make the
> generated class extend, say, your own Midlet.
> Miscellaneous
> 
>      * Tests!
>      * Cactus integration
>      * Nicer and more runners
>      * Run tests on different system to presentation (Cactus style).
>      * User configurable Framework
>      * Sort out build.xml
>      * Web JUnit test collector - so swingui, for instance, can give a
> choice of tests to run.
>      * Better documentation!
>      * Mutable security manager with asserts for use of permissions.
>      * Static java.lang.reflect.Proxy replacement
>      * Generate source and debugging for generated classes
>      * Create TestPackage/AllTests classes.
> 
> End note
> 
> I only wrote this because I was writing a Go midlet (with server
stuff)
> and didn't like the smell of J2ME unit and playing with BCEL looked
fun.
> I haven't approached this project with my professional standards, and
> have got pissed off with fiddling about with random pieces of
software.
> For instance, it would make vastly more sense to have written a
> prototype using in-class-loader-context reflection to generate Java
> source files. Would a tidied version be of much use to anyone? I can't
> really answer that. Perhaps integrated as part of say, Cactus, the
cost
> of entry would be sufficiently low.
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cactus-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: cactus-user-help@jakarta.apache.org