You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Emmanuel Bourg <eb...@apache.org> on 2009/05/30 13:43:45 UTC

[all] Testing interface implementations

In Commons CLI there are 3 parsers implementing the same interface, and 
soon another implementation will be added. There is an abstract test 
case with the test methods, and a concrete subclass for each parser. The 
concrete test case instantiates the parser and disables some tests by 
overwriting them if they are known to fail.

The abstract test is growing and becomes difficult to manage. I may 
split it into smaller classes grouping the tests by category, but I 
don't want to have 4 subclasses for every abstract class. This is where 
multi inheritance would be useful.

Anyone knows a trick or a new test frameworks that may help testing the 
implementations of an interface?

Emmanuel Bourg

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [all] Testing interface implementations

Posted by John Bollinger <th...@yahoo.com>.


Emmanuel Bourg <eb...@apache.org> wrote:
> In Commons CLI there are 3 parsers implementing the same interface, and soon 
> another implementation will be added. There is an abstract test case with the 
> test methods, and a concrete subclass for each parser. The concrete test case 
> instantiates the parser and disables some tests by overwriting them if they are 
> known to fail.
> 
> The abstract test is growing and becomes difficult to manage. I may split it 
> into smaller classes grouping the tests by category, but I don't want to have 4 
> subclasses for every abstract class. This is where multi inheritance would be 
> useful.
> 
> Anyone knows a trick or a new test frameworks that may help testing the 
> implementations of an interface?

The part about "The concrete test case [...] disables some tests [...] if they are
known to fail" seems a bit dodgy to me from a design perspective.  I offer as
evidence the very problem you describe.

I have not looked at the test code, but your plan to split up the abstract test
class is a good opportunity to solve this design problem.  Here is what I
would try to do (assuming JUnit 3.x; otherwise YMMV):

1) Attempt to split the base class so that each resulting test class is either
100% go or 100% no-go for each parser.  I would think that adopting Phil's
"semantic integrity" idea ought to do the job*, though I don't see a need for
the test classes to form a hierarchy**.

* If the known failures that are hidden by the current scheme are not directly
related to semantic considerations then why can't the tests be rewritten
so that all the implementations pass?

2) Instead of creating subclasses for (test class, implementation) pairs,
create a parser factory for each parser implementation, and give each test
class a list of the parser factories it should test.  IOW, use composition
instead of inheritance.  This isn't all that different from what you have
already, where the most important function of each concrete test class is
to function as a parser factory.

3) In implementing (2), build dynamic TestSuites in order to provide a
separate test for each (test method, parser implementation) pair.  This
requires a static suite() method and puts much more burden on it than
in most test classes, but that is the price you pay for splitting up the
tests without either reducing test granularity or creating n * m
subclasses**.

Here's a schematic example:

public class CommonParserTests /* no need to extend TestCase */ {

    private final ParserFactory factory;

    public CommonParserTests(ParserFactory factory) {
        this.factory = factory;
    }

    public void testDontHaveACow() {
        CommandLineParser parser = factory.createParser();

        CommandLine line = parser.parse(new Options(),
                new String[] {"Don't", "have", "a", "cow,", "man!"});

        // assertXXX...
    }

    // additional test methods as needed...

    public static Test suite() {
        TestSuite suite = new TestSuite();

        for (ParserFactory factory : new ParserFactory[] {
                new BasicParser(), new GnuParser(), new PosixParser()}) {
            CommonParserTests tests = new CommonParserTests(factory);

            suite.addTest(new TestCase("Don't have a cow, man!") {
                protected void runTest() {
                    tests.testDontHaveACow();
                }
            });

            // additional inner TestCases, one for each test method
        }

        return suite;
    }
}


** Phil's suggestion of arranging the abstract test classes as a
hierarchy provides a simpler way of avoiding n * m concrete subclasses,
PROVIDED THAT there are no tests that crosscut part of the hierarchy.
That is, tests that want to appear in at least two, but not all, of the
abstract test classes don't fit well in an hierarchical scheme.


Regards,

John


      


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [all] Testing interface implementations

Posted by Emmanuel Bourg <eb...@apache.org>.
Phil Steitz a écrit :

> You have probably thought of this and it may be distasteful, but one way 
> to break things apart would be to break the single abstract test class 
> into an artificial hierarchy.  If there is a way to do this with some 
> kind of semantic integrity (e.g., if it can be made to coincide with the 
> "known to fail" relation)  that could be a simple solution.  

Yes that's probably the only solution. Thank you for the input.

Emmanuel Bourg



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: [all] Testing interface implementations

Posted by Phil Steitz <ph...@gmail.com>.
Emmanuel Bourg wrote:
> In Commons CLI there are 3 parsers implementing the same interface, 
> and soon another implementation will be added. There is an abstract 
> test case with the test methods, and a concrete subclass for each 
> parser. The concrete test case instantiates the parser and disables 
> some tests by overwriting them if they are known to fail.
>
> The abstract test is growing and becomes difficult to manage. I may 
> split it into smaller classes grouping the tests by category, but I 
> don't want to have 4 subclasses for every abstract class. This is 
> where multi inheritance would be useful.
>
> Anyone knows a trick or a new test frameworks that may help testing 
> the implementations of an interface?  
You have probably thought of this and it may be distasteful, but one way 
to break things apart would be to break the single abstract test class 
into an artificial hierarchy.  If there is a way to do this with some 
kind of semantic integrity (e.g., if it can be made to coincide with the 
"known to fail" relation)  that could be a simple solution.  The 
[collections] test framework uses this approach, though the inheritance 
hierarchy among the test classes generally corresponds to specialization 
in the implementation classes.

Phil
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org