Re: Integration vs acceptance tests [was (MNG-591) Integration tests lifecycle]

[John Fallows <jo...@gmail.com>]

On 7/25/05, Rafal Krzewski <Ra...@caltha.pl> wrote:
> John Fallows wrote:
> 
> > Is it always considered a build failure if integration tests do not pass 100%?
> 
> I did you mean acceptance tests?

I'm glad you raised this, Rafal.  I agree with your description below.  

My main point is to question the need for separation in source
directory structure between unit tests and integration tests, if they
are both required to pass at 100%, as you have explained.

Further, it would seem to make sense to separate out acceptance tests
into their own source directory so that they could easily be isolated,
and therefore not be required to pass at 100% success.

> In my understanding integration tests should pass 100% each time they
> are run. The difference between unit and integration tests is only in
> granularity. Unit tests focus on each component (one or few classes) in
> separation, and integration tests focus on many components wired
> together into a larger artifact - a GUI or web application for example.
> 
> Acceptance tests were (I believe) first introduced in eXtreme
> Programming methodology. For each milestone (iteration) of application
> development a number of features is scheduled, and a tests that verify
> if the application provides the feature are created. At the beginning of
> an iteration all of it's acceptance tests are expected to fail, and at
> the end all are expected to pass - any failing tests indicate features
> that must be rescheduled for further iterations. After the iteration is
> completed the acceptance tests become regression tests that guard
> developed features from being broken further down the road.
> 
> An acceptance test could be both an unit test or integration test,
> depending on the type application. Usually the customer cares about
> features that are accessible from GUI or web front end and therefore
> acceptance tests will be integration tests, but when developing an
> utility library unit tests could be valid acceptance tests.
> 
> As I understand it, acceptance test has the following features:
> 
> 1) it expresses a contract item between the customer and the developer,
> and therefore it should have an associated bugtracker item / XP story
> card id for reference.
> 2) belongs to a specific milestone - should have a version associated
> with it.
> 3) it's failure should not cause a SNAPSHOT build to fail.
> 4) it's failure should cause a release build to fail iff release version
> = tests milestone.
> 
> I think acceptance tests support would be an interesting enhancement to
> maven testing framework.
> Probably the simplest way of dealing with it would be adding a JavaDoc
> annotation to the test methods, and using XDoclet to extract the
> information:
> 
> /**
>   * Test adding a book to the catalogue.
>   * @acceptance 1.0 LIBRARY-9
>   */
> public void testAddBook() {
>      // ...
> }
> 
> The information could be used to display an useful report combined with
> versions information from the POM: total/completed features per version,
> and as dashboard data source.
> 
> Rafal
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
> 
> 

Kind Regards,
John Fallows.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org