Checking in tests that fail

[Benson Margulies <bi...@basistech.com>]

Do you have a policy for checking in tests that demonstrate a defect?
When diagnosing a problem, one can sometimes end up building up a
significant piece of code that will (with any luck) be a regression test
later. The same person who makes the test may not make the eventual fix.
It might make sense to check these in but skip them in the default
build.

Re: Checking in tests that fail

[Daniel Kulp <dk...@apache.org>]

On Tuesday 04 September 2007, Benson Margulies wrote:
> Do you have a policy for checking in tests that demonstrate a defect?
> When diagnosing a problem, one can sometimes end up building up a
> significant piece of code that will (with any luck) be a regression
> test later. The same person who makes the test may not make the
> eventual fix. It might make sense to check these in but skip them in
> the default build.

That's fine.   There are two options for these types of things:

1) Assert the failure in the test and put a comment that explains it - 
thus when it's fixed, the test will need to be updated to no-longer 
assert the failure.

2) @Ignore the test, but put a description in the @Ignore to explain why 
it's ignored.   Currently, the description is not used by the maven 
tools, but hopefully a new version of surefire will report the comments.

In both cases, make sure there is a JIRA logged and put the JIRA ID in 
the comment.


-- 
J. Daniel Kulp
Principal Engineer
IONA
P: 781-902-8727    C: 508-380-7194
daniel.kulp@iona.com
http://www.dankulp.com/blog