Why isn't Surefire more easier and more sensible to use?

[Ben Hutchison <be...@ibsglobalweb.com>]

Ive recently started using Surefire as part of a Maven project.

I was really surprised how inconvenient it was to use, and how things 
about it seemed, well... stupid.

1. When running a suite of tests with "surefire:test 
<http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html>", 
the main thing you want to see a summary of all failed tests. It is THE 
most important and normal thing to look at. I was surprised to discover 
Surefire doesnt produce this essential item by default. Instead, it 
creates many individual reports, one per test, whether they passed or 
failed.

2. Then, one has to run a second plugin and special goal just to get the 
summary, which is what any normal person is going to be looking for. It 
puts the summary in a different directory from the individual files. And 
it doesnt produce this report as part of the normal build.

3. It generates the XML intermediate files that the report needs. It 
also generates identical txt files, for no apparent reason. And 
bizarrely, you can turn the XML off, but you cant turn the txt off.

And then you get weird situations where because the tests fail, the 
build fails, and this actually stops the report generation which will 
tell you what test failed :(
http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216

The whole thing seems like a classic case of programmer-centric, not 
user-centric, thinking. This tool doesn't seem to have started with what 
a typical user would want and then worked backwards to meet this goal.

I dont particularly enjoy criticizing some programmer's unpaid 
contribution, but Surefire is a widely used piece of infrastructure, and 
it should work better than it does.

-Ben


-- 

	

*Ben Hutchison
Senior Developer
* Level 2 476 St Kilda Road Melbourne VIC 3004
T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 | 
www.ibsglobalweb.com <http://www.ibsglobalweb.com/>