You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Alfred Nathaniel <a....@bluewin.ch> on 2005/03/09 00:11:59 UTC
[RT] Who may follow a retired Anteater?
Extensive automated functional testing is the best way to keep up
software quality. There is general consensus on that, and everybody is
doing it - right?
The industry standard for testing Java classes is JUnit. When it comes
to testing web request/response conversations, the choice is less
obvious. There are various tools which vary in maturity, usability, and
feature set.
In Cocoon there is currently a small suite of Anteater tests. Anteater
was introduced to Cocoon by Ovidiu Predescu who was both co-author of
Anteater and the driving force in the early days of Cocoon flow.
Meanwhile Ovidiu has moved on to other priorities. While Cocoon flow
was enthusiastically carried on by others, for Anteater it was a fatal
blow. Since Sep 2003 the Anteater project is officially retired [1],
and up to now no one came forward to take the project to new shores.
Although Anteater is in a usable state, its todo list [2] is far from
being empty. It is time to think about alternatives, in order not to
tie a strategic effort such as automated testing to a dead product.
[1] http://sourceforge.net/forum/forum.php?forum_id=311245
[2] http://aft.sourceforge.net/todo.html
PRODUCT SURVEY
All Anteater and alike do is to allow sending an HTTP request and
checking the response for the expected result. Differences then may
show up in
a) the effort for writing a new test from scratch,
b) the language used for writing the tests,
c) the coverage of HTTP nitty-gritties,
d) the possible test conditions in the result checking,
e) the flexibility to base a second request on the response of the frist
Here I do a short survey of existing Open Source tools. All information
is taken only from browsing around the project websites. Don't nail me
if I misunderstood something or if the doc is not in sync with the code.
ANTEATER - http://aft.sourceforge.net
Anteater with its tight Ant integration allowing to formulate requests
and checks as Ant tasks makes it easy to write new tests. HTTP coverage
is not complete, however. For example, https, cookies, and file upload
is not supported.
Response checking is fairly complete. Web conversations are possible
but limited by simple property assignments from response checking.
LATKA - http://jakarta.apache.org/commons/latka/
Latka is very similar to Anteater, both in the functionality and in the
fact that the community is dead. So it is not really an alternative.
HTTPUNIT - http://www.httpunit.org
HttpUnit is a package of Java classes to write web conversations as
JUnit tests. Therefore tests have to be written in Java but the package
provides convenient methods to do so.
HttpUnit is still alive (last release Oct 2004) although not very active
ATM. But it is basically a one-man show and it uses the MIT license.
HTMLUNIT - http://htmlunit.sourceforge.net/
HtmlUnit is similar to HttpUnit but concentrates more on dealing with
HTML documents. Handling the HTTP nitty-gritty is left to
commons-httpclient.
The community is small but seems healthy. Last release was in Jan 2005.
CANOO WEBTEST - http://webtest.canoo.com/
Canoo WebTest is similar to Anteater by wrapping all request/response
actions into Ant tasks. WebTest, however, has a slimmer code base since
it uses HtmlUnit as underlying implementation (after recently replacing
HttpUnit!).
Canoo WebTest is an Open Source project backed by a Swiss IT company.
The community is larger than just company employees and seems to be
healthy and active. The license terms are home-made but sound like
old-style BSD including the advertising statement.
CONCLUSION
Anteater is dead, and we should get away from it before it starts
smelling. Canoo WebTest seems to be an attractive alternative for
preserving the Anteater paradigm of tight Ant integration.
Does anybody have practical experience with Canoo WebTest which speaks
for/against this product? Or any other product? Or is nobody testing?
Looking forward to your feedback,
Alfred.
Re: [RT] Who may follow a retired Anteater?
Posted by Reinhard Poetz <re...@apache.org>.
Alfred Nathaniel wrote:
> Anteater is dead, and we should get away from it before it starts
> smelling. Canoo WebTest seems to be an attractive alternative for
> preserving the Anteater paradigm of tight Ant integration.
>
> Does anybody have practical experience with Canoo WebTest which speaks
> for/against this product? Or any other product? Or is nobody testing?
>
> Looking forward to your feedback,
A few days ago a came across "Silenium"
(http://paulhammant.com/blog/selenium.html) which could be worth having a look
at it too.
I don't have practical experiences with any of the mentioned tools, but heard
good things about Canoo WebTest and Paul's blog is encouraging.
What I would really love to see is a GUI (Eclipse plug-in) to write the tests,
at least the most parts of them. And this GUI should describe the tests in some
well know programming language or in XML. AFAIK, any of the named alternatives
provides such a tool.
Last thought, it would be great to have an extensive testsuite, but as always,
somebody has to work on this.
Alfred, I can't help you with the actual work, but if you work on this and if
you want you can get my feedback on your plans and your work and I would also
take care that your tests don't stay too long in Bugzilla ;-)
--
Reinhard Pötz Independant Consultant, Trainer & (IT)-Coach
{Software Engineering, Open Source, Web Applications, Apache Cocoon}
web(log): http://www.poetz.cc
--------------------------------------------------------------------
Re: [RT] Who may follow a retired Anteater?
Posted by Bertrand Delacretaz <bd...@apache.org>.
Le 21 mars 05, à 15:59, Alfred Nathaniel a écrit :
> ...I have now rewritten all existing Anteater tests using HtmlUnit....
Cool, looking forward to it!
-Bertrand
Re: [RT] Who may follow a retired Anteater?
Posted by Alfred Nathaniel <a....@bluewin.ch>.
On Thu, 2005-03-10 at 13:28, Luca Morandini wrote:
> Ugo Cei wrote:
> >
> > I used WebTest for a while, but in the end I found that writing tests in
> > XML is way too verbose and clumsy.
>
> I had the same experience... later on I moved to HTMLUnit ([1]), which
> offers a lot:
> 1) Different browsers emulation.
> 2) Retrieving of XML pages as DOM Documents (org.w3c.dom.Document) for
> further processing by Java.
> 3) Being based on JUnit, the "green bar" works in Eclipse for HTMLUnit
> as well.
> 4) Calling of Javascript functions embedded in the HTLM page from the
> Java test driver.
> 5) Availability of Jelly ([2]) as scripting language (well, I opted for
> Java, so I cannot comment on this feature).
>
> But, best of all, it lets you re-create the whole browser environment
> within your Java test driver, which, IMO, is better suited for
> functional testing than the request/response paradigm of HTTPUnit.
>
> I was sceptical about using Java for testing, but the Java/JUnit
> environment in Eclipse gave me the "quick" feeling of a scripting language.
Thanks for all your comments. I can only fully concur with these
statements.
I had a go at Canoo Webtest for replacing the Cocoon Anteater tests but
shelfed it again quickly. Webtest is very elaborate for testing HTML
forms but has no provisions for low-level stuff such as HTTP header
checking.
Then I tried HtmlUnit, and that looks much better. With a few
convenience methods in a common base class, simple test cases can be
written in Java just as concise as in Anteater. So I also don't see
anymore the added value of writing test in XML.
I have now rewritten all existing Anteater tests using HtmlUnit. The
last to be done are the WebDAV tests where HtmlUnit falls short for only
supporting GET and POST requests. There I'll have to fallback to
calling httpclient directly.
The most difficult part of the exercise was the integration into the
build process. I set it up that HtmlUnit is optional and has to be
installed separately (similar to Anteater). If it flies, it should
later be integrated, at the expense of a couple of additional jars.
I need a bit more time to wrap it all up, and then I'll submit it a
patch with the whole stuff.
NB the HtmlUnit version of the flowscriptReload test runs without
problems whereas the Anteater version shows intermittent failures.
Cheers, Alfred.
Re: [RT] Who may follow a retired Anteater?
Posted by Luca Morandini <lu...@tin.it>.
Ugo Cei wrote:
> Il giorno 09/mar/05, alle 08:01, Bertrand Delacretaz ha scritto:
>
>> Le 9 mars 05, à 00:11, Alfred Nathaniel a écrit :
>>
>>> ...Does anybody have practical experience with Canoo WebTest which
>>> speaks
>>> for/against this product? Or any other product? Or is nobody
>>> testing?..
>
> I used WebTest for a while, but in the end I found that writing tests in
> XML is way too verbose and clumsy.
I had the same experience... later on I moved to HTMLUnit ([1]), which
offers a lot:
1) Different browsers emulation.
2) Retrieving of XML pages as DOM Documents (org.w3c.dom.Document) for
further processing by Java.
3) Being based on JUnit, the "green bar" works in Eclipse for HTMLUnit
as well.
4) Calling of Javascript functions embedded in the HTLM page from the
Java test driver.
5) Availability of Jelly ([2]) as scripting language (well, I opted for
Java, so I cannot comment on this feature).
But, best of all, it lets you re-create the whole browser environment
within your Java test driver, which, IMO, is better suited for
functional testing than the request/response paradigm of HTTPUnit.
I was sceptical about using Java for testing, but the Java/JUnit
environment in Eclipse gave me the "quick" feeling of a scripting language.
Regards,
[1] http://htmlunit.sourceforge.net/
[2] http://jakarta.apache.org/commons/jelly/
--------------------
Luca Morandini
www.lucamorandini.it
--------------------
Re: [RT] Who may follow a retired Anteater?
Posted by Ugo Cei <ug...@apache.org>.
Il giorno 09/mar/05, alle 08:01, Bertrand Delacretaz ha scritto:
> Le 9 mars 05, à 00:11, Alfred Nathaniel a écrit :
>> ...Does anybody have practical experience with Canoo WebTest which
>> speaks
>> for/against this product? Or any other product? Or is nobody
>> testing?..
I used WebTest for a while, but in the end I found that writing tests
in XML is way too verbose and clumsy. Also, keeping a lot of tests up
to date with respect to a changing Web UI was too much of a pain.
>
> Although I do not see an urgent need to move away from anteater, I've
> been using HttpUnit successfully in other projects and it does a good
> job. For HTML pages at least, but I'm not sure about its support for
> XML responses, which we want to test as well.
>
> But - writing "page" tests (http/xml or http/html) in compiled code is
> a pain IMHO, a scripting environment (ideally without angle brackets)
> is much more convenient, to shorten the test writing cycle.
>
> I can't help thinking that using rhino scripts to drive HttpUnit (if
> it does XML and XPath checks) or something similar would be cool...
>
> -Bertrand
Have you seen Selenium? [1] Looks interesting, but I haven't had the
time to take it for a test ride, yet.
Ugo
[1] http://agylen.com/blojsom/blog/devel/?permalink=Selenium.txt
--
Ugo Cei - http://agylen.com/blojsom/blog/
Re: [RT] Who may follow a retired Anteater?
Posted by Bertrand Delacretaz <bd...@apache.org>.
Le 9 mars 05, à 00:11, Alfred Nathaniel a écrit :
> ...Does anybody have practical experience with Canoo WebTest which
> speaks
> for/against this product? Or any other product? Or is nobody
> testing?..
Although I do not see an urgent need to move away from anteater, I've
been using HttpUnit successfully in other projects and it does a good
job. For HTML pages at least, but I'm not sure about its support for
XML responses, which we want to test as well.
But - writing "page" tests (http/xml or http/html) in compiled code is
a pain IMHO, a scripting environment (ideally without angle brackets)
is much more convenient, to shorten the test writing cycle.
I can't help thinking that using rhino scripts to drive HttpUnit (if it
does XML and XPath checks) or something similar would be cool...
-Bertrand