Servlet Unit Testing strategies

[Vincent Massol <vm...@octo.fr>]

Hi Tomcat developers,

I'd like to have your opinion on servlet unit testing. I have written a
simple extension to JUnit called J2EEUnit to do that but I'd like to knwo if
there is a general consensus on servlet unit testing, regarding it's
usefulness :

Do you think :

1) It is useless. Just need to put a good facade in front of your business
code and for the remaining part related to Servlets, functional tests are
fine (using a tool such as HttpUnit),

2) It is nice to be able to unit test the part of the code related to
servlets (i.e. the controller in a MVC model).It is possible to come up with
a tool to do that (ex: J2EEUnit)

3) With the current servlet API it is not possible to come up with a
framework that will let you properly do unit tests. It will have lots of
shortcomings. The only solution is to extend the Servlet API by our own
Servlet Test API. Then have Tomcat implement it. Once we have this, it will
then be possible to create a tool/framework that make use of this Test API
to unit test servlet. The next step will to promote this API so that it
becomes a standard and other Servlet engine implement it.

For point 3), are you aware of any project/discussion/JSR on the subject ?

Thanks a lot.
Vincent Massol
J2EEUnit author

Re: Servlet Unit Testing strategies

[cm...@yahoo.com]

> > But if you have to make setPropertiesFileName public, or to make design
> > choices for the component based on the testing framework requirements -
> > no, I don't think that's good. If the component has a
> > setPropertiesFileName it is because that's the intended behavior.
> >
> 
> How would you unit test it then ?

Unit testing works great in some cases, but not for everything. When you
need to change the interfaces and code so that it can be unit-tested - you
have a problem. 


> 1) Ok, let's take another sample : let's take almost any Apache code that
> deals with Servlet(Turbine, Struts, Jetspeed, ... ). You'll see that you
> have code logic that is mixed with Servlet objects. Now let's imagine that
> you want to unit test a method. You'll have to *modify* it to try to move as
> much of the logic away from the Servlet code to try to build a facade. So

I don't think "you have to unit test a method" is a good enough
argument. You have to modify your test procedure if unit testing doesn't
work for the code - not to modify the code ( there are enough other
reasons to modify the code anyway ). Of course, if the code is not
flexible enough or the API it provide doesn't solve the problems, or it's
too complex - it must be changed. 

> Hope I convinced you a bit more, even a slight bit more would make me happy
> .... :-)

You don't have to convince me - I am convinced on the importance of
testing the code, and more options the better. I'm just scared on the idea
to change the code because the unit-test framework can't deal with it. 

Costin

Re: Servlet Unit Testing strategies

[Vincent Massol <vm...@octo.fr>]

Interesting ... Let me try one more time ... :)

----- Original Message -----
From: <cm...@yahoo.com>
To: <to...@jakarta.apache.org>
Sent: Friday, February 23, 2001 10:43 PM
Subject: Re: Servlet Unit Testing strategies


>
> > 2) For any test framework that you use, you may have to modify your own
> > class to help testing it ... but this is not bad, this usually brings a
> > better design ... For example : let's say you have a class that uses a
> > Properties file to read some data. Let's imagine you want to test your
class
> > that reads it and retrieve some properties. If you want to test all the
> > cases, you have to provide several versions of this properties file :
some
> > with incorrect values, undefined properties, ... It means that even if
you
> > don't need it, you may have to add a setPropertiesFileName() to your
class
> > so that when you write your test class you can use the different test
> > properties file, ...
>
> But if you have to make setPropertiesFileName public, or to make design
> choices for the component based on the testing framework requirements -
> no, I don't think that's good. If the component has a
> setPropertiesFileName it is because that's the intended behavior.
>

How would you unit test it then ?

> Unit testing is fine if the framework can test the APIs, it's very bad if
> the APIs have to be changed so that the framework can test them. I don't
> think the goal is to create components that can be tested by a certain
> framework, but to create testing tools that can test the components we
> need.
>

1) Ok, let's take another sample : let's take almost any Apache code that
deals with Servlet(Turbine, Struts, Jetspeed, ... ). You'll see that you
have code logic that is mixed with Servlet objects. Now let's imagine that
you want to unit test a method. You'll have to *modify* it to try to move as
much of the logic away from the Servlet code to try to build a facade. So
you're changing your code for a testing purpose  .... :) However, it is a
good thing because it make think about your code design and refactor it.

2) I am pretty sure that the next great language that will come up (after
Java) will have features to test as much of the code as possible with the
minimum amount of work : programming by contract style, maybe a way to
specify sets of test environment values and expected results (as part of
your class) that will be exercised during compilation time, ... In that
sense, I would say that thinking about testing issues when designing a
container is probably a good thing !

>
> Costin
>
>

Hope I convinced you a bit more, even a slight bit more would make me happy
.... :-)

Vincent.

Re: Servlet Unit Testing strategies

[cm...@yahoo.com]

> 2) For any test framework that you use, you may have to modify your own
> class to help testing it ... but this is not bad, this usually brings a
> better design ... For example : let's say you have a class that uses a
> Properties file to read some data. Let's imagine you want to test your class
> that reads it and retrieve some properties. If you want to test all the
> cases, you have to provide several versions of this properties file : some
> with incorrect values, undefined properties, ... It means that even if you
> don't need it, you may have to add a setPropertiesFileName() to your class
> so that when you write your test class you can use the different test
> properties file, ...

But if you have to make setPropertiesFileName public, or to make design
choices for the component based on the testing framework requirements -
no, I don't think that's good. If the component has a
setPropertiesFileName it is because that's the intended behavior.

Unit testing is fine if the framework can test the APIs, it's very bad if
the APIs have to be changed so that the framework can test them. I don't
think the goal is to create components that can be tested by a certain
framework, but to create testing tools that can test the components we
need.


Costin

Re: Servlet Unit Testing strategies

[Vincent Massol <vm...@octo.fr>]

----- Original Message -----
From: <cm...@yahoo.com>
To: <to...@jakarta.apache.org>
Sent: Friday, February 23, 2001 8:10 PM
Subject: Re: Servlet Unit Testing strategies


> > Hi Tomcat developers,
> >
> > I'd like to have your opinion on servlet unit testing. I have written a
> > simple extension to JUnit called J2EEUnit to do that but I'd like to
knwo if
> > there is a general consensus on servlet unit testing, regarding it's
> > usefulness :
> >
> > Do you think :
> >
> > 1) It is useless. Just need to put a good facade in front of your
business
> > code and for the remaining part related to Servlets, functional tests
are
> > fine (using a tool such as HttpUnit),
>
> +0.5 I think functional tests ( using HttpUnit or GTest ) are extremely
> important for servlets ( while unit tests are important for beans and
> components used inside servlets ). That doesn't mean unit tests for
> servlets are useless - but that servlets/JSPs shouldn't have a lot of
> functionality, but delegate to beans/ejbs/taglibs that can be unit tested.

I agree, however there are some code, like Controllers, Reverse Proxy, ...
that contains some logic very linked to the Servlet API. Putting a facade
for these are quite cumversome because they manipulate a lot of Servlet
objects and writing a facade means a lot of marshalling / methods ... But I
agree that most of the code can be unit tested independently of a servlet
context ...

>
> Also, I think that for tomcat testing ( i.e. the container ) functional
> tests are more apropriate and efficient, and easier to write ( but doesn't
> mean we shouldn't have unit tests  for individual components ).
>
> > 2) It is nice to be able to unit test the part of the code related to
> > servlets (i.e. the controller in a MVC model).It is possible to come up
with
> > a tool to do that (ex: J2EEUnit)
>
> I'm not sure I understand this - but any test is nice as long as it's easy
> enough to write.
>

It simply means that within the framework you can separately call methods
that require access to valid HttpServletRequest, HttpServletResponse,
HttpSession, ServletConfig, ... objects

>
> > 3) With the current servlet API it is not possible to come up with a
> > framework that will let you properly do unit tests. It will have lots of
> > shortcomings. The only solution is to extend the Servlet API by our own
> > Servlet Test API. Then have Tomcat implement it. Once we have this, it
will
> > then be possible to create a tool/framework that make use of this Test
API
> > to unit test servlet. The next step will to promote this API so that it
> > becomes a standard and other Servlet engine implement it.
>
> -1. Extending the servlet API or implementing any extension in tomcat is
> not an option, and a test framework that requires changes in the design or
> implementation of the tested application is _bad_.
>
> It doesn't matter how good the test framework is, if you have to change
> your code to use it it's not good enough.
>
> ( IMHO )
>

2 points here :

1) I did not mean changing the Servlet API ... I meant, make it easier for a
testing framework to be able to unit test servlets/Filters/Taglib/... by
offering a helper API. For example, the JVMPI offers an API that ease
writing profiling tools but it does not change the Java language. I meant
more a helper framework on the container side, not on the servlet API side.

2) For any test framework that you use, you may have to modify your own
class to help testing it ... but this is not bad, this usually brings a
better design ... For example : let's say you have a class that uses a
Properties file to read some data. Let's imagine you want to test your class
that reads it and retrieve some properties. If you want to test all the
cases, you have to provide several versions of this properties file : some
with incorrect values, undefined properties, ... It means that even if you
don't need it, you may have to add a setPropertiesFileName() to your class
so that when you write your test class you can use the different test
properties file, ...

> Costin
>
>

Thanks a lot for taking the time to answer.
Vincent.

Re: Servlet Unit Testing strategies

[cm...@yahoo.com]

> Hi Tomcat developers,
> 
> I'd like to have your opinion on servlet unit testing. I have written a
> simple extension to JUnit called J2EEUnit to do that but I'd like to knwo if
> there is a general consensus on servlet unit testing, regarding it's
> usefulness :
> 
> Do you think :
> 
> 1) It is useless. Just need to put a good facade in front of your business
> code and for the remaining part related to Servlets, functional tests are
> fine (using a tool such as HttpUnit),

+0.5 I think functional tests ( using HttpUnit or GTest ) are extremely
important for servlets ( while unit tests are important for beans and
components used inside servlets ). That doesn't mean unit tests for
servlets are useless - but that servlets/JSPs shouldn't have a lot of
functionality, but delegate to beans/ejbs/taglibs that can be unit tested.

Also, I think that for tomcat testing ( i.e. the container ) functional
tests are more apropriate and efficient, and easier to write ( but doesn't
mean we shouldn't have unit tests  for individual components ).

> 2) It is nice to be able to unit test the part of the code related to
> servlets (i.e. the controller in a MVC model).It is possible to come up with
> a tool to do that (ex: J2EEUnit)

I'm not sure I understand this - but any test is nice as long as it's easy
enough to write.


> 3) With the current servlet API it is not possible to come up with a
> framework that will let you properly do unit tests. It will have lots of
> shortcomings. The only solution is to extend the Servlet API by our own
> Servlet Test API. Then have Tomcat implement it. Once we have this, it will
> then be possible to create a tool/framework that make use of this Test API
> to unit test servlet. The next step will to promote this API so that it
> becomes a standard and other Servlet engine implement it.

-1. Extending the servlet API or implementing any extension in tomcat is
not an option, and a test framework that requires changes in the design or
implementation of the tested application is _bad_. 

It doesn't matter how good the test framework is, if you have to change
your code to use it it's not good enough.

( IMHO )

Costin