You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Ryan Schmidt <su...@ryandesign.com> on 2005/12/05 21:04:09 UTC
OT: Regression tests
On Dec 5, 2005, at 13:22, Duncan Murdoch wrote:
> On 12/5/2005 12:08 PM, Scott Palmer wrote:
>
>> On 5-Dec-05, at 11:21 AM, Duncan Murdoch wrote:
>>
>>> You need to have the person fixing the bug add a regression
>>> test, and a policy that nobody commits code that fails the
>>> regression tests. This prevents the scenario you talk about,
>>> and also the one where someone comes along later, after the lock
>>> has been released, and reintroduces the bug.
>>
>> It sounds like a good idea, until you try it is "real life" where
>> the regression test code is 100x more complex than the feature it
>> is testing. I've been wanting to try test driven development for
>> years and I always come up against that roadblock. Maybe it's
>> easier for form-based web apps or something.
>> Sorry, off-topic. Feel free to start a new topic if there is
>> some hope of clueing me in to the secret :)
>
> I think it depends a lot on the application. I'm working on a
> language interpreter (www.r-project.org); there it is very easy to
> put together tests, they're just code snippets in the interpreted
> language. So much so that I wouldn't call it "test driven
> development", as much as "test supported development". It's easy
> to write tests and run them, so regression errors are pretty uncommon.
I'd also really like to understand how to actually use such tests to
support my development. I certainly understand that if I have a
function multiply() which takes two arguments and multiplies them
that I can write a test which feeds the function 6 and 7 and checks
to make sure the answer that pops out is 42. I'm not writing
functions like that, though, and I don't know anybody who does. I'm
part of a team writing, among other things, a CRM system for a
customer doing business in 16 countries. Perhaps the bug of the day
is that the daily currency conversion rate importer didn't work. Or
perhaps when we turn on our SQL debugging functions, then the search
results are wrong. Or perhaps the column headers in Italian are too
long so they wrap to the next line and mess up the layout of the
generated PDFs. These are all things that have actually happened in
this project, and I'm unclear how I would write a testing system that
would have prevented any of it.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: OT: Regression tests
Posted by Eric S <ej...@americanlowlife.com>.
Ryan Schmidt wrote:
> I'd also really like to understand how to actually use such tests to
> support my development. I certainly understand that if I have a
> function multiply() which takes two arguments and multiplies them
> that I can write a test which feeds the function 6 and 7 and checks
> to make sure the answer that pops out is 42. I'm not writing
> functions like that, though, and I don't know anybody who does. I'm
> part of a team writing, among other things, a CRM system for a
> customer doing business in 16 countries. Perhaps the bug of the day
> is that the daily currency conversion rate importer didn't work. Or
> perhaps when we turn on our SQL debugging functions, then the search
> results are wrong. Or perhaps the column headers in Italian are too
> long so they wrap to the next line and mess up the layout of the
> generated PDFs. These are all things that have actually happened in
> this project, and I'm unclear how I would write a testing system that
> would have prevented any of it.
Warning: I'm not an evangelist or even well-read on the topic, but I
can relate it to how I've used it when I've had the chance.
One key thing is that if you design the software with testing in mind,
you tend to break down and modularize the software to a higher degree,
so a single function/method/whatever tends to be shorter and have fewer
branches in it, breaking the complexity out into separate
functions/methods/whatevers. You then have separate tests for each of
the fragments broken out.
In the end, rather than trying to test a single function for every
conceivable input you might pass it, you just need to make sure that it
can handle any given type of return from the lower levels and that all
the paths through itself get tested. You then have separate tests for
the lower level routines. This tends to reduce the overall number of
tests you need to run and makes it easier to come up with the relevant
test cases.
Yes, there are times when test cases are hard to design due to the
nature of the return (making sure that xhtml/css displays properly in IE
and FF, for example), and if it's the kind of software that makes
determining all the failure cases difficult, then it won't add as much
either. However, ultimately, all computer programs break down to a
series of steps that when looked at in small enough groups, are quite
predictable as to what a failure condition is and what might cause one.
I don't think unit testing is something that works miracles, but it
certainly helps in the cases I've seen it used. The only project I've
wanted to use it for that it wouldn't help was because the design of the
system was so poor that you couldn't test anything without involving
most of the system.
Anyway, this is OT so I'll shut up now.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: OT: Regression tests
Posted by Duncan Murdoch <su...@murdoch-sutherland.com>.
On 12/5/2005 4:04 PM, Ryan Schmidt wrote:
> On Dec 5, 2005, at 13:22, Duncan Murdoch wrote:
>
>> On 12/5/2005 12:08 PM, Scott Palmer wrote:
>>
>>> On 5-Dec-05, at 11:21 AM, Duncan Murdoch wrote:
>>>
>>>> You need to have the person fixing the bug add a regression
>>>> test, and a policy that nobody commits code that fails the
>>>> regression tests. This prevents the scenario you talk about,
>>>> and also the one where someone comes along later, after the lock
>>>> has been released, and reintroduces the bug.
>>>
>>> It sounds like a good idea, until you try it is "real life" where
>>> the regression test code is 100x more complex than the feature it
>>> is testing. I've been wanting to try test driven development for
>>> years and I always come up against that roadblock. Maybe it's
>>> easier for form-based web apps or something.
>>> Sorry, off-topic. Feel free to start a new topic if there is
>>> some hope of clueing me in to the secret :)
>>
>> I think it depends a lot on the application. I'm working on a
>> language interpreter (www.r-project.org); there it is very easy to
>> put together tests, they're just code snippets in the interpreted
>> language. So much so that I wouldn't call it "test driven
>> development", as much as "test supported development". It's easy
>> to write tests and run them, so regression errors are pretty uncommon.
>
> I'd also really like to understand how to actually use such tests to
> support my development. I certainly understand that if I have a
> function multiply() which takes two arguments and multiplies them
> that I can write a test which feeds the function 6 and 7 and checks
> to make sure the answer that pops out is 42. I'm not writing
> functions like that, though, and I don't know anybody who does. I'm
> part of a team writing, among other things, a CRM system for a
> customer doing business in 16 countries. Perhaps the bug of the day
> is that the daily currency conversion rate importer didn't work. Or
> perhaps when we turn on our SQL debugging functions, then the search
> results are wrong. Or perhaps the column headers in Italian are too
> long so they wrap to the next line and mess up the layout of the
> generated PDFs. These are all things that have actually happened in
> this project, and I'm unclear how I would write a testing system that
> would have prevented any of it.
Regression tests won't prevent those the first time, but they'll prevent
re-occurrences, or similar problems. For example:
You write a test to import a currency conversion rate, check it works.
You write a test to turn on SQL debugging, check that a particular
answer is unchanged.
You measure the length of the italian column headers.
I'd say the first two would likely be useful, the last one not (because
there are lots of languages, and lots of places where the length
matters). But if the solution to the last one was to change your
formatting routines rather than to retranslate the header, maybe just
checking the Italian would be a good enough test.
The first two may never expose a regression error, but you'll eventually
have hundreds of tests like this, and in aggregate they'll actually be
pretty useful at preventing regressions. They're also of some use in
documenting boundary cases: e.g. if your program crashed when someone
entered a price of 0.0, then the regression test will document what you
think should really happen in that case.
Duncan Murdoch
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Regression tests
Posted by Gavin Lambert <ga...@compacsort.com>.
Quoth Ryan Schmidt <ma...@ryandesign.com>:
> I'd also really like to understand how to actually use such tests to
> support my development. I certainly understand that if I have a
> function multiply() which takes two arguments and multiplies them
> that I can write a test which feeds the function 6 and 7 and checks
> to make sure the answer that pops out is 42. I'm not writing
> functions like that, though, and I don't know anybody who does. I'm
> part of a team writing, among other things, a CRM system for a
> customer doing business in 16 countries. Perhaps the bug of the day
> is that the daily currency conversion rate importer didn't work. Or
> perhaps when we turn on our SQL debugging functions, then the search
> results are wrong. Or perhaps the column headers in Italian are too
> long so they wrap to the next line and mess up the layout of the
> generated PDFs. These are all things that have actually happened in
> this project, and I'm unclear how I would write a testing
> system that would have prevented any of it.
The main thing you need to do to make life easier is to reduce coupling
in your code. Make sure each class only has one responsibility, and use
abstract interface patterns to disconnect classes, so that you can "mock
out" other classes and test each class in isolation (in addition to
testing interaction between classes).
Also make sure that each method only has one responsibility, and keep
them short. The general rule of thumb is that if it doesn't fit on a
single screen, then it's too long (and probably trying to do too much)
and ought to be refactored. This not only helps code readability, but
also gives you smaller pieces to test.
For example, for your daily currency conversion rate importer, you could
have one class that's solely responsible for downloading the file from
wherever it is kept, and a different class that then parses that into
some meaningful objects, and a third class that schedules the other two
on a daily basis. That way you can test the parsing entirely divorced
from the downloading, and vice versa.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org