Re: Should patches also update functional tests?

[Jack Klebanoff <kl...@Mutagen.Net>]

A good test suite is invaluable. It is very important that it be kept 
working.  Then when one makes a code change that causes a test to fail 
you can be fairly sure that your change broke something. You can fix it 
before the change is submitted, keeping Derby's quality up. If the test 
suite is weak or if it shows failures then Derby's quality will 
deteriorate. It will be very hard to get it back up.

I think that we should never accept a patch that breaks a test. If the 
test is in error then the patch should include a fix for the test.

We should strongly encourage all patches to include tests. A committer 
should think twice, or three times, before committing a patch without tests.

Jack

> Satheesh Bandaram wrote:
> I would like to seek suggestions or comments regarding submitting 
> patches with appropriate test changes. Since Derby now has functional 
> tests, should we suggest or even require patches to be submitted along 
> with a test case added? This will only apply if a test case is 
> available and doesn't require undue amount of time and resources to 
> test the patch.
>
> Obvious advantages are:
>
>    1. We can be sure we are indeed addressing the problem. Unless 
> committers write a test case based on the code changed, we may not 
> know for sure we did fix the problem correctly.
>    2. Can prevent future regressions in the code, where the same 
> defect is reintroduced because of another patch or enhancement.
>    3. Strengthens Derby funtional test suites and test coverage.
>
> This may introduce additional burden on patch submissions, hence, we 
> could make this voluntary to start with. How do other Apache projects 
> handle this?
>
> Satheesh

Re: Should patches also update functional tests?

[Daniel John Debrunner <dj...@debrunners.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I would like to add that if you submit a patch it would be very useful
for the committers if you stated which (if any) tests suites on which
jvm(s) you ran (with no failures :-). Obviously you can submit a patch
without running tests, but it may deley the commit of that patch, until
a committer has run what they deem to be sufficient tests.

As for test cases with patches, I would strongly encourage the addition
of test cases, typically added to existing tests. This will get easier
once IBM has contributed the remaining tests for Cloudscape/Derby. I
think also a committer has the right to ask a patch submitter to
demonstrate that a patch does indeed solve the problem, if it is not
obvious from the fix. The easiest way to do this would be a test case.

Dan.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFBpLNeIv0S4qsbfuQRAgZKAKCU0fEMPJbUYwwGiDeCg6Hx5+3ybgCg0L8y
NGtCrVjXlOKOLhtsVOGZ5NQ=
=HN9P
-----END PGP SIGNATURE-----