You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Ryan Bloom <rb...@covalent.net> on 2002/07/08 16:05:58 UTC
New Test Suite
Okay, I have looked at two different test suite options.
1) Check -- GPL licensed. Doesn't work on Windows.
http://check.sourceforge.net/
2) CuTest -- Zlib license. Doesn't currently work on Unix, but I
have patches already. I'll be submitting them later today.
http://cutest.sourceforge.net/
CuTest is two files, one .c and one .h, so it is very small and it
should be able to do the job. Plus, we can continue to improve it if we
have to. The maintainer is currently using CuTest for his job, so it is
still under active maintenance. (There is one bug that I have found, it
is a difference in the way that Windows and Unix treat vsnprintf and
strlen. I am looking into fixing it).
Check is a lot more than two files, and the port to Windows could be
kind of tough. The problem is that it requires fork() for a lot of what
it does, and although I haven't looked at the code in great depth, the
parts I have looked at just look very Unix based. It is under active
maintenance, but very slowly (According to the original author).
My personal preference is to avoid the GPL all together and go with the
simple CuTest suite. Any thoughts?
Ryan
----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA
RE: New Test Suite
Posted by Ryan Bloom <rb...@covalent.net>.
I'll look at this tonight.
Ryan
----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA
> -----Original Message-----
> From: Sander Striker [mailto:striker@apache.org]
> Sent: Tuesday, July 09, 2002 9:29 AM
> To: sussman@collab.net
> Cc: rbb@covalent.net; dev@apr.apache.org
> Subject: RE: New Test Suite
>
> > From: sussman@collab.net [mailto:sussman@collab.net]
> > Sent: 09 July 2002 16:47
>
> > "Sander Striker" <st...@apache.org> writes:
> >
> > > Why don't you take a look at the test framework that was cooked up
> > > in subversion?
> > >
> > > http://svn.collab.net/repos/svn/trunk/subversion/tests/
> > >
> > > It is currently only used for subversion, but I think you will be
able
> > > to leverage that code with little effort. Also, on the plus side,
it
> > > is APR based [back to eating our own dogfood ;]
> >
> > How is it APR based? I don't follow you, Sander. It seems that one
> > could test anything at all with our system.
>
> Whoa, you're right. I quickly glanced through (still haven't written
> any tests) and saw the code in svn_tests_main.c, but:
>
> /*
> * tests-main.c: shared main() & friends for SVN test-suite programs
> *
>
> /me blushes
>
> So, Ryan, please take a closer look at it, since it might just do what
> you are looking for.
>
> Sander
RE: New Test Suite
Posted by Sander Striker <st...@apache.org>.
> From: sussman@collab.net [mailto:sussman@collab.net]
> Sent: 09 July 2002 16:47
> "Sander Striker" <st...@apache.org> writes:
>
> > Why don't you take a look at the test framework that was cooked up
> > in subversion?
> >
> > http://svn.collab.net/repos/svn/trunk/subversion/tests/
> >
> > It is currently only used for subversion, but I think you will be able
> > to leverage that code with little effort. Also, on the plus side, it
> > is APR based [back to eating our own dogfood ;]
>
> How is it APR based? I don't follow you, Sander. It seems that one
> could test anything at all with our system.
Whoa, you're right. I quickly glanced through (still haven't written
any tests) and saw the code in svn_tests_main.c, but:
/*
* tests-main.c: shared main() & friends for SVN test-suite programs
*
/me blushes
So, Ryan, please take a closer look at it, since it might just do what
you are looking for.
Sander
Re: New Test Suite
Posted by Ben Collins-Sussman <su...@collab.net>.
"Sander Striker" <st...@apache.org> writes:
> Why don't you take a look at the test framework that was cooked up
> in subversion?
>
> http://svn.collab.net/repos/svn/trunk/subversion/tests/
>
> It is currently only used for subversion, but I think you will be able
> to leverage that code with little effort. Also, on the plus side, it
> is APR based [back to eating our own dogfood ;]
How is it APR based? I don't follow you, Sander. It seems that one
could test anything at all with our system.
For testing library interfaces, we have a simple C framework that runs
standalone, self-verifying tests. For example:
$ ./skel-test list
skel-test 1: parse implicit-length atoms
skel-test 2: parse explicit-length atoms
skel-test 3: parse invalid atoms
skel-test 4: parse lists
skel-test 5: unparse implicit-length atoms
skel-test 6: unparse lists
$ ./skel-test
PASS: skel-test 1: parse implicit-length atoms
PASS: skel-test 2: parse explicit-length atoms
PASS: skel-test 3: parse invalid atoms
PASS: skel-test 4: parse lists
PASS: skel-test 5: unparse implicit-length atoms
PASS: skel-test 6: unparse lists
For testing the svn binary as a black-box user application, we have a
similar framework written in python:
$ ./basic_tests.py list
Test # Test Description
------ ----------------
1 basic checkout of a wc
2 basic status command
3 basic commit command
4 basic update command
5 basic corruption detection
6 receiving text merges as part of an update
7 basic conflict creation and resolution
8 basic cleanup command
9 basic revert command
10 basic switch command
11 basic delete command
$ ./basic_tests.py
PASS: basic_tests.py 1: basic checkout of a wc
PASS: basic_tests.py 2: basic status command
PASS: basic_tests.py 3: basic commit command
PASS: basic_tests.py 4: basic update command
...
RE: New Test Suite
Posted by Ryan Bloom <rb...@covalent.net>.
> > Okay, I have looked at two different test suite options.
> >
> > 1) Check -- GPL licensed. Doesn't work on Windows.
> > http://check.sourceforge.net/
> >
> > 2) CuTest -- Zlib license. Doesn't currently work on Unix, but I
> > have patches already. I'll be submitting them later today.
> > http://cutest.sourceforge.net/
>
> Why don't you take a look at the test framework that was cooked up
> in subversion?
>
> http://svn.collab.net/repos/svn/trunk/subversion/tests/
>
> It is currently only used for subversion, but I think you will be able
> to leverage that code with little effort. Also, on the plus side, it
> is APR based [back to eating our own dogfood ;]
This is actually the one time that we can't eat our own dog food.
Remember that the whole point of the test suite is to prove that you
have implemented APR correctly. If we implement the test suite on top
of APR, then you need to have an APR implementation before you can test
it. That just doesn't work. The test suite needs to be so generic that
I can begin to test the file part of APR before I ever touched the
process/thread part.
Ryan
RE: New Test Suite
Posted by Sander Striker <st...@apache.org>.
> From: Ryan Bloom [mailto:rbb@covalent.net]
> Sent: 08 July 2002 16:06
> Okay, I have looked at two different test suite options.
>
> 1) Check -- GPL licensed. Doesn't work on Windows.
> http://check.sourceforge.net/
>
> 2) CuTest -- Zlib license. Doesn't currently work on Unix, but I
> have patches already. I'll be submitting them later today.
> http://cutest.sourceforge.net/
Why don't you take a look at the test framework that was cooked up
in subversion?
http://svn.collab.net/repos/svn/trunk/subversion/tests/
It is currently only used for subversion, but I think you will be able
to leverage that code with little effort. Also, on the plus side, it
is APR based [back to eating our own dogfood ;]
[...]
> My personal preference is to avoid the GPL all together and go with the
> simple CuTest suite. Any thoughts?
+1 on avoiding the GPL.
> Ryan
Sander
Re: New Test Suite
Posted by Pier Fumagalli <pi...@betaversion.org>.
Ryan Bloom <rb...@covalent.net> wrote:
> Check is a lot more than two files, and the port to Windows could be
> kind of tough. The problem is that it requires fork() for a lot of what
> it does, and although I haven't looked at the code in great depth, the
> parts I have looked at just look very Unix based.
It looks like they'd need to abstract it on top of APR! ????? :) :)
> My personal preference is to avoid the GPL all together and go with the
> simple CuTest suite. Any thoughts?
I'd like to avoid GPL all together...
Pier
--
[Perl] combines all the worst aspects of C and Lisp: a billion of different
sublanguages in one monolithic executable. It combines the power of C with
the readability of PostScript. [Jamie Zawinski - DNA Lounge - San Francisco]
RE: New Test Suite
Posted by Ryan Bloom <rb...@covalent.net>.
I take back what I said about Check being GPL, it is LGPL now. That
means that our tests wouldn't be infected by the GPL license.
Are there any thoughts on either of these? I have already submitted my
first set of patches to CuTest, so it will compile with gcc as soon as
they are accepted. I'll have the test suite working soon.
Unfortunately, Windows seems to be absolutely borked with regard to
_vsnprintf, so supposedly portable C code, isn't. Thanks go to JimJag
for helping track down the exact problem.
Ryan
----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA
> -----Original Message-----
> From: Ryan Bloom [mailto:rbb@covalent.net]
> Sent: Monday, July 08, 2002 7:06 AM
> To: dev@apr.apache.org
> Subject: New Test Suite
>
>
> Okay, I have looked at two different test suite options.
>
> 1) Check -- GPL licensed. Doesn't work on Windows.
> http://check.sourceforge.net/
>
> 2) CuTest -- Zlib license. Doesn't currently work on Unix, but I
> have patches already. I'll be submitting them later today.
> http://cutest.sourceforge.net/
>
> CuTest is two files, one .c and one .h, so it is very small and it
> should be able to do the job. Plus, we can continue to improve it if
we
> have to. The maintainer is currently using CuTest for his job, so it
is
> still under active maintenance. (There is one bug that I have found,
it
> is a difference in the way that Windows and Unix treat vsnprintf and
> strlen. I am looking into fixing it).
>
> Check is a lot more than two files, and the port to Windows could be
> kind of tough. The problem is that it requires fork() for a lot of
what
> it does, and although I haven't looked at the code in great depth, the
> parts I have looked at just look very Unix based. It is under active
> maintenance, but very slowly (According to the original author).
>
> My personal preference is to avoid the GPL all together and go with
the
> simple CuTest suite. Any thoughts?
>
> Ryan
>
> ----------------------------------------------
> Ryan Bloom rbb@covalent.net
> 645 Howard St. rbb@apache.org
> San Francisco, CA
>