You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mod_python-dev@quetz.apache.org by Nicolas Lehuen <ni...@lehuen.com> on 2006/03/28 16:40:19 UTC
Latest tests
Hi,
Just FYI, here are the results of my latest build & tests with mod_python
SVN revision 387864 :
+1 ActivePython 2.4.2.10 / Apache 2.0.55 / Windows XP SP2 / old importer
+1 ActivePython 2.4.2.10 / Apache 2.0.55 / Windows XP SP2 / new importer
+1 ActivePython 2.3.5.? / Apache 2.0.55 / Windows 2000 SP4 / old importer
+1 ActivePython 2.3.5.? / Apache 2.0.55 / Windows 2000 SP4 / new importer
+1 Python 2.2.3 / Apache 2.0.55 / Windows 2000 SP4 / old importer
-1 Python 2.2.3 / Apache 2.0.55 / Windows 2000 SP4 / new importer
* A lot of failures caused by the new importer, as expected
With mod_python SVN revision 389505 :
-1 ActivePython 2.4.1.247 / Apache 2.2.0 / Windows XP SP2 / new importer
* Still the same problem in test_req_auth_type, I still need to check
this
BTW, one of my fantasy is that when the test suite is refactored, we use
BuildBot to automate the tests on multiple platforms.
Regards,
Nicolas
Re: Grouping tests
Posted by Jim Gallacher <jp...@jgassociates.ca>.
This isn't too different from what I have in mind, except that my main
testrunner.py script uses introspection to discover the tests. Adding a
new test set won't require editing testrunner at all. The testrunner
will also filter the tests according to certain criteria, such as mpm or
os, so that tests for a specific platform are automatically selected.
More later.
Jim
Mike Looijmans wrote:
> What I have been doing in a totally unrelated Python project is to
> create test groups simply by putting them into separate modules. The
> main test module test.py looks like this:
>
> ## (test.py)
> import unittest
> from test_something import *
> from test_someother import *
> from test_yetmore import *
> if __name__ == '__main__':
> unittest.main(module=__import__(__name__))
> ##
>
> This works because unittest takes all 'things' that start with 'test' in
> the provided module and runs them. So anything we bring into our
> namespace gets run. This also makes it possible to import tests from
> other projects, and share these tests between projects.
> The other test_ modules look much alike:
>
> ## (test_something.py)
> import unittest
> import test_peer
> class test04MultiPeerSystem(test_peer.BaseTestRealThing):
> def test03DiscoveryC08B20(self):
> "Multiple clients w/ discovery: 8 peers, 20 blocks each"
> self.runDiscoveryTest(nclients = 8, nblocks=20)
> if __name__ == '__main__':
> unittest.main(module=__import__(__name__))
> ##
>
> This makes it very easy to handle test subsets, and run single test
> suites. Just run
>
> $python test.py
>
> to run ALL the tests. To run just a single set, run
>
> $python test_something.py
>
> And to run a single test, either of these will do:
>
> $python test_something.py test04MultiPeerSystem
> $python test.py test04MultiPeerSystem
>
> The real power shows when you want to run 4 or 5 test sets, and/or only
> parts of some test sets. Just create a new "main" test unit that imports
> the desired ones, and you're set:
>
> ## (test_few.py)
> import unittest
> from test_something import *
> from test_someother import TestOnlyThis
> if __name__ == '__main__':
> unittest.main(module=__import__(__name__))
> ##
>
> Because some tests take very long to run (in my vocabulary, "long" means
> more than a second), this saves me a lot of time when working on a part
> of a big project, where I don't need to run all tests all the time.
>
> --
> Mike Looijmans
> Philips Natlab / Topic Automation
>
>
> Jim Gallacher wrote:
> ...
>
>> I've been playing with some ideas for a new test framework, using a
>> subclass of unittest.TestLoader to find and configure tests. I want to
>> play around with it for another day or so before sharing but at this
>> point I'm pretty confident it'll work.
>>
>> Creating a new set of tests could be as simple as:
>>
>> testsuites/core/simpletest.py
>> ----------------------------
>>
>> from testlib import VirtualHostTest
>>
>> class MyTests(VirtualHostTest):
>> def test_hello_world(self):
>> rsp = self.send_request()
>> self.failUnless(rsp == 'test ok')
>>
>> def test_goodbye_world(self):
>> rsp = self.send_request()
>> self.failUnless(rsp == 'test ok')
>>
>>
>> htdocs/testhandlers/core/simpletest.py
>> --------------------------------------
>>
>> from mod_python import
>>
>> def test_hello_world(req):
>> req.write('test ok')
>> return apache.OK
>>
>> def test_goodbye_world(req):
>> req.write('test ok')
>> return apache.OK
>>
>> $ python testrunner.py
>>
>> Things like virtual host names and handler directives required for
>> configuration or send_request() are automatically derived from the
>> test class and test method names. It will still be possible to provide
>> custom apache configuration directives in a manner similar to that
>> which we currently use, but for most tests this will not be required.
>>
>> Usage would look something like this:
>>
>> Run all the tests
>> $ python testrunner.py
>>
>> Run one test
>> $ python testrunner.py -t core.simpletest.MyTests.test_hello_world
>>
>> Run a group of tests (this would load the TestCase subclasses in
>> testsuites/sessions/filesession.py):
>>
>> $ python testrunner.py -t sessions.filesession
>>
>>
>> Jim
>
>
>
Grouping tests (was: Latest tests)
Posted by Mike Looijmans <nl...@natlab.research.philips.com>.
What I have been doing in a totally unrelated Python project is to create test groups simply by
putting them into separate modules. The main test module test.py looks like this:
## (test.py)
import unittest
from test_something import *
from test_someother import *
from test_yetmore import *
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
This works because unittest takes all 'things' that start with 'test' in the provided module and
runs them. So anything we bring into our namespace gets run. This also makes it possible to import
tests from other projects, and share these tests between projects.
The other test_ modules look much alike:
## (test_something.py)
import unittest
import test_peer
class test04MultiPeerSystem(test_peer.BaseTestRealThing):
def test03DiscoveryC08B20(self):
"Multiple clients w/ discovery: 8 peers, 20 blocks each"
self.runDiscoveryTest(nclients = 8, nblocks=20)
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
This makes it very easy to handle test subsets, and run single test suites. Just run
$python test.py
to run ALL the tests. To run just a single set, run
$python test_something.py
And to run a single test, either of these will do:
$python test_something.py test04MultiPeerSystem
$python test.py test04MultiPeerSystem
The real power shows when you want to run 4 or 5 test sets, and/or only parts of some test sets.
Just create a new "main" test unit that imports the desired ones, and you're set:
## (test_few.py)
import unittest
from test_something import *
from test_someother import TestOnlyThis
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
Because some tests take very long to run (in my vocabulary, "long" means more than a second), this
saves me a lot of time when working on a part of a big project, where I don't need to run all tests
all the time.
--
Mike Looijmans
Philips Natlab / Topic Automation
Jim Gallacher wrote:
...
> I've been playing with some ideas for a new test framework, using a
> subclass of unittest.TestLoader to find and configure tests. I want to
> play around with it for another day or so before sharing but at this
> point I'm pretty confident it'll work.
>
> Creating a new set of tests could be as simple as:
>
> testsuites/core/simpletest.py
> ----------------------------
>
> from testlib import VirtualHostTest
>
> class MyTests(VirtualHostTest):
> def test_hello_world(self):
> rsp = self.send_request()
> self.failUnless(rsp == 'test ok')
>
> def test_goodbye_world(self):
> rsp = self.send_request()
> self.failUnless(rsp == 'test ok')
>
>
> htdocs/testhandlers/core/simpletest.py
> --------------------------------------
>
> from mod_python import
>
> def test_hello_world(req):
> req.write('test ok')
> return apache.OK
>
> def test_goodbye_world(req):
> req.write('test ok')
> return apache.OK
>
> $ python testrunner.py
>
> Things like virtual host names and handler directives required for
> configuration or send_request() are automatically derived from the test
> class and test method names. It will still be possible to provide custom
> apache configuration directives in a manner similar to that which we
> currently use, but for most tests this will not be required.
>
> Usage would look something like this:
>
> Run all the tests
> $ python testrunner.py
>
> Run one test
> $ python testrunner.py -t core.simpletest.MyTests.test_hello_world
>
> Run a group of tests (this would load the TestCase subclasses in
> testsuites/sessions/filesession.py):
>
> $ python testrunner.py -t sessions.filesession
>
>
> Jim
Re: Latest tests
Posted by Jim Gallacher <jp...@jgassociates.ca>.
Nicolas Lehuen wrote:
> Hi,
>
> Just FYI, here are the results of my latest build & tests with mod_python
> SVN revision 387864 :
>
> +1 ActivePython 2.4.2.10 / Apache 2.0.55 / Windows XP SP2 / old importer
> +1 ActivePython 2.4.2.10 / Apache 2.0.55 / Windows XP SP2 / new importer
> +1 ActivePython 2.3.5.? / Apache 2.0.55 / Windows 2000 SP4 / old importer
> +1 ActivePython 2.3.5.? / Apache 2.0.55 / Windows 2000 SP4 / new importer
> +1 Python 2.2.3 / Apache 2.0.55 / Windows 2000 SP4 / old importer
>
> -1 Python 2.2.3 / Apache 2.0.55 / Windows 2000 SP4 / new importer
> * A lot of failures caused by the new importer, as expected
>
> With mod_python SVN revision 389505 :
>
> -1 ActivePython 2.4.1.247 / Apache 2.2.0 / Windows XP SP2 / new importer
> * Still the same problem in test_req_auth_type, I still need to check
> this
>
> BTW, one of my fantasy is that when the test suite is refactored, we use
> BuildBot to automate the tests on multiple platforms.
I've been playing with some ideas for a new test framework, using a
subclass of unittest.TestLoader to find and configure tests. I want to
play around with it for another day or so before sharing but at this
point I'm pretty confident it'll work.
Creating a new set of tests could be as simple as:
testsuites/core/simpletest.py
----------------------------
from testlib import VirtualHostTest
class MyTests(VirtualHostTest):
def test_hello_world(self):
rsp = self.send_request()
self.failUnless(rsp == 'test ok')
def test_goodbye_world(self):
rsp = self.send_request()
self.failUnless(rsp == 'test ok')
htdocs/testhandlers/core/simpletest.py
--------------------------------------
from mod_python import
def test_hello_world(req):
req.write('test ok')
return apache.OK
def test_goodbye_world(req):
req.write('test ok')
return apache.OK
$ python testrunner.py
Things like virtual host names and handler directives required for
configuration or send_request() are automatically derived from the test
class and test method names. It will still be possible to provide custom
apache configuration directives in a manner similar to that which we
currently use, but for most tests this will not be required.
Usage would look something like this:
Run all the tests
$ python testrunner.py
Run one test
$ python testrunner.py -t core.simpletest.MyTests.test_hello_world
Run a group of tests (this would load the TestCase subclasses in
testsuites/sessions/filesession.py):
$ python testrunner.py -t sessions.filesession
Jim