You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Lieven Govaerts <lg...@mobsol.be> on 2006/11/06 00:36:42 UTC
Parallelizing the python testsuite
Attached patch is my work in progress towards a parallelized python test
suite. It's not completely finished, basically it needs some more
testing, some fixes (not all the tests are passing), code cleanup and a
log message. But, I send it to the list anyway, so people interested in
this patch can already have a look at it .
There are two new cmd-line parameters for a test:
- add a -p/--parallel cmd-line option to the tests; triggers parallel
execution
- add a -c option (child process) to the tests; disables setup and
cleanup of temporary folders
When run with -p the main python process will create some threads each
spawning another child process, this for each test in the list. The
output of the child process (stdout, stderr, exit code) is cached, and
will only be printed when all the tests were run.
Currently the max. nr. of child processes is 10, but I propose to make
that a value of the -p option.
The two problems I had parallelizing the testsuite are/were with:
- tests that change the current directory or the environment ($EDITOR):
these were handled by changing the original implementation with threads
to child processes. We had a long discussion on IRC about this, current
implementation is based on the consensus of that discussion.
- tests that change the authz file: for dav tests, the authz file has to
be put in a specific unchangeable location, which makes it impossible to
run multiple authz tests in parallel. I've currently solved this with
adding an option to completely disable parallel execution, which is now
activited for authz_tests.py and svnsync_tests.py.
I have some other changes in run_tests.py and win_tests.py to support
the -p option, but those aren't included in the patch.
Things I didn't test: whether or not the exit_code is correctly
returned, ra_dav, ra_svn & non-Windows platforms.
All input is welcome.
Oh before I forget, the good part: both when run on hard disk or ram
disk, running a file like basic_tests.py improved performance from 50
seconds to 20 seconds, so that's about 2,5x faster.
Lieven
Re: Parallelizing the python testsuite
Posted by Lieven Govaerts <sv...@mobsol.be>.
Ivan Zhakov wrote:
> On 11/9/06, Lieven Govaerts <sv...@mobsol.be> wrote:
>> Quoting Ivan Zhakov <ch...@gmail.com>:
>>
>> > On 11/9/06, Ivan Zhakov <ch...@gmail.com> wrote:
>> > > On 11/6/06, Lieven Govaerts <lg...@mobsol.be> wrote:
>> > > >
>> > > > Attached patch is my work in progress towards a parallelized
>> python test
>> > > > suite. It's not completely finished, basically it needs some more
>> > > > testing, some fixes (not all the tests are passing), code
>> cleanup and a
>> > > > log message. But, I send it to the list anyway, so people
>> interested in
>> > > > this patch can already have a look at it .
>> > > Hi Lieven,
>> > > Great work! For present time I'm switching to Mac, so I've tested
>> your
>> > patch:
>> > > [[
>> > > chmbook:~/Subversion/trunk/subversion/tests/cmdline ivan$
>> ./basic_tests.py
>> > -p
>> > > ................................Traceback (most recent call last):
>> > > File "./basic_tests.py", line 1773, in ?
>> > > svntest.main.run_tests(test_list)
>> > > File
>> >
>> "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
>> > > line 996, in run_tests
>> > > exit_code = _internal_run_tests(test_list, testnums, parallel)
>> > > File
>> >
>> "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
>> > > line 868, in _internal_run_tests
>> > > finished_tests.sort(key=lambda test: test.index)
>> > > TypeError: sort() takes no keyword arguments
>> > > ]]
>> > >
>> > > May be it's because it doesn't support Python 2.3?
>>
>> The lambda feature is introduced in 2.4. I have alternative code,
>> I'll change to
>> patch to use that.
>>
>> > Also catch my addition to your patch to implement make check PARALLEL
>> > command to test on non-windows platform.
>> >
>> Yes, thanks for that, I was hoping for someone to add that :)
>>
>> I have to change some things in the sandbox set up code, running
>> basic_tests on
>> ra_dav in parallel doesn't work (see the other mail thread: "[svn] x
>> [fsfs]
>> basic_tests.py failures").
>>
> Yes, I've seen that thread. I think it's good idea to send updated
> your patch with my changes to list.
>
Attached is the new version of the patch. Note that some commits have
been made to trunk in the meantime, which solve some problems with tests
behaving badly when run in parallel;
The new patch allows you to specify the number of tests run in parallel,
it still defaults to 10.
One problem: I've tested this on my Mac buildslave, and it always fails
in some test. Sometimes a child process appears to be hanging, thereby
generating an exception in the parent process. If I don't kill the child
process, it will hang forever (well, a long time atleast). This
exception is shown in the logs *before* I kill the child process:
Exception in thread Thread-47:Traceback (most recent call last):
File
"/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/threading.py",
line 442, in __bootstrap
self.run()
File
"/Users/lgo/slavedir/osx10.4-gcc4.0.1-ia32/build/subversion/tests/cmdline/svntest/main.py",
line 724, in run
self.result, self.stdout_lines, self.stderr_lines =\
File
"/Users/lgo/slavedir/osx10.4-gcc4.0.1-ia32/build/subversion/tests/cmdline/svntest/main.py",
line 286, in spawn_process
pid, wait_code = os.wait()
OSError: [Errno 10] No child processes
Lieven
Re: Parallelizing the python testsuite
Posted by Ivan Zhakov <ch...@gmail.com>.
On 11/9/06, Lieven Govaerts <sv...@mobsol.be> wrote:
> Quoting Ivan Zhakov <ch...@gmail.com>:
>
> > On 11/9/06, Ivan Zhakov <ch...@gmail.com> wrote:
> > > On 11/6/06, Lieven Govaerts <lg...@mobsol.be> wrote:
> > > >
> > > > Attached patch is my work in progress towards a parallelized python test
> > > > suite. It's not completely finished, basically it needs some more
> > > > testing, some fixes (not all the tests are passing), code cleanup and a
> > > > log message. But, I send it to the list anyway, so people interested in
> > > > this patch can already have a look at it .
> > > Hi Lieven,
> > > Great work! For present time I'm switching to Mac, so I've tested your
> > patch:
> > > [[
> > > chmbook:~/Subversion/trunk/subversion/tests/cmdline ivan$ ./basic_tests.py
> > -p
> > > ................................Traceback (most recent call last):
> > > File "./basic_tests.py", line 1773, in ?
> > > svntest.main.run_tests(test_list)
> > > File
> > "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> > > line 996, in run_tests
> > > exit_code = _internal_run_tests(test_list, testnums, parallel)
> > > File
> > "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> > > line 868, in _internal_run_tests
> > > finished_tests.sort(key=lambda test: test.index)
> > > TypeError: sort() takes no keyword arguments
> > > ]]
> > >
> > > May be it's because it doesn't support Python 2.3?
>
> The lambda feature is introduced in 2.4. I have alternative code, I'll change to
> patch to use that.
>
> > Also catch my addition to your patch to implement make check PARALLEL
> > command to test on non-windows platform.
> >
> Yes, thanks for that, I was hoping for someone to add that :)
>
> I have to change some things in the sandbox set up code, running basic_tests on
> ra_dav in parallel doesn't work (see the other mail thread: "[svn] x [fsfs]
> basic_tests.py failures").
>
Yes, I've seen that thread. I think it's good idea to send updated
your patch with my changes to list.
--
Ivan Zhakov
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Parallelizing the python testsuite
Posted by Lieven Govaerts <sv...@mobsol.be>.
Quoting Ivan Zhakov <ch...@gmail.com>:
> On 11/9/06, Ivan Zhakov <ch...@gmail.com> wrote:
> > On 11/6/06, Lieven Govaerts <lg...@mobsol.be> wrote:
> > >
> > > Attached patch is my work in progress towards a parallelized python test
> > > suite. It's not completely finished, basically it needs some more
> > > testing, some fixes (not all the tests are passing), code cleanup and a
> > > log message. But, I send it to the list anyway, so people interested in
> > > this patch can already have a look at it .
> > Hi Lieven,
> > Great work! For present time I'm switching to Mac, so I've tested your
> patch:
> > [[
> > chmbook:~/Subversion/trunk/subversion/tests/cmdline ivan$ ./basic_tests.py
> -p
> > ................................Traceback (most recent call last):
> > File "./basic_tests.py", line 1773, in ?
> > svntest.main.run_tests(test_list)
> > File
> "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> > line 996, in run_tests
> > exit_code = _internal_run_tests(test_list, testnums, parallel)
> > File
> "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> > line 868, in _internal_run_tests
> > finished_tests.sort(key=lambda test: test.index)
> > TypeError: sort() takes no keyword arguments
> > ]]
> >
> > May be it's because it doesn't support Python 2.3?
The lambda feature is introduced in 2.4. I have alternative code, I'll change to
patch to use that.
> Also catch my addition to your patch to implement make check PARALLEL
> command to test on non-windows platform.
>
Yes, thanks for that, I was hoping for someone to add that :)
I have to change some things in the sandbox set up code, running basic_tests on
ra_dav in parallel doesn't work (see the other mail thread: "[svn] x [fsfs]
basic_tests.py failures").
Lieven
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Parallelizing the python testsuite
Posted by Ivan Zhakov <ch...@gmail.com>.
On 11/9/06, Ivan Zhakov <ch...@gmail.com> wrote:
> On 11/6/06, Lieven Govaerts <lg...@mobsol.be> wrote:
> >
> > Attached patch is my work in progress towards a parallelized python test
> > suite. It's not completely finished, basically it needs some more
> > testing, some fixes (not all the tests are passing), code cleanup and a
> > log message. But, I send it to the list anyway, so people interested in
> > this patch can already have a look at it .
> Hi Lieven,
> Great work! For present time I'm switching to Mac, so I've tested your patch:
> [[
> chmbook:~/Subversion/trunk/subversion/tests/cmdline ivan$ ./basic_tests.py -p
> ................................Traceback (most recent call last):
> File "./basic_tests.py", line 1773, in ?
> svntest.main.run_tests(test_list)
> File "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> line 996, in run_tests
> exit_code = _internal_run_tests(test_list, testnums, parallel)
> File "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
> line 868, in _internal_run_tests
> finished_tests.sort(key=lambda test: test.index)
> TypeError: sort() takes no keyword arguments
> ]]
>
> May be it's because it doesn't support Python 2.3?
>
Also catch my addition to your patch to implement make check PARALLEL
command to test on non-windows platform.
--
Ivan Zhakov
Re: Parallelizing the python testsuite
Posted by Ivan Zhakov <ch...@gmail.com>.
On 11/6/06, Lieven Govaerts <lg...@mobsol.be> wrote:
>
> Attached patch is my work in progress towards a parallelized python test
> suite. It's not completely finished, basically it needs some more
> testing, some fixes (not all the tests are passing), code cleanup and a
> log message. But, I send it to the list anyway, so people interested in
> this patch can already have a look at it .
Hi Lieven,
Great work! For present time I'm switching to Mac, so I've tested your patch:
[[
chmbook:~/Subversion/trunk/subversion/tests/cmdline ivan$ ./basic_tests.py -p
................................Traceback (most recent call last):
File "./basic_tests.py", line 1773, in ?
svntest.main.run_tests(test_list)
File "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
line 996, in run_tests
exit_code = _internal_run_tests(test_list, testnums, parallel)
File "/Users/ivan/Subversion/trunk/subversion/tests/cmdline/svntest/main.py",
line 868, in _internal_run_tests
finished_tests.sort(key=lambda test: test.index)
TypeError: sort() takes no keyword arguments
]]
May be it's because it doesn't support Python 2.3?
--
Ivan Zhakov
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org