Parallelizing the test suite

[David Glasser <gl...@mit.edu>]

Some very superficial experiments [1] seem to show that running
multiple tests in parallel can give a reasonable speedup.  Somebody
with more free time than myself may want to look into (optionally)
running tests in parallel, either at the level of running several
python scripts at the same time, or more likely making the test runner
fork itself.  It's not trivial (organizing the success and failure
results being the hardest part) but it could speed up the test suite,
which is always a good thing.  (Not to mention potentially finding
weird server concurrency bugs.)

--dave


[1] Specifically, timing "./prop_tests.py & sleep 1; ./diff_tests.py"
vs "./prop_tests.py; ./diff_tests.py", which took 1m44 and 2m20s
respectively.  They do fight a little over the Greek repo in the
former.

-- 
David Glasser | glasser@mit.edu | http://www.davidglasser.net/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Parallelizing the test suite

[Ivan Zhakov <ch...@gmail.com>]

On 10/6/06, David Glasser <gl...@mit.edu> wrote:
> Some very superficial experiments [1] seem to show that running
> multiple tests in parallel can give a reasonable speedup.  Somebody
> with more free time than myself may want to look into (optionally)
> running tests in parallel, either at the level of running several
> python scripts at the same time, or more likely making the test runner
> fork itself.  It's not trivial (organizing the success and failure
> results being the hardest part) but it could speed up the test suite,
> which is always a good thing.  (Not to mention potentially finding
> weird server concurrency bugs.)
Sounds good. My feeling sayes me that test-suite spend most time in
sleep_for_timestamp which is executed after each svn operation. I've
made test to remove this sleep -- it's speedup testsuite twice. So
I'll very happy if somebody implement parallelizing testsute.

> [1] Specifically, timing "./prop_tests.py & sleep 1; ./diff_tests.py"
> vs "./prop_tests.py; ./diff_tests.py", which took 1m44 and 2m20s
> respectively.  They do fight a little over the Greek repo in the
> former.
>
> --

-- 
Ivan Zhakov

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Parallelizing the test suite

[Lieven Govaerts <sv...@mobsol.be>]

David Glasser wrote:
> Some very superficial experiments [1] seem to show that running
> multiple tests in parallel can give a reasonable speedup.  Somebody
> with more free time than myself may want to look into (optionally)
> running tests in parallel, either at the level of running several
> python scripts at the same time, or more likely making the test runner
> fork itself.  It's not trivial (organizing the success and failure
> results being the hardest part) but it could speed up the test suite,
> which is always a good thing.  (Not to mention potentially finding
> weird server concurrency bugs.)
>
Interesting!
>
> [1] Specifically, timing "./prop_tests.py & sleep 1; ./diff_tests.py"
> vs "./prop_tests.py; ./diff_tests.py", which took 1m44 and 2m20s
> respectively.  They do fight a little over the Greek repo in the
> former.
>
Whenever you run a python test file it's setting up the base repo and 
working copy in local_tmp, those things need to be adapted to enable 
parallelization.

I like the suggestion very much, if I have some time this weekend I'll 
have a look at it.

regards,

Lieven.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Parallelizing the test suite

[Madan U Sreenivasan <ma...@collab.net>]

On Fri, 06 Oct 2006 20:30:53 +0530, David Glasser <gl...@mit.edu> wrote:

> Some very superficial experiments [1] seem to show that running
> multiple tests in parallel can give a reasonable speedup.  Somebody
> with more free time than myself may want to look into (optionally)
> running tests in parallel, either at the level of running several
> python scripts at the same time, or more likely making the test runner
> fork itself.  It's not trivial (organizing the success and failure
> results being the hardest part) but it could speed up the test suite,
> which is always a good thing.  (Not to mention potentially finding
> weird server concurrency bugs.)
>

looove the idea!

But, AFAIK , test runs on a different repository, which means the server  
concurrency item that you mention may not happen.

Regards,
Madan.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org