You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2015/04/03 14:38:00 UTC
[Bug 7163] New: Parallel make test
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Bug ID: 7163
Summary: Parallel make test
Product: Spamassassin
Version: SVN Trunk (Latest Devel Version)
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P2
Component: Building & Packaging
Assignee: dev@spamassassin.apache.org
Reporter: jquinn+SAbug@pccc.com
Running "make test" takes a while, and it would be nice to make it support
parallel testing to make it go faster for easier rapid development.
The test suite is executed as a single job, so -j will not work to speed it up
as you would usually expect. Fortunately, Test::Harness looks for a few
environment variables and one of them lets us set jobs. Unfortunately, I get
the following error when trying to use it on the current trunk, among others:
Apr 3 08:23:42.600 [20246] warn: locker: error accessing
./log/user_state/auto-whitelist.lock: No such file or directory at
../blib/lib/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 106.
There are likely more issues, will include a full log in a bit. In addition to
being able to do faster testing, we can gain some extra opportunities for
concurrency and maybe make scanning faster as well.
Further Reading:
http://search.cpan.org/~leont/Test-Harness-3.35/lib/Test/Harness.pm
http://stackoverflow.com/questions/9147196/makefile-pass-jobs-param-to-sub-makefiles
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
--- Comment #5 from Todd Rinaldo <to...@cpanel.net> ---
Assuming the test can't be broken up Test::Parallel works just fine.
I think this requires you don't care about windows.
Todd
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Henrik Krohns <he...@hege.li> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |hege@hege.li
Target Milestone|3.4.3 |4.0.0
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Henrik Krohns <ap...@hege.li> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|NEW |RESOLVED
--- Comment #6 from Henrik Krohns <ap...@hege.li> ---
Major test cleanups. Parallel testing supported (HARNESS_OPTIONS=j8 make test),
configuration t/testrules.yml.
Committed revision 1888649.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Kevin A. McGrail <km...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kmcgrail@apache.org
--- Comment #2 from Kevin A. McGrail <km...@apache.org> ---
In addition to running the plans on test in parallel, it would seem that
running separate t/ files in parallel might be much quicker.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Todd Rinaldo <to...@cpanel.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |toddr@cpanel.net
--- Comment #3 from Todd Rinaldo <to...@cpanel.net> ---
I'm planning on looking at this some time soon.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Joe Quinn <jq...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jquinn+SAbug@pccc.com
Target Milestone|Undefined |3.4.2
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
--- Comment #1 from Joe Quinn <jq...@pccc.com> ---
Created attachment 5292
--> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5292&action=edit
Log of make test
Full test log attached. The command I ran was:
HARNESS_OPTIONS=j2 make test
>From quickly scanning the file, I see a variety of issues that seem to largely
revolve around file manipulation. The test did run faster by 150 seconds, even
for being on a single-processor VM.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@spamassassin.apache.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
--- Comment #7 from Kevin A. McGrail <km...@apache.org> ---
Love speed up!
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
Kevin A. McGrail <km...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|3.4.2 |3.4.3
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7163] Parallel make test
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7163
--- Comment #4 from Kevin A. McGrail <km...@apache.org> ---
(In reply to Todd Rinaldo from comment #3)
> I'm planning on looking at this some time soon.
Thanks. Right now it takes 16 mins on my rig for a non-root make test. So
every little snippet means a quarter hour gone. If I can close that gap down,
it will help.
The harness options for parallel in a single test doesn't look that promising
but the concept that perhaps SATest.pm is a framework to run more than one t/
file at a time is interesting to me.
Both would be ideal but perhaps overthinking it!
KAM
--
You are receiving this mail because:
You are the assignee for the bug.