[Bug 3602] New: use a t/config file for 'make test' data, instead of asking during 'perl Makefile.PL'

[bu...@bugzilla.spamassassin.org]

http://bugzilla.spamassassin.org/show_bug.cgi?id=3602

           Summary: use a t/config file for 'make test' data, instead of
                    asking during 'perl Makefile.PL'
           Product: Spamassassin
           Version: SVN Trunk (Latest Devel Version)
          Platform: Other
        OS/Version: other
            Status: NEW
          Severity: minor
          Priority: P5
         Component: Building & Packaging
        AssignedTo: spamassassin-dev@incubator.apache.org
        ReportedBy: jm@jmason.org


IMO, we should define a small configuration file in the "t" directory to control
aspects of how the test scripts are run, what they test, and what servers (if
required) they use during the test process.

This will avoid having to ask more and more annoying questions during the "perl
Makefile.PL" process.

Most users installing SA don't need to specify SQL servers, run net tests, etc.;
only developers or contributors need bother with that, and they can take the
time to read some brief doco on how that config file works, how to edit it, etc.


Moving some discussion of this idea from bug 3573:

me:
> btw -- for future thought, we should stop asking stupid "make
> test"-related questions in Makefile.PL, it's far too annoying for most
> people.  instead we should just have a simple config file in the t dir
> to list all these settings.

theo:
> So then it'll be "perl Makefile.PL", "make", "edit the t/config file",
> "make test"?

Sidney:
> I would hope that editing the t/config file would be done only in
> unusual circumstances. The regression tests should be a sanity check for
> everything that is built. The network tests are tricky, since it makes
> sense to build SpamAssassin with the ability to perform network
> functions without the build environment having the ability to run them.
> But I would look at it on a case by case basis, whether it is possible
> to run a regression test with no user inout, or if not whether it is
> less confusing to ask a question or to require editing a config file.

me:
> I'm seeing the information as a kind of site-specific policy on what
> tests to run.  Most CPAN-using users don't need to run SSL tests, or
> LDAP/SQL tests where explicit user setup is required; however, we as
> devs should be using those.

I'd also expand on that to note that regression tests that do not require user
input, or the details of an SQL/LDAP server to use, etc., should still always be
run.   In other words, the generic tests work fine; this'd be only for complex
tests that require setup in advance, or may fail based on external factors (such
as the net tests).



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.