You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2009/07/02 13:09:46 UTC
dkim2.t failures
2009/7/2 Mark Martinec <Ma...@ijs.si>:
> On Thu, 02 Jul 2009 00:13:52 +0200, Kevin A. McGrail <km...@pccc.com>
> wrote:
>
>> I'm running long tests and net tests and getting a lot of failures in the
>> DKIM area. I'm running 0.28.
>>
>> Is this a known issue? Should I upgrade DKIM and try again which would
>> like make me suggest a higher version requirement for the makefile.
>
> Don't know, it passes all tests for me.
> Does it happen to others too?
yep, me too.
> Is the plugin enabled at all?
> I would expect at least the tests for a presence of a signature
> (the DKIM_SIGNED) should work regardless of the version of Mail::DKIM.
>
> The DKIM_VALID would probably fail in some tests with older
> version on Mail::DKIM, but most of them should still pass.
here's what I see (with Mail::DKIM 0.19):
t/dkim2....1..80
# Running under perl version 5.008008 for linux
# Current time local: Thu Jul 2 11:05:07 2009
# Current time GMT: Thu Jul 2 11:05:07 2009
# Using Test.pm version 1.25
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/l
ocalrules.tmp -p log/test_default.cf -t < data/dkim/test-pass-16.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 1 in t/SATest.pm at line 707
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 2 in t/SATest.pm at line 707 fail #2
Output can be examined in: log/d.dkim/1
# Failed test 3 in t/dkim2.t at line 50
# t/dkim2.t line 50 is: ok ok_all_patterns();
Checking DKIM_SIGNED
not ok 1
Checking DKIM_VALID
not ok 2
not ok 3
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/l
ocalrules.tmp -p log/test_default.cf -t < data/dkim/test-pass-11.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 4 in t/SATest.pm at line 707 fail #3
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 5 in t/SATest.pm at line 707 fail #4
Output can be examined in: log/d.dkim/4
# Failed test 6 in t/dkim2.t at line 50 fail #2
Checking DKIM_SIGNED
not ok 4
Checking DKIM_VALID
not ok 5
not ok 6
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/l
ocalrules.tmp -p log/test_default.cf -t < data/dkim/test-pass-08.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 7 in t/SATest.pm at line 707 fail #5
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 8 in t/SATest.pm at line 707 fail #6
Output can be examined in: log/d.dkim/7
# Failed test 9 in t/dkim2.t at line 50 fail #3
Checking DKIM_SIGNED
not ok 7
Checking DKIM_VALID
not ok 8
not ok 9
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/l
ocalrules.tmp -p log/test_default.cf -t < data/dkim/test-pass-15.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 10 in t/SATest.pm at line 707 fail #7
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 11 in t/SATest.pm at line 707 fail #8
Output can be examined in: log/d.dkim/10
# Failed test 12 in t/dkim2.t at line 50 fail #4
Checking DKIM_SIGNED
not ok 10
Checking DKIM_VALID
not ok 11
not ok 12
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/l
ocalrules.tmp -p log/test_default.cf -t < data/dkim/test-pass-14.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 13 in t/SATest.pm at line 707 fail #9
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 14 in t/SATest.pm at line 707 fail #10
Output can be examined in: log/d.dkim/13
# Failed test 15 in t/dkim2.t at line 50 fail #5
Checking DKIM_SIGNED
not ok 13
Checking DKIM_VALID
not ok 14
not ok 15
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-09.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 16 in t/SATest.pm at line 707 fail #11
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 17 in t/SATest.pm at line 707 fail #12
Output can be examined in: log/d.dkim/16
# Failed test 18 in t/dkim2.t at line 50 fail #6
Checking DKIM_SIGNED
not ok 16
Checking DKIM_VALID
not ok 17
not ok 18
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-12.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 19 in t/SATest.pm at line 707 fail #13
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 20 in t/SATest.pm at line 707 fail #14
Output can be examined in: log/d.dkim/19
# Failed test 21 in t/dkim2.t at line 50 fail #7
Checking DKIM_SIGNED
not ok 19
Checking DKIM_VALID
not ok 20
not ok 21
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-19.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 22 in t/SATest.pm at line 707 fail #15
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 23 in t/SATest.pm at line 707 fail #16
Output can be examined in: log/d.dkim/22
# Failed test 24 in t/dkim2.t at line 50 fail #8
Checking DKIM_SIGNED
not ok 22
Checking DKIM_VALID
not ok 23
not ok 24
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-13.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 25 in t/SATest.pm at line 707 fail #17
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 26 in t/SATest.pm at line 707 fail #18
Output can be examined in: log/d.dkim/25
# Failed test 27 in t/dkim2.t at line 50 fail #9
Checking DKIM_SIGNED
not ok 25
Checking DKIM_VALID
not ok 26
not ok 27
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-18.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 28 in t/SATest.pm at line 707 fail #19
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 29 in t/SATest.pm at line 707 fail #20
Output can be examined in: log/d.dkim/28
# Failed test 30 in t/dkim2.t at line 50 fail #10
Checking DKIM_SIGNED
not ok 28
Checking DKIM_VALID
not ok 29
not ok 30
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-02.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 31 in t/SATest.pm at line 707 fail #21
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 32 in t/SATest.pm at line 707 fail #22
Output can be examined in: log/d.dkim/31
# Failed test 33 in t/dkim2.t at line 50 fail #11
Checking DKIM_SIGNED
not ok 31
Checking DKIM_VALID
not ok 32
not ok 33
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-17.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 34 in t/SATest.pm at line 707 fail #23
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 35 in t/SATest.pm at line 707 fail #24
Output can be examined in: log/d.dkim/34
# Failed test 36 in t/dkim2.t at line 50 fail #12
Checking DKIM_SIGNED
not ok 34
Checking DKIM_VALID
not ok 35
not ok 36
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-01.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 37 in t/SATest.pm at line 707 fail #25
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 38 in t/SATest.pm at line 707 fail #26
Output can be examined in: log/d.dkim/37
# Failed test 39 in t/dkim2.t at line 50 fail #13
Checking DKIM_SIGNED
not ok 37
Checking DKIM_VALID
not ok 38
not ok 39
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-06.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 40 in t/SATest.pm at line 707 fail #27
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 41 in t/SATest.pm at line 707 fail #28
Output can be examined in: log/d.dkim/40
# Failed test 42 in t/dkim2.t at line 50 fail #14
Checking DKIM_SIGNED
not ok 40
Checking DKIM_VALID
not ok 41
not ok 42
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-03.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 43 in t/SATest.pm at line 707 fail #29
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 44 in t/SATest.pm at line 707 fail #30
Output can be examined in: log/d.dkim/43
# Failed test 45 in t/dkim2.t at line 50 fail #15
Checking DKIM_SIGNED
not ok 43
Checking DKIM_VALID
not ok 44
not ok 45
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-05.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 46 in t/SATest.pm at line 707 fail #31
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 47 in t/SATest.pm at line 707 fail #32
Output can be examined in: log/d.dkim/46
# Failed test 48 in t/dkim2.t at line 50 fail #16
Checking DKIM_SIGNED
not ok 46
Checking DKIM_VALID
not ok 47
not ok 48
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-07.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 49 in t/SATest.pm at line 707 fail #33
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 50 in t/SATest.pm at line 707 fail #34
Output can be examined in: log/d.dkim/49
# Failed test 51 in t/dkim2.t at line 50 fail #17
Checking DKIM_SIGNED
not ok 49
Checking DKIM_VALID
not ok 50
not ok 51
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-10.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 52 in t/SATest.pm at line 707 fail #35
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 53 in t/SATest.pm at line 707 fail #36
Output can be examined in: log/d.dkim/52
# Failed test 54 in t/dkim2.t at line 50 fail #18
Checking DKIM_SIGNED
not ok 52
Checking DKIM_VALID
not ok 53
not ok 54
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-pass-04.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 50.
# Failed test 55 in t/SATest.pm at line 707 fail #37
Not found: DKIM_VALID = DKIM_VALID at t/dkim2.t line 50.
# Failed test 56 in t/SATest.pm at line 707 fail #38
Output can be examined in: log/d.dkim/55
# Failed test 57 in t/dkim2.t at line 50 fail #19
Checking DKIM_SIGNED
not ok 55
Checking DKIM_VALID
not ok 56
not ok 57
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-01.msg
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 60.
ok 58
ok 59
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-04.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 60 in t/SATest.pm at line 707 fail #39
Output can be examined in: log/d.dkim/60
# Failed test 62 in t/dkim2.t at line 72
# t/dkim2.t line 72 is: ok ok_all_patterns();
Checking DKIM_SIGNED
not ok 60
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 61
not ok 62
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-08.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 63 in t/SATest.pm at line 707 fail #40
Output can be examined in: log/d.dkim/63
# Failed test 65 in t/dkim2.t at line 72 fail #2
Checking DKIM_SIGNED
not ok 63
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 64
not ok 65
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-05.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 66 in t/SATest.pm at line 707 fail #41
Output can be examined in: log/d.dkim/66
# Failed test 68 in t/dkim2.t at line 72 fail #3
Checking DKIM_SIGNED
not ok 66
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 67
not ok 68
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-06.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 69 in t/SATest.pm at line 707 fail #42
Output can be examined in: log/d.dkim/69
# Failed test 71 in t/dkim2.t at line 72 fail #4
Checking DKIM_SIGNED
not ok 69
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 70
not ok 71
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-02.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 72 in t/SATest.pm at line 707 fail #43
Output can be examined in: log/d.dkim/72
# Failed test 74 in t/dkim2.t at line 72 fail #5
Checking DKIM_SIGNED
not ok 72
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 73
not ok 74
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-07.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 75 in t/SATest.pm at line 707 fail #44
Output can be examined in: log/d.dkim/75
# Failed test 77 in t/dkim2.t at line 72 fail #6
Checking DKIM_SIGNED
not ok 75
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 76
not ok 77
/usr/bin/perl -T -w ../spamassassin.raw -C log/test_rules_copy
--siteconfigpath log/localrules.tmp -p log/test_default.cf -t <
data/dkim/test-fail-03.msg
Not found: DKIM_SIGNED = DKIM_SIGNED at t/dkim2.t line 72.
# Failed test 78 in t/SATest.pm at line 707 fail #45
Output can be examined in: log/d.dkim/78
# Failed test 80 in t/dkim2.t at line 72 fail #7
Checking DKIM_SIGNED
not ok 78
Checking for anti-pattern DKIM_VALID at t/dkim2.t line 72.
ok 79
not ok 80
FAILED tests 1-57, 60, 62-63, 65-66, 68-69, 71-72, 74-75, 77-78, 80
Failed 71/80 tests, 11.25% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/dkim2.t 80 71 88.75% 1-57 60 62-63 65-66 68-69 71-72 74-
75 77-78 80
Failed 1/1 test scripts, 0.00% okay. 71/80 subtests failed, 11.25% okay.
> Please do try with Mail::DKIM 0.36, to see if that helps.
>
> Btw, you could run the following one-liner, giving it on stdin
> each of the 'pass' sample messages int/data/dkim/ :
>
>
> perl -MMail::DKIM::Verifier -ne '
> BEGIN{$dkim=Mail::DKIM::Verifier->new_object};
> s/\r?\n\z/\015\012/; $dkim->PRINT($_); END{$dkim->CLOSE;
> printf("%s\n",$_->result_detail) for $dkim->signatures}'
> < t/data/dkim/test-pass-19.msg
Can't locate object method "signatures" via package
"Mail::DKIM::Verifier" at -e line 1, <> line 14.
END failed--call queue aborted, <> line 14.
--j.
Re: dkim2.t failures
Posted by Mark Martinec <Ma...@ijs.si>.
> r790749 | jm | 2009-07-03 00:19:05 +0200 (Fri, 03 Jul 2009) | 1 line
> improve the dkim2.t test situation;
> I now get only tests 11-12 failing
Thanks for updating the data/01_test_rules.cf.
> I now get only tests 11-12 failing
I see that the order of tests depended on a file system
(directory order), which isn't very useful. Fixed:
Speed up the t/dkim2.t by 40% by using a persistent SA object and
avoid starting a command line spamassassin for each test message.
Do tests in sorted order of test messages instead of in the
seemingly random order of files as read from a directory.
Print out a name of each test message.
Sending t/dkim2.t
Committed revision 791499.
I'd expect the failing message to be the test-pass-07.msg
if you were using an old version of Mail::DKIM module.
(btw, as I'm not very familiar with conventions used in
tests, it may be useful if somebody glances over the
updated dkim2.t, especially its BEGIN section, as I just
copied it from somewhere else, not knowing wheather it is
really needed in this test)
Mark
Re: dkim2.t failures
Posted by Mark Martinec <Ma...@ijs.si>.
On Thu, 02 Jul 2009 13:09:46 +0200, Justin Mason <jm...@jmason.org> wrote:
> 2009/7/2 Mark Martinec <Ma...@ijs.si>:
>> On Thu, 02 Jul 2009 00:13:52 +0200, Kevin A. McGrail <km...@pccc.com>
>> wrote:
>>
>>> I'm running long tests and net tests and getting a lot of failures in the
>>> DKIM area. I'm running 0.28.
>>>
>>> Is this a known issue? Should I upgrade DKIM and try again which would
>>> like make me suggest a higher version requirement for the makefile.
>>
>> Don't know, it passes all tests for me.
>> Does it happen to others too?
>
> yep, me too.
There are no DKIM-related rules in the testing .cf file.
Something like the following should do, although I'm not sure
what is the right place to put these:
at the end of data/01_test_rules.cf place the:
ifplugin Mail::SpamAssassin::Plugin::DKIM
full DKIM_SIGNED eval:check_dkim_signed()
full DKIM_VALID eval:check_dkim_valid()
full DKIM_VALID_AU eval:check_dkim_valid_author_sig()
endif
Mark