You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ruleqa@spamassassin.apache.org by John Hardin <jh...@impsec.org> on 2018/10/26 14:38:12 UTC
Re: Rule updates are too old - 2018-10-26
On Fri, 26 Oct 2018, darxus@chaosreigns.com wrote:
> 20181025: Spam or ham is below threshold of 150,000: http://ruleqa.spamassassin.org/?daterev=20181025
> 20181025: Spam: 96744, Ham: 376566
It looks like the llanga and mmiroslaw masschecks are still having timing
issues, and this is dropping the cumulative corpora below the publication
threshold.
mmiroslaw, llanga:
Could you check your masscheck schedules and durations again?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
...the Fates notice those who buy chainsaws...
-- www.darwinawards.com
-----------------------------------------------------------------------
5 days until Halloween
Re: Rule updates are too old - 2018-10-26
Posted by John Hardin <jh...@impsec.org>.
On Sun, 28 Oct 2018, Marcin Mirosław wrote:
> W dniu 2018-10-26 o 16:38, John Hardin pisze:
>> On Fri, 26 Oct 2018, darxus@chaosreigns.com wrote:
>>
>>> 20181025: Spam or ham is below threshold of 150,000:
>>> http://ruleqa.spamassassin.org/?daterev=20181025
>>> 20181025: Spam: 96744, Ham: 376566
>>
>> It looks like the llanga and mmiroslaw masschecks are still having
>> timing issues, and this is dropping the cumulative corpora below the
>> publication threshold.
>>
>> mmiroslaw, llanga:
>>
>> Could you check your masscheck schedules and durations again?
>
> I have no idea how to read ruleqa web :(
It's pretty simple. Just visit https://ruleqa.spamassassin.org/ and look
at the daterevs at the top. If your corpora are in the same row as all the
others you're golden.
If your corpora appears on a different row, even if the SVN revision
number is the same, then the results data is split and probably isn't
being considered as an aggregate for score generation.
It looks like you're okay right now.
> I didn't shift time when masscheck starts. Masscheck start at 11:57:05
> UTC and finishes after a few minutes.
>
> Marcin
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
...the Fates notice those who buy chainsaws...
-- www.darwinawards.com
-----------------------------------------------------------------------
3 days until Halloween
Re: Rule updates are too old - 2018-10-26
Posted by Marcin Mirosław <ma...@mejor.pl>.
W dniu 2018-10-26 o 16:38, John Hardin pisze:
> On Fri, 26 Oct 2018, darxus@chaosreigns.com wrote:
>
>> 20181025: Spam or ham is below threshold of 150,000:
>> http://ruleqa.spamassassin.org/?daterev=20181025
>> 20181025: Spam: 96744, Ham: 376566
>
> It looks like the llanga and mmiroslaw masschecks are still having
> timing issues, and this is dropping the cumulative corpora below the
> publication threshold.
>
> mmiroslaw, llanga:
>
> Could you check your masscheck schedules and durations again?
Hi!
I have no idea how to read ruleqa web :( I didn't shift time when
masscheck starts. Masscheck start at 11:57:05 UTC and finishes after a
few minutes.
Marcin
Re: Rule updates are too old - 2018-10-26
Posted by Henrik K <he...@hege.li>.
On Sun, Oct 28, 2018 at 01:42:06PM +0200, Henrik K wrote:
> On Sun, Oct 28, 2018 at 06:28:38AM -0500, Dave Jones wrote:
> >
> > If there is a good repo for upgrading the system perl on a C6 box
> > correctly, I would do it.
>
> It's an age old question. I doubt you'll find any other reliable and secure
> solution besides compiling it yourself.
Well I have to correct myself.
https://wiki.centos.org/AdditionalResources/Repositories/SCL
https://www.softwarecollections.org/en/scls/?search=perl
rh-perl524 installed fine on my CentOS 6.
So if any of you guys are still using pesky old system perls, have a look..
it's always better to use a dedicated perl for SA anyway.
Re: Rule updates are too old - 2018-10-26
Posted by Henrik K <he...@hege.li>.
On Sun, Oct 28, 2018 at 12:04:37PM -0700, John Hardin wrote:
>
> Is llanga even on this list? Can we get contact info to contact him directly
> and ask that he review his masscheck scheduling?
Yeah I was wondering this myself, why do you have people there without any
contact information..
Can't we just delay and fake he's results to the main batch?
Re: Rule updates are too old - 2018-10-26
Posted by John Hardin <jh...@impsec.org>.
It looks like the timing issues got better - the llanga results are still
coming in off-schedule but that's been the case for a long time. Everybody
else is in the main batch.
Is llanga even on this list? Can we get contact info to contact him
directly and ask that he review his masscheck scheduling?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
...the Fates notice those who buy chainsaws...
-- www.darwinawards.com
-----------------------------------------------------------------------
3 days until Halloween
Re: Rule updates are too old - 2018-10-26
Posted by Henrik K <he...@hege.li>.
On Sun, Oct 28, 2018 at 06:57:23AM -0500, Dave Jones wrote:
>
> FYI. Still getting the same error on a different line... :)
I think you have to wait till tomorrow for the tagged version to update.
Re: Rule updates are too old - 2018-10-26
Posted by Dave Jones <da...@apache.org>.
On 10/28/18 6:52 AM, Dave Jones wrote:
> On 10/28/18 6:42 AM, Henrik K wrote:
>> On Sun, Oct 28, 2018 at 06:28:38AM -0500, Dave Jones wrote:
>>>
>>> If there is a good repo for upgrading the system perl on a C6 box
>>> correctly, I would do it.
>>
>> It's an age old question. I doubt you'll find any other reliable and
>> secure
>> solution besides compiling it yourself.
>>
>
> In the Python world, the same problem exists and upgrading the system
> Python will break yum. The RackSpace IUS repo has this solved pretty
> well for C6 and C7. RHEL8/C8 will start using dnf so the Python problem
> should go away and Python 3 should (hopefuly) be the default.
>
> I was hoping there was a similar situation for perl where I could add a
> trustworthy repo and get a newer perl I could add to the front of the
> PATH for the masscheck launch. A quick googling around shows I am out
> of luck so I would need to compile from source. I am not against trying
> this out but I generally make it my last resort for maintainability
> reasons.
FYI. Still getting the same error on a different line... :)
I am looking to compile a newer version of perl to resolve this problem
so you may want to back out the last change. I don't want to hold up
moving forward to a newer version of perl as the minimum requirements.
Syncing nightly_mass_check
SVN revision = 1844997
+ ./mass-check --hamlog=ham-ena-week2.log --spamlog=spam-ena-week2.log
-j 8 --progress ham:dir:/home/hamspam/Week2.Ham/cur
spam:dir:/home/hamspam/Week2.Spam/cur
Bareword found where operator expected at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 770, near "s/\t/ /gr"
Bareword found where operator expected at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 778, near "s/\t/ /gr"
syntax error at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 770, near "s/\t/ /gr"
syntax error at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 778, near "s/\t/ /gr"
Re: Rule updates are too old - 2018-10-26
Posted by Dave Jones <da...@apache.org>.
On 10/28/18 6:42 AM, Henrik K wrote:
> On Sun, Oct 28, 2018 at 06:28:38AM -0500, Dave Jones wrote:
>>
>> If there is a good repo for upgrading the system perl on a C6 box
>> correctly, I would do it.
>
> It's an age old question. I doubt you'll find any other reliable and secure
> solution besides compiling it yourself.
>
In the Python world, the same problem exists and upgrading the system
Python will break yum. The RackSpace IUS repo has this solved pretty
well for C6 and C7. RHEL8/C8 will start using dnf so the Python problem
should go away and Python 3 should (hopefuly) be the default.
I was hoping there was a similar situation for perl where I could add a
trustworthy repo and get a newer perl I could add to the front of the
PATH for the masscheck launch. A quick googling around shows I am out
of luck so I would need to compile from source. I am not against trying
this out but I generally make it my last resort for maintainability reasons.
Re: Rule updates are too old - 2018-10-26
Posted by Henrik K <he...@hege.li>.
On Sun, Oct 28, 2018 at 06:28:38AM -0500, Dave Jones wrote:
>
> If there is a good repo for upgrading the system perl on a C6 box
> correctly, I would do it.
It's an age old question. I doubt you'll find any other reliable and secure
solution besides compiling it yourself.
Re: Rule updates are too old - 2018-10-26
Posted by Dave Jones <da...@apache.org>.
On 10/28/18 6:13 AM, Henrik K wrote:
> On Sun, Oct 28, 2018 at 05:43:46AM -0500, Dave Jones wrote:
>>
>> My ena-week2-3 masscheckers have been having this problem below for a couple
>> of months now but I haven't had the time to figure this out.
>
> Why didn't you post this immediately. :-)
>
I have been swamped at my day job for the past 4 months basically not
had a day off in months. Like I said below. This wasn't a problem
until recently when we dropped below the masscheck limits and rule
updates stopped.
>> could be related to all of the work Kevin did to get SA 3.4.2 out based on
>> timing.
> ...
>> near "s/\t/ /gr"
>
> This requires perl 5.14.
>
>> They are running CentOS 6.10 and perl v5.10.1.
>
I would gladly update the perl on those boxes but I can't upgrade the OS
due to some software running on it that doesn't support systemd yet. I
had plans to upgrade this software over the summer but the company I
work for got a new contract in May which has taken up all of my time.
> Masschecking uses trunk, which officially requires perl 5.14 and even
> enforces that during install, so that's what you (we) should use. I don't
> think there are any general problems using older, but you should really
> upgrade so we don't get any unicode related differences between masschecker
> results.
>
If there is a good repo for upgrading the system perl on a C6 box
correctly, I would do it. I try to install RPMs as much as possible
from EPEL for any perl modules but there are some that came from CPAN.
I don't want to break the existing perl program on those 2 VMs but I
would give it a try and rollback the changes if anything became broken.
As I have said before, I am not a perl person so I would have to take
things slowly where I am normally pretty fast.
> In the mean time, I downgraded the wrap function again for pre-5.14 method:
> http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Util.pm?r1=1844277&r2=1845014
>
I noticed. Thank you.
> I think we should try to refrain from using newer perl functions until 4.0.0
> is officially released..
>
Re: Rule updates are too old - 2018-10-26
Posted by Henrik K <he...@hege.li>.
On Sun, Oct 28, 2018 at 05:43:46AM -0500, Dave Jones wrote:
>
> My ena-week2-3 masscheckers have been having this problem below for a couple
> of months now but I haven't had the time to figure this out.
Why didn't you post this immediately. :-)
> could be related to all of the work Kevin did to get SA 3.4.2 out based on
> timing.
...
> near "s/\t/ /gr"
This requires perl 5.14.
> They are running CentOS 6.10 and perl v5.10.1.
Masschecking uses trunk, which officially requires perl 5.14 and even
enforces that during install, so that's what you (we) should use. I don't
think there are any general problems using older, but you should really
upgrade so we don't get any unicode related differences between masschecker
results.
In the mean time, I downgraded the wrap function again for pre-5.14 method:
http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Util.pm?r1=1844277&r2=1845014
I think we should try to refrain from using newer perl functions until 4.0.0
is officially released..
Re: Rule updates are too old - 2018-10-26
Posted by Dave Jones <da...@apache.org>.
On 10/26/18 1:04 PM, Dave Jones wrote:
> On 10/26/18 9:38 AM, John Hardin wrote:
>> On Fri, 26 Oct 2018, darxus@chaosreigns.com wrote:
>>
>>> 20181025: Spam or ham is below threshold of 150,000:
>>> http://ruleqa.spamassassin.org/?daterev=20181025
>>> 20181025: Spam: 96744, Ham: 376566
>>
>> It looks like the llanga and mmiroslaw masschecks are still having
>> timing issues, and this is dropping the cumulative corpora below the
>> publication threshold.
>>
>> mmiroslaw, llanga:
>>
>> Could you check your masscheck schedules and durations again?
>>
>
> I have been having some problems with a few weeks of my masscheck that I
> have not had the time to look into. I was hoping that 2 weeks out of 5
> weeks of the ena corpora wouldn't let this drop below the minimum
> threshold. I will try to look at this deeper but there were a bunch of
> perl errors in my masscheck logs that I don't understand and I am
> swamped at my day job basically working 7 days a week for 3 months now.
>
> Dave
My ena-week2-3 masscheckers have been having this problem below for a
couple of months now but I haven't had the time to figure this out. I
suspect this could be related to all of the work Kevin did to get SA
3.4.2 out based on timing. We still had enough corpora to keep the
rules updating until recently when I lowered my mail flow into my
ena-week0 due to overloading of my mail queues.
SVN revision = 1844997
+ ./mass-check --hamlog=ham-ena-week3.log --spamlog=spam-ena-week3.log
-j 8 --progress ham:dir:/home/hamspam/Week3.Ham/cur
spam:dir:/home/hamspam/Week3.Spam/cur
Bareword found where operator expected at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 770, near "s/\t/ /gr"
Bareword found where operator expected at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 778, near "s/\t/ /gr"
syntax error at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 770, near "s/\t/ /gr"
syntax error at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 778, near "s/\t/ /gr"
BEGIN not safe after errors--compilation aborted at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/Util.pm
line 1928.
Compilation failed in require at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/ArchiveIterator.pm
line 28.
BEGIN failed--compilation aborted at
/home/masscheck/work/nightly_mass_check/masses/../lib/Mail/SpamAssassin/ArchiveIterator.pm
line 28.
Compilation failed in require at ./mass-check line 177.
BEGIN failed--compilation aborted at ./mass-check line 177.
I am not good at perl but I can at least tell this is the Util.pm that
comes down with the masscheck rsync out of the SA SVN. Can anyone tell
me what I need to look for on those 2 VMs masschecking ena-week2 and
ena-week3? They are running CentOS 6.10 and perl v5.10.1. The other 3
masschecker VMs that are still working are running Centos 7.5 and perl
v5.16.3. I appears I need to upgrade the perl on the C6 boxes somehow.
Dave
Re: Rule updates are too old - 2018-10-26
Posted by Dave Jones <da...@apache.org>.
On 10/26/18 9:38 AM, John Hardin wrote:
> On Fri, 26 Oct 2018, darxus@chaosreigns.com wrote:
>
>> 20181025: Spam or ham is below threshold of 150,000:
>> http://ruleqa.spamassassin.org/?daterev=20181025
>> 20181025: Spam: 96744, Ham: 376566
>
> It looks like the llanga and mmiroslaw masschecks are still having
> timing issues, and this is dropping the cumulative corpora below the
> publication threshold.
>
> mmiroslaw, llanga:
>
> Could you check your masscheck schedules and durations again?
>
I have been having some problems with a few weeks of my masscheck that I
have not had the time to look into. I was hoping that 2 weeks out of 5
weeks of the ena corpora wouldn't let this drop below the minimum
threshold. I will try to look at this deeper but there were a bunch of
perl errors in my masscheck logs that I don't understand and I am
swamped at my day job basically working 7 days a week for 3 months now.
Dave