Upgrading from 2.63 to 3.1.8

[Jean-Michel Barbet <Je...@subatech.in2p3.fr>]

Hello,

I am in the process of upgrading SA from version 2.63 to 3.1.8.
I have done it on a test machine for a few mail adresses including
mine and I came across several problems. I am running spamd and
spamc is invoqued from procmail in /etc/procmailrc (system-wide
invocation)

1) spamd stopped analyzing mails after some time. I found the following
    message in /var/log/maillog :

    spamd[1958]: prefork: select returned -1! recovering: Bad file
    descriptor

    I found this problem is described here :

    http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4590
    http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5313
    http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5422

    I applied the patch from 5313 by downloading an nightly build
    from CVS, extracting the two modules : SubProcBackChannel.pm
    and SpamdForkScaling.pm and putting them in place.

    => this problem seems to be cured and I hope version 3.2.0 will
       not show the problem.

2) Since problem 1 is fixed, I observed at least once that I had an
    extra spamd child owned by a user and that this spamd child were
    busy with the ~/.spamassassin/bayes_toks.
    A lockfile ~/.spamassassin/bayes_lock was in place with 2 identical
    lines containing the PID of this spamd child.
    A message could be found in /var/log/message :

    spamd[17207]: bayes: bayes db version 2 is not able to be used,
    aborting! at
/usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/BayesStore/DBM.pm line 
196, <GEN239> line 954.

    This spamdd process was apparently stuck because it stayed for
    hours. Then following an action that I forgot, the lockfile
    disapeared but a ~/.spamassassin/.nfs000xxxxxxx appeared.
    Only when I manually killed the spamd child, this .nfsxxxx
    file vanished.

    I know the format of the bayes DB has changed but as I understood,
    the conversion is supposed to be performed by SA itself when
    opening the DB R/W. It does not seem to work well.

    => Any experiences of a migration of the bayes DB files when
       upgrading from 2.6x to 3.1.x ?

    => What is the best way to deal with this ?

Thank you all.

Jean-Michel BARBET.

-- 
------------------------------------------------------------------------
Jean-michel BARBET                    | Tel: +33 (0)2 51 85 84 86
Laboratoire SUBATECH Nantes France    | Fax: +33 (0)2 51 85 84 79
CNRS-IN2P3/Ecole des Mines/Universite | E-Mail: barbet@subatech.in2p3.fr
------------------------------------------------------------------------

Re: Upgrading from 2.63 to 3.1.8

[Jean-Michel Barbet <Je...@subatech.in2p3.fr>]

Matt Kettler wrote:

> Yes, it's covered in the UPGRADE file
[..]
> sa-learn --sync

Thanks Mark,

Meanwhile, I discovered that a few messages were still delivered
and filtered by SA 2.6 on another mail delivery server and I think
that's what was causing trouble. I made sure mail was coming through
SA 3.1.8 only and I ran the following script on behalf of concerned
users (procmail tests the presence of ~/.nofilterspam before calling
spamc for users who do not want SA and I use this to prevent SA
interfering during the conversion) :

#!/bin/sh
#
if [ -f ~/.nofilterspam ] ; then
   echo "SPAM filtering disabled by ~/.nofilterspam"
else
   touch ~/.nofilterspam
   if /usr/bin/sa-learn -D --sync
   then
      echo "Conversion to bayes DB3 was successful"
   else
      echo "Conversion to bayes DB3 FAILED !"
      rm  ~/.nofilterspam
      exit 1
   fi
   rm  ~/.nofilterspam
fi
exit 0

Problem is that it seems I would have to run this script for 200 or so
users after I put SA 3.1.8 in prod on our main server...

Thanks.

JM


-- 
------------------------------------------------------------------------
Jean-michel BARBET                    | Tel: +33 (0)2 51 85 84 86
Laboratoire SUBATECH Nantes France    | Fax: +33 (0)2 51 85 84 79
CNRS-IN2P3/Ecole des Mines/Universite | E-Mail: barbet@subatech.in2p3.fr
------------------------------------------------------------------------

Re: Upgrading from 2.63 to 3.1.8

[Matt Kettler <mk...@verizon.net>]

Jean-Michel Barbet wrote:
>
>
>    => Any experiences of a migration of the bayes DB files when
>       upgrading from 2.6x to 3.1.x ?
Yes, it's covered in the UPGRADE file
>
>    => What is the best way to deal with this ?
sa-learn --sync
>
> Thank you all.
>
> Jean-Michel BARBET.
>