[Bug 6930] New: sa-update unable to deal with a previous failure (empty mirrors file)

[bu...@bugzilla.spamassassin.org]

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6930

            Bug ID: 6930
           Summary: sa-update unable to deal with a previous failure
                    (empty mirrors file)
           Product: Spamassassin
           Version: SVN Trunk (Latest Devel Version)
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: sa-update
          Assignee: dev@spamassassin.apache.org
          Reporter: Mark.Martinec@ijs.si
    Classification: Unclassified

(opening a new PR for documentation purposes, keeping issues apart)

Quanah Gibson-Mount reported on a mailing list (2013-04-02):

$ ./sa-update --updatedir=/tmp/qsa
error: no mirrors available for channel updates.spamassassin.org
channel: no mirrors available, channel failed
$ cd /tmp/qsa/updates_spamassassin_org/
$ cat MIRRORED.BY
$


Kevin A. McGrail wrote:
>> Can you rename the MIRRORED.BY file in
>> /tmp/qsa/updates_spamassassin_org/MIRRORED.BY and try again?
>>
>> Is the MIRRORED.BY file blank?
> Yes, it was zero bytes.
>
> And now that I removed it, it worked?!  Crazy. :)
>
> I think the first time I failed to set my proxy before running 
> sa-update, and then subsequent runs all failed.  Perhaps if 
> MIRRORED.BY is 0, it should be removed by sa-update before trying to 
> retrieve it again?

Good to hear.

I thought the following should fix it:

--- sa-update.raw       (revision 1452851)
+++ sa-update.raw       (working copy)
@@ -596,6 +596,9 @@
        } elsif (time - $mirby_time > $IGNORE_MIRBY_OLDER_THAN) {
          dbg("channel: MIRRORED.BY file is too old, forcing refresh");
          $mirby_force_reload = 1;
+      } elsif (-s $mirby_path == 0) {
+        dbg("channel: MIRRORED.BY file is zero bytes, forcing refresh");
+        $mirby_force_reload = 1;
        } else {
          dbg("channel: reading MIRRORED.BY file $mirby_path");
          local $/ = undef;

However, it didn't.

I found that this command (effectively) wouldn't overwrite a

  cd /var/lib/spamassassin/3.004000/updates_spamassassin_org/ && 
/usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 
30 --max-time 300 --fail -o MIRRORED.BY -z MIRRORED.BY -- 
http://spamassassin.apache.org/updates/MIRRORED.BY

I think the -z is saying it has to be newer than the existing file.

So I added an unlink to the code for the MIRRORED.BY file if it's 0 
bytes.  Mark, do you concur?  You know sa-update like the back of your hand!

svn commit -m 'Fix for MIRRORED.BY file being 0 byte' sa-update.raw
Sending        sa-update.raw
Transmitting file data .
Committed revision 1463659.




me writes (in an unrelated Bug 6654):

More importantly (regarding sa-update) is the problem reported
on the mailing list by Quanah Gibson-Mount on 2013-04-02,
where sa-update can get in a state (following a previous failure)
unable to resolve without deleting some file. I think Kevin
already committed one fix, although it seems to me it could be
further improved (finding the original program logic somewhat
unusual - like noticing the mirrors file is stale, using it
for an update, and only updating the mirrors file later on).
I'll see what can be done - it's not my familiar territory.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 6930] sa-update unable to deal with a previous failure (empty mirrors file)

[bu...@bugzilla.spamassassin.org]

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6930

Kevin A. McGrail <km...@pccc.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |kmcgrail@pccc.com
         Resolution|---                         |FIXED

--- Comment #2 from Kevin A. McGrail <km...@pccc.com> ---
Closing as resolved.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 6930] sa-update unable to deal with a previous failure (empty mirrors file)

[bu...@bugzilla.spamassassin.org]

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6930

Mark Martinec <Ma...@ijs.si> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|Undefined                   |3.4.0

--- Comment #1 from Mark Martinec <Ma...@ijs.si> ---
Ok, I did some reshuffling of the code. It should now be able
to deal with empty or missing mirrors file;  avoids using an
old mirrors file for rule updates and only later updating the
mirrors file;  and avoids the problem where a curl download is
setting a timestamp on a downloadad file to a server's time,
which caused sa-update to always consider such file old and
refreshed it every time.

trunk:
  Bug 6930: sa-update unable to deal with a previous failure (empty mirrors
file)
Sending sa-update.raw
Committed revision 1475932.


This is a somewhat hefty diff (also reordered two conditionals
to have a short-code branch first for easier reading).

I did test several scenarios, but I'd still appreciate
a second eye on testing unusual situations.

-- 
You are receiving this mail because:
You are the assignee for the bug.