You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@issues.apache.org on 2010/11/06 08:42:26 UTC
[Bug 6511] New: daryl.dostech.ca lacks SA updates, and sa-update
using wrong exit code
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
Summary: daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Product: Spamassassin
Version: 3.3.1
Platform: PC
OS/Version: FreeBSD
Status: NEW
Severity: normal
Priority: P2
Component: sa-update
AssignedTo: dev@spamassassin.apache.org
ReportedBy: jdc@parodius.com
First problem:
For the past few days, intermittently, I've been seeing the following error in
my logs when running sa-update --nogpg. I believe the intermittent nature is
due to the weighting structure in MIRRORED.BY. Timestamps are included below
of when sa-update was run:
Date: Fri, 5 Nov 2010 00:00:12 -0700 (PDT)
http: GET http://daryl.dostech.ca/sa-update/asf/1030858.tar.gz request failed:
404 Not Found: <!DOCTYPE HTML PUBLIC "-//IETF//DTD
HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not
Found</h1> <p>The requested URL
/sa-update/asf/1030858.tar.gz was not found on this server.</p> <hr>
<address>Apache/2.2.6 (Fedora) Server at daryl.dostech.ca Port
80</address> </body></html>
Date: Sat, 6 Nov 2010 00:00:11 -0700 (PDT)
http: GET http://daryl.dostech.ca/sa-update/asf/1031475.tar.gz request failed:
404 Not Found: <!DOCTYPE HTML PUBLIC "-//IETF//DTD
HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not
Found</h1> <p>The requested URL
/sa-update/asf/1031475.tar.gz was not found on this server.</p> <hr>
<address>Apache/2.2.6 (Fedora) Server at daryl.dostech.ca Port
80</address> </body></html>
I can confirm that this server is responsible for some form of rule updates:
$ cat /var/db/spamassassin/3.003001/updates_spamassassin_org/MIRRORED.BY
# test mirror: zone, cached via Coral
#http://buildbot.spamassassin.org.nyud.net:8090/updatestage/
http://daryl.dostech.ca/sa-update/asf/ weight=5
http://www.sa-update.pccc.com/ weight=5
Who runs this server, why is it in the sa-update list when it doesn't have
proper packages, why are updates on a host that's maintained by a
publicly-proclaimed "Lazy Network Engineer"[1], and why are recent updates
missing?
Second problem, as a result of the first:
sa-update is supposed to return a non-zero exit code when downloading updates
fails. Yet when the above situation occurs, the exit code is zero (it should
not be), which causes our updater cronjob script to restart spamd for no
justified reason.
Here's the relevant /bin/sh code:
/usr/local/bin/sa-update --nogpg || rc=$?
if [ $rc -eq 0 ]; then
echo "Downloaded and installed new rules."
echo
echo "Restarting sa-spamd:"
echo
/usr/local/etc/rc.d/sa-spamd restart
elif [ $rc -eq 1 ]; then
echo "No new rules available."
rc=0
else
echo "sa-update failed with exit code $rc. Please refer to the
sa-update(1)"
echo "man page for details."
fi
The sa-update(1) man page states the following:
EXIT CODES
An exit code of 0 means an update was available, and was downloaded and
installed successfully if --checkonly was not specified.
An exit code of 1 means no fresh updates were available.
An exit code of 2 means that at least one update is available but that
a lint check of the site pre files failed. The site pre files must
pass a lint check before any updates are attempted.
An exit code of 4 or higher, indicates that errors occurred while
attempting to download and extract updates.
Please investigate both of these problems. I can provide full logs or whatever
other details are necessary.
[1]: http://daryl.dostech.ca/
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #4 from Karsten Bräckelmann <gu...@rudersport.de> 2010-11-06 15:09:01 UTC ---
(In reply to comment #2)
> Regarding the mirror being broken: I've added spamassassin@dostech.ca to the CC
> list in hopes that this can be investigated.
While that's perfectly legit and generally precisely the thing to do with
bugzilla, in this case, he gets a copy on the dev list anyway. :)
> Regarding the exit code 0 situation -- I've reviewed the sa-update code, and I
> see exactly how this situation is occurring. It's actually by design.
Ah, good one! I've seen a 404 only with third-party channels so far, single
mirror. That explains this.
> Regarding Daryl and the daryl.dostech.ca mirror -- it's not personal, it's just
> that I can't find any mention of this mirror in the SA documentation or on the
> SA Wiki. These sorts of things should be more public. It also doesn't reflect
> well on the Apache Foundation when you have a mirror which appears to be broken
> and maintained by someone who's self-proclaimed "lazy" (which in turn makes the
> visitor wonder, in this circumstance, just how valid that claim is), even if he
> is a chairman.
While I kind of understand that argument, and the mirrors probably could be
documented better...
This is open source, and donated resources in this case. An OSS developer's
*personal* site or blog must not be confused with the project.
> I'm sorry if it sounds like I'm overreacting, but given the situation and the
> circumstances, my first inclination was to assume someone had circumvented one
> of the SA mirrors and induced an HTTP redirect to some "mystery" box that had
> been compromised [...]
Why do you run sa-update with the --nogpg option? With the default of using GPG
there would have been no reason to worry. Skipping that security measure was
your deliberate decision.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
Humor
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
Comment #5 from Daryl C. W. O'Shea
> 5. If only I was so lazy... I wouldn't need as much humour to get through life.
+1
Sarcasm is a form of humor. And it appears to be rather widespread among
active open source community members. You got to deal with the situation
somehow...
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
[Bug 6511] daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #5 from Daryl C. W. O'Shea <sp...@dostech.ca> 2010-11-06 16:00:17 UTC ---
1. Did this problem actually cause some sort of damage, beyond triggering a
restart (that would have happened if there was an update anyway)?
2. Why are people running sa-update with --nogpg? Mirror compromise is a
legitimate concern (for any type of mirror, not just sa-update ones) that is
easily avoidable. Hopefully people aren't also running this with the
--allowplugins option too.
3. The issue has been resolved. Thank you for bringing it to our attention. I
think mirror monitoring is something we need to start doing. I've opened bug
6512 about this.
4. Do I wish this didn't happen? Yes. Could I have caught it faster? Maybe,
if I wasn't so busy working to pay for stuff like food and shelter. Let alone
material crap from my Amazon wishlist that so far in the course of the better
part of a decade I've only received one item from...
http://www.amazon.com/o/registry/2UUNX1ZJ2Y6S4 ...there's a good opportunity
for somebody to be generous person #2.
5. If only I was so lazy... I wouldn't need as much humour to get through life.
I'll leave this bug open for discussion about the return codes.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
Re: What are our mirrors?
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 06/11/2010 4:59 PM, Karsten Bräckelmann wrote:
>> [...] I can't find any mention of this mirror in the SA documentation or on the
>> SA Wiki. These sorts of things should be more public. [...]
>
> After stripping all the mixing up between project volunteers and donated
> resources on the one hand, and those people's personal life...
>
> I guess that's a valid point.
>
> What are those mirrors? Who runs them? Clearly, it's not SA, neither the
> ASF. Yes, it is obvious to me who is behind those servers, as it
> probably is to most folks on this list. But it is not that obvious to a
> system administrator not closely involved in the project.
Perhaps info could be added to
http://wiki.apache.org/spamassassin/RuleUpdates
Daryl
Re: What are our mirrors?
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
> $ host -t TXT mirrors.updates.spamassassin.org
> mirrors.updates.spamassassin.org descriptive text
> "http://spamassassin.apache.org/updates/MIRRORED.BY"
>
> $ lynx -source http://spamassassin.apache.org/updates/MIRRORED.BY | grep -v '^#'
> http://daryl.dostech.ca/sa-update/asf/ weight=5
> http://www.sa-update.pccc.com/ weight=5
>
>
Perhaps switch the DNS to cnames such as mirror1.spamassassin.apache.org
so there is another level of control and veracity? And if a mirror goes
done, DNS can be used to re-route things instead of just the MIRRORED.BY
file.
regards,
KAM
What are our mirrors? (was: Bug 6511)
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
> [...] I can't find any mention of this mirror in the SA documentation or on the
> SA Wiki. These sorts of things should be more public. [...]
After stripping all the mixing up between project volunteers and donated
resources on the one hand, and those people's personal life...
I guess that's a valid point.
What are those mirrors? Who runs them? Clearly, it's not SA, neither the
ASF. Yes, it is obvious to me who is behind those servers, as it
probably is to most folks on this list. But it is not that obvious to a
system administrator not closely involved in the project.
FWIW, the first step is transient to the users. It isn't even obvious to
them, that the MIRRORED.BY file is hosted on ASF. All they usually get
to see is the contents, in their local copy. Unless there's an issue,
and the URL pops up in their cron mail.
$ host -t TXT mirrors.updates.spamassassin.org
mirrors.updates.spamassassin.org descriptive text
"http://spamassassin.apache.org/updates/MIRRORED.BY"
$ lynx -source http://spamassassin.apache.org/updates/MIRRORED.BY | grep -v '^#'
http://daryl.dostech.ca/sa-update/asf/ weight=5
http://www.sa-update.pccc.com/ weight=5
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
[Bug 6511] daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
Jeremy Chadwick <jd...@parodius.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jdc@parodius.com,
| |spamassassin@dostech.ca
--- Comment #2 from Jeremy Chadwick <jd...@parodius.com> 2010-11-06 14:22:36 UTC ---
Regarding the mirror being broken: I've added spamassassin@dostech.ca to the CC
list in hopes that this can be investigated.
Regarding the exit code 0 situation -- I've reviewed the sa-update code, and I
see exactly how this situation is occurring. It's actually by design.
sa-update gets a list of mirrors (from MIRRORED.BY or via the channel), picks a
random mirror, and attempts to fetch the applicable .tar.gz from the mirror 3
times. If all 3 HTTP GET attempts fail, the code spits out a warning (the
error message shown above) and moves on to the next mirror in the hash.
The relevant code bits are between lines 617 and 649. Also be sure to look at
the http_get() function.
So the logic of the code seems to be "as long as one of the mirrors worked/has
the content we need, exit code 0 is valid". This makes sense, but the man page
could use some added clarification. My recommendation would be to improve the
sa-update man page from:
An exit code of 0 means an update was available, and was downloaded and
installed successfully if --checkonly was not specified.
...to:
An exit code of 0 means an update was available from at least one
mirror and was downloaded and installed successfully if --checkonly
was not specified. A warning will be output ("http: GET request
failed") if, after 3 repeated fetch attempts, a mirror lacks the
update or returns a non-200 HTTP status code.
Regarding Daryl and the daryl.dostech.ca mirror -- it's not personal, it's just
that I can't find any mention of this mirror in the SA documentation or on the
SA Wiki. These sorts of things should be more public. It also doesn't reflect
well on the Apache Foundation when you have a mirror which appears to be broken
and maintained by someone who's self-proclaimed "lazy" (which in turn makes the
visitor wonder, in this circumstance, just how valid that claim is), even if he
is a chairman.
I'm sorry if it sounds like I'm overreacting, but given the situation and the
circumstances, my first inclination was to assume someone had circumvented one
of the SA mirrors and induced an HTTP redirect to some "mystery" box that had
been compromised -- or that the system administrator isn't paying attention to
failures (or the failures are unbeknownst to him). I hope you understand my
POV, as I do understand and accept yours.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] mirror lacks SA updates
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
Karsten Bräckelmann <gu...@rudersport.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Platform|PC |All
Resolution| |FIXED
Summary|sa-update using wrong exit |mirror lacks SA updates
|code when a mirror is |
|missing an update |
OS/Version|FreeBSD |All
--- Comment #8 from Karsten Bräckelmann <gu...@rudersport.de> 2010-11-12 19:12:45 UTC ---
Restoring summary as per the previous discussion. Closing RESOLVED FIXED.
Thanks for the fast report about the broken mirror.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] sa-update using wrong exit code when a mirror is missing
an update
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
Daryl C. W. O'Shea <sp...@dostech.ca> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|daryl.dostech.ca lacks SA |sa-update using wrong exit
|updates, and sa-update |code when a mirror is
|using wrong exit code |missing an update
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #3 from Daryl C. W. O'Shea <sp...@dostech.ca> 2010-11-06 14:59:27 UTC ---
Wow. OK. I was not aware of the issue. I am looking into it now.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] daryl.dostech.ca lacks SA updates, and sa-update using
wrong exit code
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #1 from Karsten Bräckelmann <gu...@rudersport.de> 2010-11-06 12:56:44 UTC ---
(In reply to comment #0)
> For the past few days, intermittently, I've been seeing the following error in
> my logs when running sa-update --nogpg. I believe the intermittent nature is
> due to the weighting structure in MIRRORED.BY.
Yes.
> I can confirm that this server is responsible for some form of rule updates:
>
> $ cat /var/db/spamassassin/3.003001/updates_spamassassin_org/MIRRORED.BY
Your local copy, which might get stale if mirrors change (a known bug). Not the
problem here, it does match the master.
host -t TXT mirrors.updates.spamassassin.org
> Who runs this server, why is it in the sa-update list when it doesn't have
> proper packages, why are updates on a host that's maintained by a
> publicly-proclaimed "Lazy Network Engineer"[1], and why are recent updates
> missing?
Oh, relax, Jeremy.
The mirror is run by Daryl, a long time SA developer and currently serving as
chair or the SA PMC. Shouldn't have been that hard to find out, if you would
not have focussed on the humorous self attribution of his personal website.
That mirror is in the list, because it's serving as an sa-update mirror,
donating bandwidth to all SA users out there. The update is missing, because
something broke. Apparently. Which is, why you filed this bug...
> sa-update is supposed to return a non-zero exit code when downloading updates
> fails. Yet when the above situation occurs, the exit code is zero (it should
> not be), which causes our updater cronjob script to restart spamd for no
> justified reason.
This is weird, I've never seen sa-update return an exit code of 0 in case of a
404 error.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] sa-update using wrong exit code when a mirror is missing
an update
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #6 from Karsten Bräckelmann <gu...@rudersport.de> 2010-11-06 16:29:20 UTC ---
(In reply to comment #5)
> 1. Did this problem actually cause some sort of damage, beyond triggering a
> restart (that would have happened if there was an update anyway)?
Not an authoritative answer, which only Jeremy could provide, but...
It is my understanding there actually was *no* unnecessary restart, but a
restart due to an exit code 0, success -- warranted.
Comment 0 shows two incidents, both with different channel version numbers. And
in comment 2 Jeremy points out after digging through the code, that after the
first mirror failing, sa-update fell back to the next one, grabbed the updated
rules tarball, and correctly exited with 0.
This of course assumes there's been only one such incident per channel version
number. Jeremy, is that correct? No mention of both mirrors failing. And if
there would not have been an update, the process would have stopped right after
the DNS query, never even attempting to get to a mirror.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6511] sa-update using wrong exit code when a mirror is missing
an update
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6511
--- Comment #7 from Karsten Bräckelmann <gu...@rudersport.de> 2010-11-06 16:33:37 UTC ---
My comment 6 just previously, if confirmed by the reporter, also implies that
the Summary change is actually incorrect. There was no wrong exit code.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.