Change .spamassassin directory

[Neil Hodge <ne...@gmail.com>]

All:

Just wondering how to change the user-local prefs directory
.spamassassin.  I have tried all combinations of commands and options,
including

/usr/bin/spamassassin --configpath=/path/to/saprefs,

but I keep getting the response

"config: no rules were found!  Do you need to run 'sa-update'? at
/usr/bin/spamassassin line 403."

Neither does the symbolic link

.spamassassin -> /path/to/saprefs

work, i.e. it keeps getting changed to an actual directory.  I am
running SA 3.3.1 from within procmail.  Any ideas???  Thanks.

Neil Hodge

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 13:37 -0700, Ted Mittelstaedt wrote:
> On 7/22/2010 12:59 PM, Karsten Bräckelmann wrote:

> I can only report what I've seen - I don't use comca$t myself and I
> don't have a lot of direct experience with it.

Obviously, neither do I. :)

> > I believe we where not talking about outgoing SMTP.
> 
> I know, but you advised running your own mailserver (good advice)
> and doing your own outbound mail is part of the process IMHO.

Oh!  Nah, actually I just said *preferably* adding your own SMTP, and
meant "as a buffer between fetchmail and procmail", which I found to be
really stable. As opposed to using the MDA option of fetchmail and
skipping the default of using a local SMTP.

With a home-user setup, I would never use the SMTP for outbound unless
relaying.

> I wouldn't put dynamic space on any business that was running their
> own mailserver in the first place.  That is why I said "home mailserver"

Indeed. As I understand this thread, this is purely about home usage.
And even then, I would never use the local SMTP on dynamic IP for direct
delivery of outbound mail.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Ted Mittelstaedt <te...@ipinc.net>]

On 7/22/2010 12:59 PM, Karsten Bräckelmann wrote:
> On Thu, 2010-07-22 at 12:45 -0700, Ted Mittelstaedt wrote:
>> A lot of people have gotten home mailservers running on so-called
>> "dynamic" IP addresses - when they have discovered that Comca$t will
>> continually approve DHCP re-lease requests for the same IP address.
>>
>> Often people can go for 4-6 months at a time with the same
>> "dynamic" IP address if the gear is on a UPS
>>
>> Just setup your stuff, get an IP number, then query DNS and see
>> what the PTR record for the IP address is, then name your server
>> the same as that PTR name, and in DNS setup that name as an MX
>> host.
>>
>> If the IP changes then modify the hostname to the new PTR name and
>> modify the MX in DNS and restart the mail process on the mailserver.
>
> Whee, dyndns sounds far easier than that fragile hack.
>
>> Of course you can do the dydns.org route but then your PTR will
>> not match the hostname and some servers out there will not accept
>> mail from you if that is the case.
>
> Wait, did you just say local SMTP delivering directly?

Yes.

> Comcast dial-up
> dynamic end-user space is NOT listed on PBL?
>

My experience is that it isn't.  I'm sure they have SOME of it's 
"dynamic" space but Comca$t seems to assign static IP's for comcast
at work customers right out of the same netblocks.

Sure, you can relay outbound smtp through comcast's mailserver - if they
allow it.  Do they?  I know Verizon doesn't allow outbound
mail to be relayed through it's server unless it's a @verizon.net address.

I can only report what I've seen - I don't use comca$t myself and I
don't have a lot of direct experience with it.

> I believe we where not talking about outgoing SMTP.

I know, but you advised running your own mailserver (good advice)
and doing your own outbound mail is part of the process IMHO.

> That unrelated
> setting can stay unchanged in the MUAs. The address appears to be
> handled by Comcast anyway, so the SMTP would not be an MX. And from my
> understanding, it is unlikely Neil has control over the domain's DNS.
>
> If I would run an SMTP on dynamic space, I'd have it relay via my ISP's
> SMTP. Or my own server hosted somewhere else. :)
>

I wouldn't put dynamic space on any business that was running their
own mailserver in the first place.  That is why I said "home mailserver"

Setting up "home" DSL and Cable connections for businesses is cheating
and I would not take advice from people who advise such.  It's my
understanding Comca$t does not do statics for residential customers only
businesses under the comca$t at work product.

Ted

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 12:45 -0700, Ted Mittelstaedt wrote:
> A lot of people have gotten home mailservers running on so-called
> "dynamic" IP addresses - when they have discovered that Comca$t will
> continually approve DHCP re-lease requests for the same IP address.
> 
> Often people can go for 4-6 months at a time with the same
> "dynamic" IP address if the gear is on a UPS
> 
> Just setup your stuff, get an IP number, then query DNS and see
> what the PTR record for the IP address is, then name your server
> the same as that PTR name, and in DNS setup that name as an MX
> host.
> 
> If the IP changes then modify the hostname to the new PTR name and
> modify the MX in DNS and restart the mail process on the mailserver.

Whee, dyndns sounds far easier than that fragile hack.

> Of course you can do the dydns.org route but then your PTR will
> not match the hostname and some servers out there will not accept
> mail from you if that is the case.

Wait, did you just say local SMTP delivering directly? Comcast dial-up
dynamic end-user space is NOT listed on PBL?

I believe we where not talking about outgoing SMTP. That unrelated
setting can stay unchanged in the MUAs. The address appears to be
handled by Comcast anyway, so the SMTP would not be an MX. And from my
understanding, it is unlikely Neil has control over the domain's DNS.

If I would run an SMTP on dynamic space, I'd have it relay via my ISP's
SMTP. Or my own server hosted somewhere else. :)


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Ted Mittelstaedt <te...@ipinc.net>]

On 7/22/2010 11:51 AM, Neil Hodge wrote:
> Karsten:
>
> 2010/7/22 Karsten Bräckelmann<gu...@rudersport.de>:
>>
>> So, did you just say, that you check mail for one account (you mentioned
>> a single ISP only) from two different places, namely home and work? That
>> scenario spells IMAP to me -- your *own* IMAP server at home.
>>
>> That way, you will need a single SA only, and a single server to
>> maintain.
>>
>
> Well, I hadn't really expected to find anyone on the list who is
> willing to be so helpful, but since you are . . . :)
>
> Basically, your understanding of my description is correct.  Of
> course, my own first thought when I started thinking about this was
> precisely what you just suggested.  The problem with that is that I am
> with one of the big commercial ISPs (Comcast), and I don't have a
> static IP address, so I can't point to my router from any arbitrary
> machine on the internet.  I do believe that I can do some kind of
> dynamic DNS with my router.  Then I can probably pass incoming
> connections to port (whatever for imap) through to my home machine.
> At some point, it just seems that the "right" solution is too much
> hassle, so I was just looking for a lower energy solution.
>

A lot of people have gotten home mailservers running on so-called
"dynamic" IP addresses - when they have discovered that Comca$t will
continually approve DHCP re-lease requests for the same IP address.

Often people can go for 4-6 months at a time with the same
"dynamic" IP address if the gear is on a UPS

Just setup your stuff, get an IP number, then query DNS and see
what the PTR record for the IP address is, then name your server
the same as that PTR name, and in DNS setup that name as an MX
host.

If the IP changes then modify the hostname to the new PTR name and
modify the MX in DNS and restart the mail process on the mailserver.

Of course you can do the dydns.org route but then your PTR will
not match the hostname and some servers out there will not accept
mail from you if that is the case.

Ted

> I could completely believe that the solution I came up with might have
> some fatal flaw, but I thought I would give it a shot first . . .
>
> Neil
>

Re: Change .spamassassin directory

[RW <rw...@googlemail.com>]

On Thu, 22 Jul 2010 12:55:12 -0700
Ted Mittelstaedt <te...@ipinc.net> wrote:


> The biggest downside IMHO is that you lose functionality on the RBL
> 
> For example at 9am spammer spews.  At 9:15 spammer is listed in RBL.
> At 10:0m spammer mails you and your ISP picks up the mail because they
> aren't using RBLs.  

Many do though, in which case there's no downside.

> At 11:00 am the spammer's admin shuts the spammer
> down and delists the IP from the RBL.  At 2:00pm you come along and
> fetch your mail, and scan all the headers, checking all the IPs in
> RBLs and the spam's IPs are not in the RBL then, even though they were
> earlier.


That could happen, but it can also work the other way around and the
address gets listed in the delay. You can also pick-up extra hits
on URIBLs, pyzor  etc. In my experience a few hours delay is beneficial.

Re: Change .spamassassin directory

[Martin Gregorie <ma...@gregorie.org>]

On Thu, 2010-07-22 at 22:48 +0200, Karsten Bräckelmann wrote:
> Interesting. After years of fetchmail usage, and way above single-figure
> millions of messages fetched (just a very conservative estimate), I have
> yet to find even one mail stuck on a server.
> 
It worked well for me for a couple of years, but then the unpurged
message rate rose dramatically about a year ago. When it was
consistently getting to tens of messages a week I moved. Tis was
probably some arcane interaction between it, my ISP's POP3 server and my
broadband provider and/or BT's pipes. I must admit this is about the
only really obnoxious bug I found in fetchmail - the others are a
documentation bug (I never did work out how to make it work in
multi-user mode) and an odd shutdown bug: "service fetchmail stop"
didn't work.

> Well, I won't go down the route of arguing fetchmail, getmail, or
> alternatives. I don't mind. :)
> 
I wasn't intending to start a war - just provide a heads-up for anybody
else having fetchmail problems. If you look, you can find grousing about
undefined bugs that Eric Raymond allegedly refuses to fix, but I have no
idea whether these are genuine or just bitching by somebody with a
grudge.
 
> However, I know SA does understand fetchmail headers, and correctly
> re-starts header parsing. I guess it is the same with getmail, or does
> it simply not add any header?
> 
It adds a 'Received ..... with POP3' header, but you have to know this
was added by getmail because it doesn't identify itself.


Martin

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 21:24 +0100, Martin Gregorie wrote:
> On Thu, 2010-07-22 at 12:55 -0700, Ted Mittelstaedt wrote:
> > The truth is that ETRN is the way your supposed to do this kind of 
> > thing, fetchmail is a hack.  But even ETRN is not as good as your
> > own server.
> 
> And fetchmail is fairly buggy. When I was using it I found that if it
> got interrupted, e.g. by a remote MTA timeout toward the end of a
> session, it would leave a left a lot of mail on the remote MTA that it
> had already fetched but could never be persuaded to delete, probably
> because it saves up all its purges until the bitter end of the session. 

Interesting. After years of fetchmail usage, and way above single-figure
millions of messages fetched (just a very conservative estimate), I have
yet to find even one mail stuck on a server.


> IMO getmail is a lot better solution. It never does fetchmail's
> abandoned mail trick and, after about a year's use, I haven't found any
> other problems with it.

Well, I won't go down the route of arguing fetchmail, getmail, or
alternatives. I don't mind. :)

However, I know SA does understand fetchmail headers, and correctly
re-starts header parsing. I guess it is the same with getmail, or does
it simply not add any header?


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Martin Gregorie <ma...@gregorie.org>]

On Thu, 2010-07-22 at 12:55 -0700, Ted Mittelstaedt wrote:
> The truth is that ETRN is the way your supposed to do this kind of 
> thing, fetchmail is a hack.  But even ETRN is not as good as your
> own server.
> 
And fetchmail is fairly buggy. When I was using it I found that if it
got interrupted, e.g. by a remote MTA timeout toward the end of a
session, it would leave a left a lot of mail on the remote MTA that it
had already fetched but could never be persuaded to delete, probably
because it saves up all its purges until the bitter end of the session. 

IMO getmail is a lot better solution. It never does fetchmail's
abandoned mail trick and, after about a year's use, I haven't found any
other problems with it.


Martin

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 16:10 -0700, Ted Mittelstaedt wrote:
> On 7/22/2010 2:18 PM, Karsten Bräckelmann wrote:

> > Ted, I believe you're thinking too large-scale.
> >
> > I may be wrong, but I understand Neil is talking about a single, ISP
> > provided email address.
> 
> OK, I missed that, I was assuming he had a vanity domain, (.us or some
> such)
> 
> In that case then yes, your correct, the fetchmail/getmail/etc. solution
> is probably the only way to hack around it.
> 
> Of course, I would point out that if your ISP is doing a poor
> job of scanning for spam, that your REWARDING your ISP by continuing
> to maintain an e-mail address on their mailserver - thus INSURING that
> they have NO incentive to do a better job.  [...]

Heh. Really? ISPs filter spam? ;)  By the time I finally suspended my
ISP provided email address few years ago (rarely used, and easy to get
rid of), there was absolutely no spam filtering.

My own domain's server (vhosted at that time, and pretty much incapable
of handling the load) is another legacy story of it's own. But I quickly
showed that I *can* filter more spam than any professional service even
dares to advertise. Yes, using that fetchmail trick.


> As for the "freemail" servers, well as they are giving it to you free, 

One thing I forgot to mention earlier: I am running such a home solution
for friends of mine. Well, frankly, I got ssh access, and I set up the
server, but since then there's hardly any "running" or administration
needed. Anyway...

One of the addresses is a freemail address. And we deliberately opted
out of their spam filtering. For two reasons,  (a) to have no need for
the web interface to check for FPs, and  (b) to feed Bayes, because SA
now gets the full spectrum of spam.

Needless to say how happy she is. I already mentioned I don't have to
actively administrate the box. ;)


> you can't really complain. ;-)  I guess what's going on here is few
> pennies you are saving on paying for decent mail, your spending
> dollars of our own time cobbling together a workaround.  To each
> his own!

Since one such comment of mine got most of this thread rolling...

Another thing I realized early when supporting users is, not to solve
their issue and move on. Do explain the situation, the issue, and
educate the user. It will pay back.

Seriously, if you properly discuss an issue, explain the basics, and
provide solutions, you will generate enough "buzz words" for google to
pick it up. With time, any (common) issue will cease to be an FAQ. No
one is asking it any more, because they easily found the solution in
list archives asking a search engine.

Why did I bring it up in this context? Right. Because, yes, I did spend
considerable time on this thread. But in the end I believe, other folks
too will find this solution. And we will not have to repeat it again,
spending more resources.


Oh, and course that trade-off was not deliberate on the OPs side. He
realized, oh, there's SA, that can solve my problem! So, let me use
that. Hmm, how exactly do I integrate it? Maybe this way would do, but
now I'm running into an issue. Hey, they got a user's list. Let me ask
experienced folks...

That's what this list is for, no? :)

  guenther


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Ted Mittelstaedt <te...@ipinc.net>]

On 7/22/2010 2:18 PM, Karsten Bräckelmann wrote:
> On Thu, 2010-07-22 at 13:48 -0700, Ted Mittelstaedt wrote:
>> On 7/22/2010 1:13 PM, Karsten Bräckelmann wrote:
>
>>> Yes, I do maintain some home systems like that. With freemailers or ISP
>>> accounts it often isn't possible any other way. Polling interval of a
>>> minute or two.
>>
>> And if everyone polled every minute or 2 then the server would melt
>> down.  You should be polling every 5 or better yet every 10.  But in
>> any case why are you advocating a hack like this?  Your last post
>> advocated NOT a hack but in doing it right.  In every instance of a
>> fetchmail setup the biggest loss is that you cannot issue an error
>> 5xx to a spammer.
>
> Ted, I believe you're thinking too large-scale.
>
> I may be wrong, but I understand Neil is talking about a single, ISP
> provided email address.

OK, I missed that, I was assuming he had a vanity domain, (.us or some
such)

In that case then yes, your correct, the fetchmail/getmail/etc. solution
is probably the only way to hack around it.

Of course, I would point out that if your ISP is doing a poor
job of scanning for spam, that your REWARDING your ISP by continuing
to maintain an e-mail address on their mailserver - thus INSURING that
they have NO incentive to do a better job.  And yes I know that in
some areas the ISP is a monopoly.  In the United States we unfortunately
have an FCC that has voted to NOT regulate broadband providers (a
legacy of our last good-for-nothing President and his deregulating
ways, which precipitated the current economic collapse, but that's
another story)

As for the "freemail" servers, well as they are giving it to you free, 
you can't really complain. ;-)  I guess what's going on here is few
pennies you are saving on paying for decent mail, your spending
dollars of our own time cobbling together a workaround.  To each
his own!

Ted

> Just about the same as any Gmail, Yahoo or GMX
> address. There is NO way for a 5xx SMTP response. In fact, he was
> explicitly talking about an IMAP account, so the mail has been delivered
> already. Nothing can be done about that. Except better filtering and
> delivering spam to a dedicated folder to keep the Inbox clean.
>
> That is why I "advocated" a hack like that. Which I didn't even do,
> because it was implied by the OP. ;)
>
> I merely described better ways of implementing what is required, what
> Neil had in mind, and prevent client side filtering.
>
>

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 14:54 -0700, Neil Hodge wrote:
> > I may be wrong, but I understand Neil is talking about a single, ISP
> > provided email address. Just about the same as any Gmail, Yahoo or GMX
> > address. There is NO way for a 5xx SMTP response. In fact, he was
> > explicitly talking about an IMAP account, so the mail has been delivered
> > already. Nothing can be done about that. Except better filtering and
> > delivering spam to a dedicated folder to keep the Inbox clean.
> 
> That is a correct description of the situation.
> 
> Ted's attitude seems to be "do it right, or don't do it all", which,
> usually, I am completely for.  That being said, I don't actually build
> my own car, for the sake of it being "right".  In this case, given my
> various time constraints (trying to finish my research/dissertation),
> it is the case that the prefect is the enemy of the good.  So, either
> I will do it "wrong" (and maybe it won't work that way), or I won't do
> it at all . . .

There is nothing wrong with the (general) setup I outlined, in a
situation like yours, where the message has been delivered and accepted
by the MX already.

The "right" way Ted described, of multiple levels of defense and
rejecting spam at the SMTP level is certainly right, though only applies
in a situation where one does have control over the MX and the domain as
a whole. If, however, you are merely granted ownership of an email
address, using and depending on some third-party's SMTP accepting mail
on your behalf -- there's other shades of "right".

Perfectly acceptable shades of right.

FWIW, exactly such a situation [1] is what drove me to SA years ago in
the first place, and the beginning of a long and exciting journey. The
rest is history.

  guenther


[1] Minus the idea to run SA on multiple servers, and instead use IMAP
    right from the beginning, even though I was using a single MUA
    exclusively at that time... ;)

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Neil Hodge <ne...@gmail.com>]

Karsten, all:

>
> I may be wrong, but I understand Neil is talking about a single, ISP
> provided email address. Just about the same as any Gmail, Yahoo or GMX
> address. There is NO way for a 5xx SMTP response. In fact, he was
> explicitly talking about an IMAP account, so the mail has been delivered
> already. Nothing can be done about that. Except better filtering and
> delivering spam to a dedicated folder to keep the Inbox clean.
>

That is a correct description of the situation.

Ted's attitude seems to be "do it right, or don't do it all", which,
usually, I am completely for.  That being said, I don't actually build
my own car, for the sake of it being "right".  In this case, given my
various time constraints (trying to finish my research/dissertation),
it is the case that the prefect is the enemy of the good.  So, either
I will do it "wrong" (and maybe it won't work that way), or I won't do
it at all . . .

Neil

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 13:48 -0700, Ted Mittelstaedt wrote:
> On 7/22/2010 1:13 PM, Karsten Bräckelmann wrote:

> > Yes, I do maintain some home systems like that. With freemailers or ISP
> > accounts it often isn't possible any other way. Polling interval of a
> > minute or two.
> 
> And if everyone polled every minute or 2 then the server would melt 
> down.  You should be polling every 5 or better yet every 10.  But in
> any case why are you advocating a hack like this?  Your last post 
> advocated NOT a hack but in doing it right.  In every instance of a
> fetchmail setup the biggest loss is that you cannot issue an error
> 5xx to a spammer.

Ted, I believe you're thinking too large-scale.

I may be wrong, but I understand Neil is talking about a single, ISP
provided email address. Just about the same as any Gmail, Yahoo or GMX
address. There is NO way for a 5xx SMTP response. In fact, he was
explicitly talking about an IMAP account, so the mail has been delivered
already. Nothing can be done about that. Except better filtering and
delivering spam to a dedicated folder to keep the Inbox clean.

That is why I "advocated" a hack like that. Which I didn't even do,
because it was implied by the OP. ;)

I merely described better ways of implementing what is required, what
Neil had in mind, and prevent client side filtering.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Ted Mittelstaedt <te...@ipinc.net>]

On 7/22/2010 1:13 PM, Karsten Bräckelmann wrote:
> On Thu, 2010-07-22 at 12:55 -0700, Ted Mittelstaedt wrote:
>> On 7/22/2010 12:32 PM, Karsten Bräckelmann wrote:
>
>> The biggest downside IMHO is that you lose functionality on the RBL
>>
>> For example at 9am spammer spews.  At 9:15 spammer is listed in RBL.  At
>> 10:0m spammer mails you and your ISP picks up the mail because they
>> aren't using RBLs.  At 11:00 am the spammer's admin shuts the spammer
>> down and delists the IP from the RBL.  At 2:00pm you come along and
>> fetch your mail, and scan all the headers, checking all the IPs in
>> RBLs and the spam's IPs are not in the RBL then, even though they were
>> earlier.
>
> 4 hours between 10 am and 2 pm.
>
> Ted, you just described *client* side filtering, running when the user
> starts his MUA. Exactly what I advocated against, and instead
> recommended...
>
> Server side filtering. Even though SA runs (in the scenario I outlined)
> after some client, fetchmail, I still described server side filtering.
> Automatically, as soon as the mail reaches the server. No MUA required.
> Impossible to have a delay as you described, if set up properly.
>

Yes, I know - however lots of things can go wrong in such a setup and
the server can end up not pulling mail off the ISP's mailserver for
hours at a time.  Power loss, DSL or cable line goes down, whatever.
With a true "server" that is accepting inbound SMTP and not fetching it,
if there's a problem then the sender requeues and tries again.

>
> Yes, I do maintain some home systems like that. With freemailers or ISP
> accounts it often isn't possible any other way. Polling interval of a
> minute or two.
>

And if everyone polled every minute or 2 then the server would melt 
down.  You should be polling every 5 or better yet every 10.  But in
any case why are you advocating a hack like this?  Your last post 
advocated NOT a hack but in doing it right.  In every instance of a
fetchmail setup the biggest loss is that you cannot issue an error
5xx to a spammer.

In my SOHO mailserver setups the RBL's are always run by the MTA first 
before SA even sees the mail.  I also greylist.  And so the spammer
cannot even pass the mail at all if he's in the RBL.  SA RBL checks
still have value since they can scan the entire header whereas the
MTA only cares about the sending IP.  It's only the large shared 
customer mailservers that I don't do that and that's just because 
there's always some bozo who insists on getting mail from some 
corespondent on an RBLed server, even though it's blacklisted. But
I still greylist on everything.

> The truth to your outlined timing scenario?  The exact opposite!
>

until the cat eats the ethernet cable, etc.

Ted

> With polling that frequently, I *do* find spam ever other day, that
> *would* have hit various URI DNSBLs, if only the delay would have been 5
> minutes. Go figure.
>
>

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 12:55 -0700, Ted Mittelstaedt wrote:
> On 7/22/2010 12:32 PM, Karsten Bräckelmann wrote:

> The biggest downside IMHO is that you lose functionality on the RBL
> 
> For example at 9am spammer spews.  At 9:15 spammer is listed in RBL.  At 
> 10:0m spammer mails you and your ISP picks up the mail because they
> aren't using RBLs.  At 11:00 am the spammer's admin shuts the spammer
> down and delists the IP from the RBL.  At 2:00pm you come along and
> fetch your mail, and scan all the headers, checking all the IPs in
> RBLs and the spam's IPs are not in the RBL then, even though they were
> earlier.

4 hours between 10 am and 2 pm.

Ted, you just described *client* side filtering, running when the user
starts his MUA. Exactly what I advocated against, and instead
recommended...

Server side filtering. Even though SA runs (in the scenario I outlined)
after some client, fetchmail, I still described server side filtering.
Automatically, as soon as the mail reaches the server. No MUA required.
Impossible to have a delay as you described, if set up properly.


Yes, I do maintain some home systems like that. With freemailers or ISP
accounts it often isn't possible any other way. Polling interval of a
minute or two.

The truth to your outlined timing scenario?  The exact opposite!

With polling that frequently, I *do* find spam ever other day, that
*would* have hit various URI DNSBLs, if only the delay would have been 5
minutes. Go figure.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Ted Mittelstaedt <te...@ipinc.net>]

On 7/22/2010 12:32 PM, Karsten Bräckelmann wrote:
>
> One possible scenario I've been using a couple of times already:
>
> fetchmail to periodically poll mail every few minutes. Ideally a local
> SMTP server like postfix. procmail, which you already mentioned, feeding
> the mail to spamc and delivering spam to a dedicated folder. An IMAP
> server like dovecot.
>

if your going to go to that trouble then it's less work to just do it
right to begin with.

The truth is that ETRN is the way your supposed to do this kind of 
thing, fetchmail is a hack.  But even ETRN is not as good as your
own server.

>
> Yes, that might sound like lots of work, but actually should be rather
> easy and quick to setup. On the other hand, there *are* downsides to
> multiple SA installations, actually handling a single low-volume stream.
> Two immediately coming to mind, there sure are more.
>

[description deleted]

The biggest downside IMHO is that you lose functionality on the RBL

For example at 9am spammer spews.  At 9:15 spammer is listed in RBL.  At 
10:0m spammer mails you and your ISP picks up the mail because they
aren't using RBLs.  At 11:00 am the spammer's admin shuts the spammer
down and delists the IP from the RBL.  At 2:00pm you come along and
fetch your mail, and scan all the headers, checking all the IPs in
RBLs and the spam's IPs are not in the RBL then, even though they were
earlier.

Ted

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 11:51 -0700, Neil Hodge wrote:
> 2010/7/22 Karsten Bräckelmann <gu...@rudersport.de>:
> >
> > So, did you just say, that you check mail for one account (you mentioned
> > a single ISP only) from two different places, namely home and work? That
> > scenario spells IMAP to me -- your *own* IMAP server at home.
> >
> > That way, you will need a single SA only, and a single server to
> > maintain.
> 
> Well, I hadn't really expected to find anyone on the list who is
> willing to be so helpful, but since you are . . . :)
> 
> Basically, your understanding of my description is correct.  Of

OK. Then I guess I don't see how exactly you meant to implement this. :)

So your plan was, to access your ISPs IMAP server (and keep mail on that
server!?), while adding a second line of defense -- your own SA. That
right so far?

Unfortunately, your description of checking mail from $n places, and
having SA running at all these places, sounds like client side
filtering. Fire up the MUA, download all the junk, thump through it and
weed out all the left over spam -- time to get a coffee, if you didn't
do that in a while, say, over night.

Client side filtering; download spam, process spam locally. Server side
filtering; start your MUA and enjoy all the nasty buggers never even hit
your Inbox.

There is one piece to the puzzle, I don't understand how exactly you
want to use it -- procmail. That doesn't fit in the general picture of
client side filtering...


> course, my own first thought when I started thinking about this was
> precisely what you just suggested.  The problem with that is that I am
> with one of the big commercial ISPs (Comcast), and I don't have a
> static IP address, so I can't point to my router from any arbitrary
> machine on the internet.  I do believe that I can do some kind of
> dynamic DNS with my router.  Then I can probably pass incoming

Yes, dyndns.org for example. Your router is likely to support it, if it
supports any such service.


> connections to port (whatever for imap) through to my home machine.
> At some point, it just seems that the "right" solution is too much
> hassle, so I was just looking for a lower energy solution.

IMAP*S*, please. ;)  Or maybe even better, an openvpn tunnel. Access
your local data the way you want, while keeping a single, encrypted
connection. But yeah, that might be more work, and aways can be added
later.


> I could completely believe that the solution I came up with might have
> some fatal flaw, but I thought I would give it a shot first . . .

One possible scenario I've been using a couple of times already:

fetchmail to periodically poll mail every few minutes. Ideally a local
SMTP server like postfix. procmail, which you already mentioned, feeding
the mail to spamc and delivering spam to a dedicated folder. An IMAP
server like dovecot.


Yes, that might sound like lots of work, but actually should be rather
easy and quick to setup. On the other hand, there *are* downsides to
multiple SA installations, actually handling a single low-volume stream.
Two immediately coming to mind, there sure are more.

* You got two different Bayes databases. Thus, it will be harder to
reach the threshold of incoming ham and spam before it will start to
work. More training necessary. And they will produce different results.
Especially if one of them is likely to see some particular messages, the
other might never see. Think some stuff you generally get on weekends
only. Or during the day, weekdays.

* DNS. In addition to set-up and maintain two SA installations, you will
need local caching (non-forwarding) DNS servers. Why is that? You simply
cannot use your ISPs DNS, since it will not return hits for the most
valuable blacklists.


Personally, I would go the single-site route, bite the bullet and set up
my own IMAP server. No wait, I did. ;)

Added benefit: Virtually no network delay whatsoever from home. Plus,
you got access to the IMAP server's storage, so training bayes becomes a
breeze.

Even if you really prefer running SA client-side, IMHO there is a far
better solution than multiple SA installations. Namely the spamc -d
option, of using a common spamd (and thus fully configured and
maintained SA). Satellite systems then only need spamc installed, not SA
itself.

  guenther


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Neil Hodge <ne...@gmail.com>]

Karsten:

2010/7/22 Karsten Bräckelmann <gu...@rudersport.de>:
>
> So, did you just say, that you check mail for one account (you mentioned
> a single ISP only) from two different places, namely home and work? That
> scenario spells IMAP to me -- your *own* IMAP server at home.
>
> That way, you will need a single SA only, and a single server to
> maintain.
>

Well, I hadn't really expected to find anyone on the list who is
willing to be so helpful, but since you are . . . :)

Basically, your understanding of my description is correct.  Of
course, my own first thought when I started thinking about this was
precisely what you just suggested.  The problem with that is that I am
with one of the big commercial ISPs (Comcast), and I don't have a
static IP address, so I can't point to my router from any arbitrary
machine on the internet.  I do believe that I can do some kind of
dynamic DNS with my router.  Then I can probably pass incoming
connections to port (whatever for imap) through to my home machine.
At some point, it just seems that the "right" solution is too much
hassle, so I was just looking for a lower energy solution.

I could completely believe that the solution I came up with might have
some fatal flaw, but I thought I would give it a shot first . . .

Neil

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 11:05 -0700, Neil Hodge wrote:
> 2010/7/22 Karsten Bräckelmann <gu...@rudersport.de>:
> >
> > Anyway, I still do not see why you want to change these to begin with.
> > If you want site-wide databases, check out the wiki and its information
> > how to do that. If the reason is anything else -- you probably should
> > not do it. ;)
> 
> Ah, yes.  Well, basically, I get email from my provider via an imap
> server.  However, do not provide nearly enough spam filtering, so I
> thought I would do it myself.  I want SA to work on two different
> machines (not on the same LAN, but one at home, and one at work), as I
> check email on both of them, and I would like a similar response.  I

Still confusing. ;)

So, did you just say, that you check mail for one account (you mentioned
a single ISP only) from two different places, namely home and work? That
scenario spells IMAP to me -- your *own* IMAP server at home.

That way, you will need a single SA only, and a single server to
maintain.

Though maybe you want to explain your situation further. One thing I've
realized early during user support is, that answering the questions and
solving the issue the user wanted to do it, often is not the best way.
Understanding the issue might yield a totally different solution, the
new-to-the-project user would never have thought about.


> realize that the bayes data will diverge over time, but at least I
> would like user_prefs to be the same between the two machines.  I
> already have a setup to sync my personal data between these two
> machines, and I would like the SA directory to be in the subdirectory
> tree that gets synched.
> 
> 
> > Also, I would avoid *any* usage of 'spamassassin' command line options,
> > unless for debugging. See the note above regarding such options and
> > later use of spamd.
> 
> Well, maybe I will just break down and get client/server working now . . .

Probably a good idea. :)


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Neil Hodge <ne...@gmail.com>]

Karsten:

2010/7/22 Karsten Bräckelmann <gu...@rudersport.de>:
>
> Anyway, I still do not see why you want to change these to begin with.
> If you want site-wide databases, check out the wiki and its information
> how to do that. If the reason is anything else -- you probably should
> not do it. ;)
>

Ah, yes.  Well, basically, I get email from my provider via an imap
server.  However, do not provide nearly enough spam filtering, so I
thought I would do it myself.  I want SA to work on two different
machines (not on the same LAN, but one at home, and one at work), as I
check email on both of them, and I would like a similar response.  I
realize that the bayes data will diverge over time, but at least I
would like user_prefs to be the same between the two machines.  I
already have a setup to sync my personal data between these two
machines, and I would like the SA directory to be in the subdirectory
tree that gets synched.

>
> Also, I would avoid *any* usage of 'spamassassin' command line options,
> unless for debugging. See the note above regarding such options and
> later use of spamd.
>

Well, maybe I will just break down and get client/server working now . . .

Neil

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

Please keep list-posts on list.

On Thu, 2010-07-22 at 10:03 -0700, Neil Hodge wrote:
> 2010/7/22 Karsten Bräckelmann <gu...@rudersport.de>:
> >
> > man spamassassin-run
> >
> > That would be --prefspath. Only used for per-user preferences, usually
> > does *not* contain any rules. The --configpath points to the stock rules
> > dir by default, and should *not* be changed.
> >
> > Begs the question, why you want to change the default ~/.spamassassin
> > anyway?
> 
> So, when I read the man page, it says:
> 
> -p prefs, --prefspath=file, --prefs-file=file   Set user preferences file
> 
> When I saw this, I thought it was maybe for changing the preferences
> file _name_, but not necessarily its location.  Maybe the description
> of this option in the man page should be something like "Set user
> preferences file.  All user-local files (whitelist, bayes, etc) will
> be read from the same base directory.", or something similar.

See bayes_path, auto_whitelist_path and the bunch of related options in
the M::SA::Conf and AWL plugin documentation respectively.

Do note, that you'll need to use these with spamd anyway, and they do
work with 'spamassassin' as well.

Anyway, I still do not see why you want to change these to begin with.
If you want site-wide databases, check out the wiki and its information
how to do that. If the reason is anything else -- you probably should
not do it. ;)


> > Well, since installing SA 3.3.1, did you actually run sa-update?
> >
> > You do realize that this is now mandatory since 3.3 to grab the latest
> > stock rule-set, do you? (Alternatively, a separate rules tarball is
> > available.) SA doesn't ship with all rules bundled any longer.
> 
> Basically, I was getting this error because I was using the wrong
> command line option.
> 
> No, I have not run sa-update, nor did I realize it was required.
> While I probably would have figured it out after SA did nothing at
> all, thanks for the heads up . . . :)

If you installed SA from distro provided packages, odds are stock rules
have been installed. You will find the particular dirs and the order
they are checked by SA in 'man spamassassin'.


> > spamc / spamd rather than the plain and costly 'spamassassin' script, I
> > hope. Any chance the spamd daemon is started with the -c create user
> > prefs option?
> 
> Not yet.  I figured that until I could get the standalong exec
> working, no need to complicate the situation.  I will probably use the
> standalone for a week or two, until I am convinced it is working as
> desired, then I will switch to the client/server mode.

I would switch as soon as possible, since using the plain spamassassin
script comes with a quite severe overhead of starting a Perl process and
compiling all rules -- per message.

Also, I would avoid *any* usage of 'spamassassin' command line options,
unless for debugging. See the note above regarding such options and
later use of spamd.


> Thanks for the clarification.
> 
> Cheers,
> 
> Neil
-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Change .spamassassin directory

[Karsten Bräckelmann <gu...@rudersport.de>]

On Thu, 2010-07-22 at 08:57 -0700, Neil Hodge wrote:
> All:
> 
> Just wondering how to change the user-local prefs directory
> .spamassassin.  I have tried all combinations of commands and options,
> including
> 
> /usr/bin/spamassassin --configpath=/path/to/saprefs,

man spamassassin-run

That would be --prefspath. Only used for per-user preferences, usually
does *not* contain any rules. The --configpath points to the stock rules
dir by default, and should *not* be changed.

Begs the question, why you want to change the default ~/.spamassassin
anyway?


> but I keep getting the response
> 
> "config: no rules were found!  Do you need to run 'sa-update'? at
> /usr/bin/spamassassin line 403."

Well, since installing SA 3.3.1, did you actually run sa-update?

You do realize that this is now mandatory since 3.3 to grab the latest
stock rule-set, do you? (Alternatively, a separate rules tarball is
available.) SA doesn't ship with all rules bundled any longer.


> Neither does the symbolic link
> 
> .spamassassin -> /path/to/saprefs
> 
> work, i.e. it keeps getting changed to an actual directory.  I am
> running SA 3.3.1 from within procmail.  Any ideas???  Thanks.

spamc / spamd rather than the plain and costly 'spamassassin' script, I
hope. Any chance the spamd daemon is started with the -c create user
prefs option?


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}