You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Jeremy Morton <ad...@game-point.net> on 2013/01/13 14:24:14 UTC
Proposed spamc patch: override threshold score
Hi,
I attach a proposed patch for the spamc client which I have tested - it
seems to be working well. It's based off the latest SVN I got from:
http://svn.apache.org/repos/asf/spamassassin/trunk
It adds the option -T to allow the spam threshold value to be
overridden, which changes when spamc will return 0/1 to indicate spam,
but does *not* change any of SpamAssassin's message rewriting.
The reason I think this change is a good idea is that I (and perhaps
others) find myself wanting to be able to define a "grey area" of
e-mails, where they are rewritten as spam but not completely discarded.
This can be done by setting the threshold override higher than the
normal SpamAssassin threshold.
For example, if you want the grey area to be between 3.0 and 8.0, you'd
set the normal SpamAssassin threshold to 3.0, then call spamc with the
threshold override of 8.0. Scores under 3.0 would be OK; spamc returns
0 and the message is not rewritten. Scores between 3.0 and 8.0 would be
"grey area" - spamc still returns 0 (so a shell script will not discard
it), but the message *is* rewritten. It will be delivered to the user's
mailbox as a rewritten message. Remaining scores would cause the
message to be rewritten *and* spamc would return 1, allowing a shell
script to discard it.
What do you think? Can it be merged into the SpamAssassin trunk?
--
Best regards,
Jeremy Morton (Jez)
Re: Proposed spamc patch: override threshold score
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 1/14/2013 12:31 PM, Jeremy Morton wrote:
> On 14/01/2013 13:32, Kevin A. McGrail wrote:
>> If the patch is thorough, well documented and fairly minor, I can't see
>> how it will slow down spamc very much. Is that the key concern really?
>
> Are you asking me? I wasn't the one saying it would slow down spamc.
> :-)
>
No, asking others more integral to the project, sorry.
I haven't look at the code but I'm not opposed to fairly minor tweaks
for spamc that are based on real-world usage.
Re: Proposed spamc patch: override threshold score
Posted by Jeremy Morton <ad...@game-point.net>.
On 14/01/2013 13:32, Kevin A. McGrail wrote:
> If the patch is thorough, well documented and fairly minor, I can't see
> how it will slow down spamc very much. Is that the key concern really?
Are you asking me? I wasn't the one saying it would slow down spamc. :-)
--
Best regards,
Jeremy Morton (Jez)
Re: Proposed spamc patch: override threshold score
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 1/14/2013 3:19 AM, Jeremy Morton wrote:
> On 14/01/2013 00:39, John Hardin wrote:
>> On Sun, 13 Jan 2013, Jeremy Morton wrote:
>>
>>> On 13/01/2013 18:06, John Hardin wrote:
>>>> Agreed. Generally the caller looks for the "X-Spam-Status: Yes,
>>>> score=16.6 ..." header in the processed message to make the
>>>> deliver/discard decision. I think it's dangerously confusing to
>>>> redefine
>>>> "threshold" in this manner.
>>>> ...
>>>> I haven't looked at how the various glues process the spamc output, if
>>>> they look for ==1 rather than >0 what I suggest might break lots of
>>>> stuff.
>>>
>>> Whereas my patch wouldn't break any existing code. Why would it be
>>> "dangerously confusing" to make this change when the default behaviour
>>> is identical, and we could give a relatively simple and easy
>>> explanation as to what it does?
>>
>> Because you're redefining what "threshold" means from what it is in the
>> configuration files. Perhaps "dangerous" was overstating it a bit, but
>> it will make it less obvious to figure out when someone asks "this
>> message says it was spam but it's still being delivered!"
>
> To get to this stage, someone would need to:
> - Get the latest version of spamc
> - Look in its documentation
> - Add -T x.x to their commandline arguments
>
> OK, I guess if someone is dumb enough to add a commandline argument
> they haven't bothered to understand, it may cause a problem. But how
> often is that going to happen? And we could just add "are you calling
> spamc with the -T argument?" to the standard list of questions such
> people need to be asked already. :-)
>
If the patch is thorough, well documented and fairly minor, I can't see
how it will slow down spamc very much. Is that the key concern really?
Re: Proposed spamc patch: override threshold score
Posted by Jeremy Morton <ad...@game-point.net>.
On 14/01/2013 00:39, John Hardin wrote:
> On Sun, 13 Jan 2013, Jeremy Morton wrote:
>
>> On 13/01/2013 18:06, John Hardin wrote:
>>> Agreed. Generally the caller looks for the "X-Spam-Status: Yes,
>>> score=16.6 ..." header in the processed message to make the
>>> deliver/discard decision. I think it's dangerously confusing to redefine
>>> "threshold" in this manner.
>>> ...
>>> I haven't looked at how the various glues process the spamc output, if
>>> they look for ==1 rather than >0 what I suggest might break lots of
>>> stuff.
>>
>> Whereas my patch wouldn't break any existing code. Why would it be
>> "dangerously confusing" to make this change when the default behaviour
>> is identical, and we could give a relatively simple and easy
>> explanation as to what it does?
>
> Because you're redefining what "threshold" means from what it is in the
> configuration files. Perhaps "dangerous" was overstating it a bit, but
> it will make it less obvious to figure out when someone asks "this
> message says it was spam but it's still being delivered!"
To get to this stage, someone would need to:
- Get the latest version of spamc
- Look in its documentation
- Add -T x.x to their commandline arguments
OK, I guess if someone is dumb enough to add a commandline argument they
haven't bothered to understand, it may cause a problem. But how often
is that going to happen? And we could just add "are you calling spamc
with the -T argument?" to the standard list of questions such people
need to be asked already. :-)
--
Best regards,
Jeremy Morton (Jez)
Re: Proposed spamc patch: override threshold score
Posted by John Hardin <jh...@impsec.org>.
On Sun, 13 Jan 2013, Jeremy Morton wrote:
> On 13/01/2013 18:06, John Hardin wrote:
>> Agreed. Generally the caller looks for the "X-Spam-Status: Yes,
>> score=16.6 ..." header in the processed message to make the
>> deliver/discard decision. I think it's dangerously confusing to redefine
>> "threshold" in this manner.
>> ...
>> I haven't looked at how the various glues process the spamc output, if
>> they look for ==1 rather than >0 what I suggest might break lots of
>> stuff.
>
> Whereas my patch wouldn't break any existing code. Why would it be
> "dangerously confusing" to make this change when the default behaviour is
> identical, and we could give a relatively simple and easy explanation as to
> what it does?
Because you're redefining what "threshold" means from what it is in the
configuration files. Perhaps "dangerous" was overstating it a bit, but it
will make it less obvious to figure out when someone asks "this message
says it was spam but it's still being delivered!"
> On 13/01/2013 17:02, Axb wrote:
>> Normally this is achieved by the glue as with Amavis, MailScanner,
>> using the API or Fuglu using spamd results
>>
>> IMO, spamc should remain simple and lightweight.
>
> But those pieces of software are much more heavyweight than a spam filtering
> script, and parsing the output headers is unnecessary processing.
...which is why I kinda agreed with you.
> This change keeps spamc very lightweight but adds a very useful tool for
> anyone invoking it from a shell script.
>
> By the way, I added a bug for this. It's at:
> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6890
Thanks.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The social contract exists so that everyone doesn't have to squat
in the dust holding a spear to protect his woman and his meat all
day every day. It does not exist so that the government can take
your spear, your meat, and your woman because it knows better what
to do with them. -- Dagny @ Ace of Spades
-----------------------------------------------------------------------
4 days until Benjamin Franklin's 307th Birthday
Re: Proposed spamc patch: override threshold score
Posted by Jeremy Morton <ad...@game-point.net>.
On 13/01/2013 18:06, John Hardin wrote:
> Agreed. Generally the caller looks for the "X-Spam-Status: Yes,
> score=16.6 ..." header in the processed message to make the
> deliver/discard decision. I think it's dangerously confusing to redefine
> "threshold" in this manner.
> ...
> I haven't looked at how the various glues process the spamc output,
> if
> they look for ==1 rather than >0 what I suggest might break lots of
> stuff.
Whereas my patch wouldn't break any existing code. Why would it be
"dangerously confusing" to make this change when the default behaviour
is identical, and we could give a relatively simple and easy explanation
as to what it does?
On 13/01/2013 17:02, Axb wrote:
> Normally this is achieved by the glue as with Amavis, MailScanner,
> using
> the API or Fuglu using spamd results
>
> IMO, spamc should remain simple and lightweight.
But those pieces of software are much more heavyweight than a spam
filtering script, and parsing the output headers is unnecessary
processing. This change keeps spamc very lightweight but adds a very
useful tool for anyone invoking it from a shell script.
By the way, I added a bug for this. It's at:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6890
--
Best regards,
Jeremy Morton (Jez)
Re: Proposed spamc patch: override threshold score
Posted by John Hardin <jh...@impsec.org>.
On Sun, 13 Jan 2013, Axb wrote:
> On 01/13/2013 05:15 PM, RW wrote:
>> On Sun, 13 Jan 2013 13:24:14 +0000
>> Jeremy Morton wrote:
>>
>> > For example, if you want the grey area to be between 3.0 and 8.0,
>> > you'd set the normal SpamAssassin threshold to 3.0, then call spamc
>> > with the threshold override of 8.0. Scores under 3.0 would be OK;
>> > spamc returns 0 and the message is not rewritten. Scores between 3.0
>> > and 8.0 would be "grey area" - spamc still returns 0 (so a shell
>> > script will not discard it), but the message *is* rewritten.
>>
>> I don't like it. If spamassassin is to have support for two-level
>> classification it should IMO be in the library and set through normal
>> configuration.
>
> Normally this is achieved by the glue as with Amavis, MailScanner, using the
> API or Fuglu using spamd results
>
> IMO, spamc should remain simple and lightweight.
Agreed. Generally the caller looks for the "X-Spam-Status: Yes, score=16.6
..." header in the processed message to make the deliver/discard decision.
I think it's dangerously confusing to redefine "threshold" in this manner.
That said, however, controlling the return code of spamc would probably
simplify the caller's job in a lot of cases.
I think it would be better to avoid redefining the existing return code
semantics and instead define a new return code for "discard", which would
only be returned if a discard threshold was specified on the spamc command
line and the spam hit that threshold.
I think such a change in spamc would be minimal, something (conceptually)
along the lines of:
if (discard_thresh > spam_thresh && msg_score >= discard_thresh) {
return discard_result;
}
else if (msg_score >= spam_thresh) {
return spam_result;
} else {
return 0;
}
The first IF block would be the only new code apart from parsing the new
command line option, which is handled by a common library.
I haven't looked at how the various glues process the spamc output, if
they look for ==1 rather than >0 what I suggest might break lots of stuff.
Either way, as we're about to release a new version, *now* is the time to
make a major semantic change like this! Get it in bugzilla right away so
the devs can consider it.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Gun Control laws aren't enacted to control guns, they are enacted
to control people: catholics (1500s), japanese peasants (1600s),
blacks (1860s), italian immigrants (1911), the irish (1920s),
jews (1930s), blacks (1960s), the poor (always)
-----------------------------------------------------------------------
4 days until Benjamin Franklin's 307th Birthday
Re: Proposed spamc patch: override threshold score
Posted by Axb <ax...@gmail.com>.
On 01/13/2013 05:15 PM, RW wrote:
> On Sun, 13 Jan 2013 13:24:14 +0000
> Jeremy Morton wrote:
>
>> Hi,
>>
>> I attach a proposed patch for the spamc client which I have tested -
>> it seems to be working well. It's based off the latest SVN I got
>> from: http://svn.apache.org/repos/asf/spamassassin/trunk
>>
>> It adds the option -T to allow the spam threshold value to be
>> overridden, which changes when spamc will return 0/1 to indicate
>> spam, but does *not* change any of SpamAssassin's message rewriting.
>>
>> The reason I think this change is a good idea is that I (and perhaps
>> others) find myself wanting to be able to define a "grey area" of
>> e-mails, where they are rewritten as spam but not completely
>> discarded. This can be done by setting the threshold override higher
>> than the normal SpamAssassin threshold.
>>
>> For example, if you want the grey area to be between 3.0 and 8.0,
>> you'd set the normal SpamAssassin threshold to 3.0, then call spamc
>> with the threshold override of 8.0. Scores under 3.0 would be OK;
>> spamc returns 0 and the message is not rewritten. Scores between 3.0
>> and 8.0 would be "grey area" - spamc still returns 0 (so a shell
>> script will not discard it), but the message *is* rewritten. It will
>> be delivered to the user's mailbox as a rewritten message. Remaining
>> scores would cause the message to be rewritten *and* spamc would
>> return 1, allowing a shell script to discard it.
>>
>> What do you think?
>
>
> I don't like it. If spamassassin is to have support for two-level
> classification it should IMO be in the library and set through normal
> configuration.
Normally this is achieved by the glue as with Amavis, MailScanner, using
the API or Fuglu using spamd results
IMO, spamc should remain simple and lightweight.
Axb
Re: Proposed spamc patch: override threshold score
Posted by RW <rw...@googlemail.com>.
On Sun, 13 Jan 2013 13:24:14 +0000
Jeremy Morton wrote:
> Hi,
>
> I attach a proposed patch for the spamc client which I have tested -
> it seems to be working well. It's based off the latest SVN I got
> from: http://svn.apache.org/repos/asf/spamassassin/trunk
>
> It adds the option -T to allow the spam threshold value to be
> overridden, which changes when spamc will return 0/1 to indicate
> spam, but does *not* change any of SpamAssassin's message rewriting.
>
> The reason I think this change is a good idea is that I (and perhaps
> others) find myself wanting to be able to define a "grey area" of
> e-mails, where they are rewritten as spam but not completely
> discarded. This can be done by setting the threshold override higher
> than the normal SpamAssassin threshold.
>
> For example, if you want the grey area to be between 3.0 and 8.0,
> you'd set the normal SpamAssassin threshold to 3.0, then call spamc
> with the threshold override of 8.0. Scores under 3.0 would be OK;
> spamc returns 0 and the message is not rewritten. Scores between 3.0
> and 8.0 would be "grey area" - spamc still returns 0 (so a shell
> script will not discard it), but the message *is* rewritten. It will
> be delivered to the user's mailbox as a rewritten message. Remaining
> scores would cause the message to be rewritten *and* spamc would
> return 1, allowing a shell script to discard it.
>
> What do you think?
I don't like it. If spamassassin is to have support for two-level
classification it should IMO be in the library and set through normal
configuration.