You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Juerg Reimann <jr...@jworld.ch> on 2015/12/15 23:25:58 UTC
feed spamassassin with a catch-all address
I have a domain which gets a lot of spam to non-existent addresses. I
thought why not set that domain to catch-all and feed all non-existent
addresses directly to spamassassin. Any thoughts why this could be a bad
idea? Of course any typos from real senders would also end up in sa,
however, I believe that's in this case negligible...
Thanks,
Juerg
Re: feed spamassassin with a catch-all address
Posted by Martin Gregorie <ma...@gregorie.org>.
On Tue, 2015-12-15 at 23:25 +0100, Juerg Reimann wrote:
> I have a domain which gets a lot of spam to non-existent addresses. I
> thought why not set that domain to catch-all and feed all non
> -existent
> addresses directly to spamassassin. Any thoughts why this could be a
> bad
> idea? Of course any typos from real senders would also end up in sa,
> however, I believe that's in this case negligible...
>
That's exactly what I do.
However, I also have:
- an ISP that runs grey-listing that I have enabled
- a private SA rule that adds points for unknown user names
- a quarantine system that holds spam for 7 days before
deleting it and that lists quarantined messages and stats
as part of the daily logwatch report.
... so the very small amount of misclassified spam isn't lost.
Martin
Re: bayes problem?
Posted by Reindl Harald <h....@thelounge.net>.
Am 17.12.2015 um 12:18 schrieb Matthias Apitz:
> El día Thursday, December 17, 2015 a las 11:47:50AM +0100, Reindl Harald escribió:
>
>>
>> Am 17.12.2015 um 10:54 schrieb Matthias Apitz:
>>> Since some days (I think(!) after I run 'sa-update') a lot of technical
>>> mails are declared as SPAM due to BAYES_99 (99-100% SPAM), for example
>>> this mail I'm responding now; I saved it as 'rh.mail' and run it
>>> through:
>>
>> "sa-update" adjust scores, brings rules, disables rules but has no
>> business in change the bayes behavior
>>
>> when ham hat a BAYES_99 you misclassified mails or did not keep a
>> ham/spam balance in your training
>
> Thanks. This was it. I trained HAM as --spam due to a typo in my script.
> I just re-learned this backlog now as --ham
thats why i keep my whole corpus ever trained and added a "rebuild"
param to may script which builds bayes from scratch in a tempfolder and
at the end replaces the current one
what we also do is run a second spamd-instance anything except bayes
disbaled and pass sample mails with a script, this alerts if a
ham-sample is not BAYES_00 and spam-samples no reaching BAYES_99 with a
simple textfile which ones should be ignored
Re: bayes problem?
Posted by Matthias Apitz <gu...@unixarea.de>.
El día Thursday, December 17, 2015 a las 11:47:50AM +0100, Reindl Harald escribió:
>
> Am 17.12.2015 um 10:54 schrieb Matthias Apitz:
> > Since some days (I think(!) after I run 'sa-update') a lot of technical
> > mails are declared as SPAM due to BAYES_99 (99-100% SPAM), for example
> > this mail I'm responding now; I saved it as 'rh.mail' and run it
> > through:
>
> "sa-update" adjust scores, brings rules, disables rules but has no
> business in change the bayes behavior
>
> when ham hat a BAYES_99 you misclassified mails or did not keep a
> ham/spam balance in your training
Thanks. This was it. I trained HAM as --spam due to a typo in my script.
I just re-learned this backlog now as --ham.
matthias
--
Matthias Apitz, ✉ guru@unixarea.de, 🌐 http://www.unixarea.de/ ☎ +49-176-38902045
Re: bayes problem?
Posted by Reindl Harald <h....@thelounge.net>.
Am 17.12.2015 um 10:54 schrieb Matthias Apitz:
> Since some days (I think(!) after I run 'sa-update') a lot of technical
> mails are declared as SPAM due to BAYES_99 (99-100% SPAM), for example
> this mail I'm responding now; I saved it as 'rh.mail' and run it
> through:
"sa-update" adjust scores, brings rules, disables rules but has no
business in change the bayes behavior
when ham hat a BAYES_99 you misclassified mails or did not keep a
ham/spam balance in your training
> $ spamassassin -tD < rh.mail > rh.out 2> rh.debug
>
> The results are here
>
> http://www.unixarea.de/SA/rh.mail
> http://www.unixarea.de/SA/rh.out
> http://www.unixarea.de/SA/rh.debug
>
> Can some kind soul help me please having a look what is now wrong with my
> bayes ? Thanks in advance
bayes problem? (was: Re: feed spamassassin with a catch-all address)
Posted by Matthias Apitz <gu...@unixarea.de>.
Hello,
Since some days (I think(!) after I run 'sa-update') a lot of technical
mails are declared as SPAM due to BAYES_99 (99-100% SPAM), for example
this mail I'm responding now; I saved it as 'rh.mail' and run it
through:
$ spamassassin -tD < rh.mail > rh.out 2> rh.debug
The results are here
http://www.unixarea.de/SA/rh.mail
http://www.unixarea.de/SA/rh.out
http://www.unixarea.de/SA/rh.debug
Can some kind soul help me please having a look what is now wrong with my
bayes ? Thanks in advance
matthias
--
Matthias Apitz, ✉ guru@unixarea.de, 🌐 http://www.unixarea.de/ ☎ +49-176-38902045
Re: feed spamassassin with a catch-all address
Posted by Reindl Harald <h....@thelounge.net>.
Am 17.12.2015 um 03:54 schrieb Alex:
>>> This type of honeypot can find numerous bad actors and identify
>>> dictionary attackers. It has excellent merit and many people use this
>>> type of data. You might find it useful for blocking IPs, finding bad
>>> URLs, identifying spam for bayes, etc.
>>
>> easy to kill legit/ESP bulk and use the rest as bayes fodder...masschecks,
>> etc, etc
>
> Can anyone share the technique/mechanism they use for specifically
> collecting the IPs, URLs, etc to put them in a format useful to
> postfix/spamassassin?
>
> I have a few honeypots, but haven't had the time/resources/ability to
> actually parse the gigabytes of spam I've received and turn it into
> something useful
my honeypot is written in PHP and writes IP's with a expire-time oif 7
days to a mysql database which then froma cronjob is turned to a zone
for "rbldnsd"
well, and on port 25 it speaks basic SMTP and accepts all mail below 256
KB size and writes tehm to .eml files, around 500-1000 mails per day are
easy to review after drag&drop to thunderbird, just sort by subject, 10
seconds later there are no more than 100 left to review
Re: feed spamassassin with a catch-all address
Posted by Kris Deugau <kd...@vianet.ca>.
Alex wrote:
> Can anyone share the technique/mechanism they use for specifically
> collecting the IPs, URLs, etc to put them in a format useful to
> postfix/spamassassin?
I wrote a script to do part of the job; it's part of a local DNSBL
utility - https://secure.deepnet.cx/trac/dnsbl. (Specifically, the
extract-data script -
https://secure.deepnet.cx/svn/dnsbl/trunk/dnsbl/extract-data.) It
should be fairly trivial to modify it to walk a local mail folder
instead of an IMAP folder. Fair warning - the whole suite is a little
rough around the edges, but I've been using it in production pretty much
as-is for a number of years now.
You'll probably want to feed the results to some other script for
processing into Postfix access or body rules, SpamAssassin header or
body rules, or modify to taste to just feed the results directly into
either the DNSBL bits, or your own DNSBL solution.
-kgd
Re: feed spamassassin with a catch-all address
Posted by Alex <my...@gmail.com>.
Hi,
>> This type of honeypot can find numerous bad actors and identify
>> dictionary attackers. It has excellent merit and many people use this
>> type of data. You might find it useful for blocking IPs, finding bad
>> URLs, identifying spam for bayes, etc.
>
> easy to kill legit/ESP bulk and use the rest as bayes fodder...masschecks,
> etc, etc
Can anyone share the technique/mechanism they use for specifically
collecting the IPs, URLs, etc to put them in a format useful to
postfix/spamassassin?
I have a few honeypots, but haven't had the time/resources/ability to
actually parse the gigabytes of spam I've received and turn it into
something useful.
Much of it is from domains that were registered long ago and never
used where I'm now priming it with a few addresses scattered around
the interwebs...
Thanks for any ideas.
Alex
Re: feed spamassassin with a catch-all address
Posted by Axb <ax...@gmail.com>.
On 12/15/2015 11:33 PM, Kevin A. McGrail wrote:
> On 12/15/2015 5:25 PM, Juerg Reimann wrote:
>> I have a domain which gets a lot of spam to non-existent addresses. I
>> thought why not set that domain to catch-all and feed all non-existent
>> addresses directly to spamassassin. Any thoughts why this could be a bad
>> idea? Of course any typos from real senders would also end up in sa,
>> however, I believe that's in this case negligible...
>>
>> Thanks,
>> Juerg
> This type of honeypot can find numerous bad actors and identify
> dictionary attackers. It has excellent merit and many people use this
> type of data. You might find it useful for blocking IPs, finding bad
> URLs, identifying spam for bayes, etc.
easy to kill legit/ESP bulk and use the rest as bayes
fodder...masschecks, etc, etc
Re: feed spamassassin with a catch-all address
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 12/15/2015 5:25 PM, Juerg Reimann wrote:
> I have a domain which gets a lot of spam to non-existent addresses. I
> thought why not set that domain to catch-all and feed all non-existent
> addresses directly to spamassassin. Any thoughts why this could be a bad
> idea? Of course any typos from real senders would also end up in sa,
> however, I believe that's in this case negligible...
>
> Thanks,
> Juerg
This type of honeypot can find numerous bad actors and identify
dictionary attackers. It has excellent merit and many people use this
type of data. You might find it useful for blocking IPs, finding bad
URLs, identifying spam for bayes, etc.
Regards,
AKM
Re: feed spamassassin with a catch-all address
Posted by Reindl Harald <h....@thelounge.net>.
Am 16.12.2015 um 16:22 schrieb Shawn Bakhtiar:
>> On Dec 16, 2015, at 1:11 AM, Reindl Harald <h....@thelounge.net> wrote:
>>
>> Am 15.12.2015 um 23:25 schrieb Juerg Reimann:
>>> I have a domain which gets a lot of spam to non-existent addresses. I
>>> thought why not set that domain to catch-all and feed all non-existent
>>> addresses directly to spamassassin. Any thoughts why this could be a bad
>>> idea? Of course any typos from real senders would also end up in sa,
>>> however, I believe that's in this case negligible...
>>
>> bayes poisioning when it's done without review
>>
>> on our honeypot only 20 out of 300 mails each day are usable for training, the rest is in the meantime 64 KB large random junk
>>
>
> Instead of directly updating SA we use a program that updates our own DNS RBL using Bind DBZ and use counts as a metric incase any false positives (IE an occasional misspelled email) come through. We save the processed email in a folder for later training.
i prefer "rbldnsd" for that job, a small cronjob running every minute
and re-create the zonefile does the job well, for blacklists it's much
faster then bind
with the correct it detects automatically when the zonefile was changed
and reloads it and last but not least you can rsync /var/lib/rbldnsd/ to
other nodes and create redundancy without depend on a database
__________________________
oh well, for lists which allow rsync or some other way to fetch their
data you can have them local and save internet requests, on the MX
unbound runs as recursive dns-cache and forwards to rbldnsnd running on
a different port
forward-zone:
name: "dnsbl.thelounge.net"
forward-addr: 127.0.0.1@1053
rbldns 447 0.0 0.6 42212 27376 ? Ss Dez12 0:56
/usr/sbin/rbldnsd -f -n -r/var/lib/rbldnsd -c 60s -t 300:300:300 -e -v
-a -q -4 -b 127.0.0.1/1053 dnsbl.thelounge.net:ip4set:dnsbl.thelounge.net
Re: feed spamassassin with a catch-all address
Posted by Shawn Bakhtiar <sh...@hotmail.com>.
> On Dec 16, 2015, at 1:11 AM, Reindl Harald <h....@thelounge.net> wrote:
>
>
>
> Am 15.12.2015 um 23:25 schrieb Juerg Reimann:
>> I have a domain which gets a lot of spam to non-existent addresses. I
>> thought why not set that domain to catch-all and feed all non-existent
>> addresses directly to spamassassin. Any thoughts why this could be a bad
>> idea? Of course any typos from real senders would also end up in sa,
>> however, I believe that's in this case negligible...
>
> bayes poisioning when it's done without review
>
> on our honeypot only 20 out of 300 mails each day are usable for training, the rest is in the meantime 64 KB large random junk
>
Instead of directly updating SA we use a program that updates our own DNS RBL using Bind DBZ and use counts as a metric incase any false positives (IE an occasional misspelled email) come through. We save the processed email in a folder for later training.
Re: feed spamassassin with a catch-all address
Posted by Reindl Harald <h....@thelounge.net>.
Am 15.12.2015 um 23:25 schrieb Juerg Reimann:
> I have a domain which gets a lot of spam to non-existent addresses. I
> thought why not set that domain to catch-all and feed all non-existent
> addresses directly to spamassassin. Any thoughts why this could be a bad
> idea? Of course any typos from real senders would also end up in sa,
> however, I believe that's in this case negligible...
bayes poisioning when it's done without review
on our honeypot only 20 out of 300 mails each day are usable for
training, the rest is in the meantime 64 KB large random junk