You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Duncan, Brian M." <br...@kattenlaw.com> on 2013/06/10 20:45:07 UTC
Large # of Spam getting through all of a sudden.
Over the last 7 days I have seen a large # of Spam messages making it through our SpamAssassin 3.3.1 install. We use around 5 RBL's also.
It looks like it is all from the same sender.
They all seem to be coming from IP's all by the same netblock owner.
Here are some of them, but there are many many more.. It just started like 5 days ago.
66.96.253.241
64.120.241.228
66.197.142.29
66.197.142.23
66.197.207.152
66.197.177.174
64.191.61.25
OrgName: Network Operations Center Inc.
OrgId: NOC
Address: PO Box 591
City: Scranton
StateProv: PA
PostalCode: 18501-0591
Country: US
RegDate: 2001-04-04
Updated: 2011-09-24
Comment: Abuse Dept: abuse@hostnoc.net
Ref: http://whois.arin.net/rest/org/NOC
ReferralServer: rwhois://rwhois.hostnoc.net:4321
OrgTechHandle: SMA4-ARIN
OrgTechName: Arcus, S. Matthew
OrgTechPhone: +1-570-343-2200
OrgTechEmail: nic@hostnoc.net
OrgTechRef: http://whois.arin.net/rest/poc/SMA4-ARIN
Each IP winds up on an RBL like 24 hours later, but then they have already moved on.. It looks like they have a system setup that sends like 10-20 messages on each host, then cycles to a new message and new host and repeats only during business hours. The removal text working in the bodies of the messages seems to randomly change also.
I rarely have seen any SpamAssasin hits on the bodies of these messages.
(cached, score=-0.125, required 6.5, autolearn=not spam, RP_MATCHES_RCVD -0.12)
Anyone else see any of this activity?
I will include a couple examples below
Thanks
--------------------------------------------------------------------------------------------
Subject: Double body cleanse report average loss of 18lbs
-- This is going Viral in Hollywood. --
....................................................
I cant believe Kim Kardashian and Britney Spears used this system to cleanse their bodies before the summer.
We recently put the product to the test and results were amazing. This is what all the celebrities are doing.
There has been an astonishing loss of about 17 lb's in just one month doing this:
http://www.chestnutooh.com/summer/womens-day-blog/index.html
"I am defiantly on board with this...
Now Jennifer Anniston is to..."
This is really a great way to get ready for that Bikini.
This offer brought to you by IEE Target_Data_Networks.
1531 Eighty-Fourth Street, Brooklyn, New York, 11228
For immediate redaction from the database of the email sender and the offer originator:
http://www.chestnutooh.com/summer/o/G/default.html
Or you can use the above address to reach us with the same request by postal mail.
I am sorry if this is in a wrong category. I got a pork shoulder from a food pantry, and was wondering what to do with it. I heard I can do Pulled pork, but don't know how to transform this big mass into pulled pork. All I need to know is how to get the material. I can make a good BBQ flavoring, just need to know how to prepare it.
Thank you.
If I am in the wrong thread, please guide me and I will try not to disturb this subject again, thanks
A big tough muscle mass like a pork shoulder needs to be cooked slowly. I normally cook this in a smoker for about 2 hours a pound at 200-225. So a 9 pound butt takes around 18 hours. I look for an internal temp of 190 for pulled pork sammies.
It can also be done in a crock pot.
Low and slow is the key to this cut of meat.
40
----------------------------------------------------------------------------
Subject: Re: Changes to your score. Please review
===========================
----------------------------------------
Report Center
----------------------------------------
===========================
removed@removed.com
Notice Number: FEA50E30A55BA6E35AF70028549B89C6
June, 2013
----------------------------------------
This notice has reached you today because your score has been changed due to recent score updates.
You can find out any score adjustments in just a few minutes.
You can view your current score as of June, 2013 here: http://www.suchbah.com/changes/newchanges.html
Your score generating time: 35 seconds
Sincerely,
Score-Report-Team
End our future ad-emaiI here-
http://www.suchbah.com/endnechange/endop.html
4432 NORTH WALROND
KANSAS CITY MO 64117
Finally went to Gramercy Tavern, after a 1edb044c530fa05a016f2fbe7e94c208 good year of trying to find the right time. Friends from LA were in town and made reservations this past Monday so took care of my indecision for me. I was seriously undressed in sneakers and jeans, but luckily they sat us since we had reservations (though the manager or host 1edb044c530fa05a016f2fbe7e94c208 did give me the stare down).
The atmosphere and decor is geared more 1edb044c530fa05a016f2fbe7e94c208 towards the stuffy side, but wasn't as bad as I thought it'd be. Although the restaurant was filled with older suits there were a bunch of younger professionals inside the restaurant. As far as the food goes, you have three choices in terms of the type of prix fixe menu, the standard 3 course for $82 or the seasonal tasting menu ($116 or something) and a vegetarian one 1edb044c530fa05a016f2fbe7e94c208 for $88. We all went with the standard menu and I chose the grilled octopus appetizer, which was nothing note worthy, though the portions were fairly small. For the entree I got the halibut, which was again fairly good, but not great. My friends got the pork with foisgras, which I tried... and liked more than my halibut. I also sampled some of the 1edb044c530fa05a016f2fbe7e94c208 lobster, which was also very good.
I think where Gramercy Tavern excels and differentiates itself from other high end restaurants is the desserts and service. We all chose something 1edb044c530fa05a016f2fbe7e94c208 different. Our server suggested the warm chocolate bread pudding, which I found to be surprisingly good, but too filling. We also ordered the pineapple upside down cake, chocolate tart and the coconut tapioca. The latter was probably the consensus favorite in it's originality and taste. About the service, as it seems all of the reviewers have mentioned, it was far and above the 1edb044c530fa05a016f2fbe7e94c208 best service I've ever had. Thorough would be an understatement, which I hear is the calling card of a Danny Meyer restaurant. Servers were always omnipresent yet not visible and lurking (like it is at some restaurants). The food runners came in unison with each course of our prix fix meal. Definitely worthy of a Michelin star. I made my first trip to Grammercy 1edb044c530fa05a016f2fbe7e94c2!
08 Tavern (the Tavern Room) tonight. I had been trying for a week but a combination of a crazy work schedule, GT having too long of a wait, or GT not even putting anyone else on the list that night got in my way.
So tonight we knew what to do - go in an hour and a half before we 1edb044c530fa05a016f2fbe7e94c208 wanted to eat and get on the list. And so that's what we did. When we got back to the restaurant we didn't have to wait very long before our table was ready.
Needless to say I was pretty excited for the meal after hearing so 1edb044c530fa05a016f2fbe7e94c208 many good things and waiting so long to get in. But it just didn't live up to my expectations. Here is what we ordered:
Sweetbreads
Two oysters
Chicken 1edb044c530fa05a016f2fbe7e94c208
Lasagna
Ransom note cocktail
Apple pie dessert
The dessert was the highlight of the night with the cocktail and oysters next best. The chicken I had was good but the vegetable sides were 1edb044c530fa05a016f2fbe7e94c208 bitter. It took me a while to figure out what was off about the dish since the chicken itself was very high quality meat. I did taste some of my friend's lasagna and thought it was good but not great.
Overall I enjoyed the food but I wasn't that impressed. The portions 1edb044c530fa05a016f2fbe7e94c208 were also minuscule. Even my 5'0" female friend thought so and said she was hungry after we left.
I may be back someday to try the main dining room, but as of right now 1edb044c530fa05a016f2fbe7e94c208 I wasn't impressed enough with the tavern room to want to run back anytime soon.
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
Re: Large # of Spam getting through all of a sudden.
Posted by Ben Johnson <be...@indietorrent.org>.
On 6/10/2013 4:46 PM, David F. Skoll wrote:
> [Lost track of who wrote this]
>
>> 66.96.253.241
>> 64.120.241.228
>> 66.197.142.29
>> 66.197.142.23
>> 66.197.207.152
>> 66.197.177.174
>> 64.191.61.25
>
> Every single one of those IPs is on our "GreylistStumbler" list, meaning
> they've been greylisted, but have not been seen to pass greylisting.
>
> Implementing greylisting might stop most of the problem messages.
>
> Regards,
>
> David.
>
(Brian is the one who wrote it :))
That's an interesting observation, David.
As someone who recently implemented greylisting, its efficacy in this
particular type of situation cannot be overstated. Our spam volume
dropped from about 35% to less than 2% overnight, thanks to greylisting
at the MTA level.
While somewhat of a generic prescription, it's well-prescribed for a
reason: sort-out your Bayes situation (will probably require wiping and
starting over with a hand-sorted corpus that is *retained*) and
implement greylisting (provided you can live with its caveats).
The DNSBLs can be used to supplement the above.
Good luck, Brian!
--Ben
Re: Large # of Spam getting through all of a sudden.
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
[Lost track of who wrote this]
> 66.96.253.241
> 64.120.241.228
> 66.197.142.29
> 66.197.142.23
> 66.197.207.152
> 66.197.177.174
> 64.191.61.25
Every single one of those IPs is on our "GreylistStumbler" list, meaning
they've been greylisted, but have not been seen to pass greylisting.
Implementing greylisting might stop most of the problem messages.
Regards,
David.
Re: Large # of Spam getting through all of a sudden.
Posted by Benny Pedersen <me...@junc.eu>.
Alex skrev den 2013-06-10 22:40:
> How do you calculate the netblock, or do you just block the specific
> IP or the whole class C?
whois <ip>
shorewall iprange <netrange-start>-<netrange-end>
then shorewall show cidr results
i dont know how to make it without shorewall :)
# shorewall iprange 127.0.1.0-127.1.255.255
127.0.1.0/24
127.0.2.0/23
127.0.4.0/22
127.0.8.0/21
127.0.16.0/20
127.0.32.0/19
127.0.64.0/18
127.0.128.0/17
127.1.0.0/16
if one check spf on microsoft.com its seems thay dont use shorewall :)
--
senders that put my email into body content will deliver it to my own
trashcan, so if you like to get reply, dont do it
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi,
On Mon, Jun 10, 2013 at 8:09 PM, Alex <my...@gmail.com> wrote:
> Hi Kris,
>
> I'm trying to get your extract-data script running, and having some
> difficulties. It's dying at the $spamtest->check($mail) call. It just
> never returns. What does that function do?
>
> MSG: for (my $i=0; $i<$msgcount; $i++) {
> my $msg = $imap->message_string($msgs[$i]);
>
> print ".";
>
> my $mail = $spamtest->parse($msg);
> my $status = $spamtest->check($mail);
My apologies; I meant that to go to Kris off-list. The script I'm
referring to is here:
https://secure.deepnet.cx/trac/dnsbl/browser/trunk/dnsbl/extract-data
Thanks,
Alex
Re: Large # of Spam getting through all of a sudden.
Posted by Jonathan Nichols <jn...@pbp.net>.
On Jun 13, 2013, at 6:20 PM, Alex <my...@gmail.com> wrote:
>
>
> It's only been in the last few weeks that I've had real difficulty
> with pump-and-dump spam and needing to investigate something
> additional. Interestingly, they only seem to work during EDT business
> hours. After working with it for a few days, nearly every time I've
> researched an IP from my FPs, it's already been listed in zen.
>
Same here. All for the same company too, stock ticker named after stuff growing on our heads. They sail right through greylisting as well. (Ill paste a sample later, unfortunately I'm out of town with only a mobile device)
Re: Large # of Spam getting through all of a sudden.
Posted by Kris Deugau <kd...@vianet.ca>.
Alex wrote:
> I haven't gotten to implement the web cgi yet, and perhaps the
> question is answered there, but what criteria do you use to add the IP
> to the DNSBL once you've entered it into the web cgi?
Well, once entered through the web UI it will be exported to the DNS
data. There's no whitelist in the web UI or backend database structure
ATM to exclude entries on export. If you don't want it listed, don't
add it.
Feel free to play with the demo instance; it's got examples of just
about every possible listing situation.
What I do is to enter an IP on the first page, and see what comes up.
If there's already an extensive list of IPs, I tend to just submit and
add the new IP without further checking.
IPs that don't turn up any existing netblock entries get a WHOIS and/or
rDNS check, sometimes checking back to inspect the complete message in
more detail. Depending on what I turn up, it boils down to "can I
justify penalizing/blocking mail from this IP/netblock?"
In some cases I discover yet another customer forwarding mail from a
third-party hosting provider to their ISP account here, so I go back and
add a trust path entry, and dig in the message for the "real" source IP,
which I then check as usual.
> On Wed, Jun 12, 2013 at 12:05 PM, Kris Deugau <kd...@vianet.ca> wrote:
>> The key is to feed enough data in that you start getting the
>> IP-count-for-netblock-size thresholds passed, so that new sending IPs at
>> least get some penalty with the SA scoring.
>
> That would be snowshoe spam, then?
More or less; I haven't compared what I'm seeing locally with any other
patterns people have reported with snowshoe spam.
Arguably I should be reporting a lot of what I see go by to various
abuse contacts, but when you see the same netblock owners over and over
and over... you get the feeling that abuse@ is just an alias for
/dev/null. :/
-kgd
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi,
On Wed, Jun 12, 2013 at 12:05 PM, Kris Deugau <kd...@vianet.ca> wrote:
> Alex wrote:
>> It turned out to be a bit of local config,
>
> Care to share the specifics? I can't think of any SA configuration that
> might trigger this, TBH.
I had made some changes then ultimately overwrote it with the
original, so I'm not really sure. I think it was the extensive
processing time that was the cause.
> Well, all you get back from my script is a list of IP addresses and hit
> counts (or IPs formatted for dropping into tinydns). There's no further
> direct information about whether that IP might be a legitimate mail
> system that just passed on a spam, or a compromised Exchange server, or
> a previously unidentified white-hat ESP who got scammed long enough for
> some spammer to send a blast of mail.
It's only been in the last few weeks that I've had real difficulty
with pump-and-dump spam and needing to investigate something
additional. Interestingly, they only seem to work during EDT business
hours. After working with it for a few days, nearly every time I've
researched an IP from my FPs, it's already been listed in zen.
I'm just not sure I have the ability to scan the FPs quickly enough to
make a difference. I have to find something more automated or accept
the level of FPs I'm receiving.
It's also interesting to watch the spam as it comes in -- first it
typically hits bayes50, then after some period or number of spams, it
approaches bayes99, until finally it either stops or is over the bayes
or SA threshold and is quarantined. You can actually see the spam
being autolearned.
I have my threshold set to 13 and -1.0. I think it's somewhat
conservative and am considering lowering it, but not risk screwing up
my db.
> I don't bother checking IPs against other lists; one of the key uses I
> put the data to is in a "thin" SA instance that only checks DNSBLs and a
> handful of very accurate stock rules, all scored a little higher than
> stock. (Less than 30 rules, in total.) This skims off 80%+ of the junk
> (that would otherwise score 30+ against the full ruleset) at a fraction
> of the processing cost - and multiple DNSBL hits on the same bit of data
> are one of the key indicators here.
I've been just running it against my production SA install and sending
the output to a file, and coming back to analyze it later.
> Out of the box, the intent is to use it for scoring in SA; if you get a
> cluster of IPs reported in a leaf allocation, they will have one flag
> set for the IP itself, and may have up to 3 others set entirely
> automatically for the direct registrar allocation, one intermediate
> reallocation, and the final leaf assignment.
I haven't gotten to implement the web cgi yet, and perhaps the
question is answered there, but what criteria do you use to add the IP
to the DNSBL once you've entered it into the web cgi?
> The key is to feed enough data in that you start getting the
> IP-count-for-netblock-size thresholds passed, so that new sending IPs at
> least get some penalty with the SA scoring.
That would be snowshoe spam, then?
Thanks,
Alex
Re: Large # of Spam getting through all of a sudden.
Posted by Kris Deugau <kd...@vianet.ca>.
Alex wrote:
> It turned out to be a bit of local config,
Care to share the specifics? I can't think of any SA configuration that
might trigger this, TBH.
> but mostly not expecting it
> to take so long to check() a single message. I'm sorry for the
> trouble; perhaps I was impatient due to not understanding the SA perl
> API.
*nod* Actually, until checking more closely due to this thread, I
wasn't aware that this was doing a full SA ruleset run on the message.
I could arguably fiddle things to use a (mostly) empty configuration to
speed things up, but at a bare minimum you would still need to have the
trust path available as local configuration.
> Do you think there's anything that can be done to further automate the
> process of inspecting the IP addresses it produces? If I understand
> your docs correctly, it's an entirely manual process with inserting
> them into your web cgi and determining whether it should be added to
> your DNSBL, correct?
Well, all you get back from my script is a list of IP addresses and hit
counts (or IPs formatted for dropping into tinydns). There's no further
direct information about whether that IP might be a legitimate mail
system that just passed on a spam, or a compromised Exchange server, or
a previously unidentified white-hat ESP who got scammed long enough for
some spammer to send a blast of mail.
Internal whitelist notwithstanding, I regularly find IPs that probably
shouldn't be listed on a blacklist (even one that's just scored in SA),
and I prefer to keep the final decision in human hands. If I were
seeing 10x the volume I might add more automation to try to detect some
of these cases, and live with causing FPs on the leftovers.
For some types of nominally legitimate relay systems, I've found it
useful to be informed about the IP, so I can go back and look up the
message in order to report it as spam to the relay provider.
> I'm finding that by the time I can collect the FNs and run the script
> on them, a few spot-checks show they're already listed in zen.
I don't bother checking IPs against other lists; one of the key uses I
put the data to is in a "thin" SA instance that only checks DNSBLs and a
handful of very accurate stock rules, all scored a little higher than
stock. (Less than 30 rules, in total.) This skims off 80%+ of the junk
(that would otherwise score 30+ against the full ruleset) at a fraction
of the processing cost - and multiple DNSBL hits on the same bit of data
are one of the key indicators here.
> I've taken a handful of IPs that it's produced and added them to a
> postfix client restriction, which has worked quite well, until I can
> find the time to implement the local DNSBL.
>
> I'd like to be able to enhance the script to run it against a
> whitelist of IPs
Long on the to-do list, but not (yet) important enough (to me) to
implement, is to (ab)use the database to flag netblocks and/or IPs as
"white" instead of "black".
> or some other way to make this process less manually
> intensive.
TBH there's not much more I would automate out of the workflow I use. A
human decision-maker is required in the loop to double-check the
sorting; I've lost count of the number of times I've missed a
spammy-looking legitimate message that I recovered for proper handling
while checking the extracted IPs (and less frequently, URIs).
> For now, I'm just picking a few that have been reported
> most frequently and checking them manually in my mail logs, and adding
> them if necessary.
Out of the box, the intent is to use it for scoring in SA; if you get a
cluster of IPs reported in a leaf allocation, they will have one flag
set for the IP itself, and may have up to 3 others set entirely
automatically for the direct registrar allocation, one intermediate
reallocation, and the final leaf assignment.
The key is to feed enough data in that you start getting the
IP-count-for-netblock-size thresholds passed, so that new sending IPs at
least get some penalty with the SA scoring.
-kgd
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi,
Kris wrote:
> As best I can recall it runs some trailing bits of what you might
> reasonably call "message parsing", and at least the first stages of
> running rule checks. I couldn't find a middle ground that only did the
> real minimum necessary for extracting the relay IPs and URIs from the
> message.
It turned out to be a bit of local config, but mostly not expecting it
to take so long to check() a single message. I'm sorry for the
trouble; perhaps I was impatient due to not understanding the SA perl
API.
Do you think there's anything that can be done to further automate the
process of inspecting the IP addresses it produces? If I understand
your docs correctly, it's an entirely manual process with inserting
them into your web cgi and determining whether it should be added to
your DNSBL, correct?
I'm finding that by the time I can collect the FNs and run the script
on them, a few spot-checks show they're already listed in zen.
I've taken a handful of IPs that it's produced and added them to a
postfix client restriction, which has worked quite well, until I can
find the time to implement the local DNSBL.
I'd like to be able to enhance the script to run it against a
whitelist of IPs or some other way to make this process less manually
intensive. For now, I'm just picking a few that have been reported
most frequently and checking them manually in my mail logs, and adding
them if necessary.
Thanks,
Alex
Re: Large # of Spam getting through all of a sudden.
Posted by Kris Deugau <kd...@vianet.ca>.
Alex wrote:
>>> Hi Kris,
>>>
>>> I'm trying to get your extract-data script running, and having some
>>> difficulties. It's dying at the $spamtest->check($mail) call. It just
>>> never returns. What does that function do?
As best I can recall it runs some trailing bits of what you might
reasonably call "message parsing", and at least the first stages of
running rule checks. I couldn't find a middle ground that only did the
real minimum necessary for extracting the relay IPs and URIs from the
message.
> Yes, I'm sorry, I should have made it more clear that I understood
> that. It correctly instantiates the new SA instance:
>
> my $spamtest = Mail::SpamAssassin->new();
>
> then logs in, opens the mbox folder, reads it in,
Just to confirm; you've changed the IMAP user/password/server to a
suitable account? The type of storage under the IMAP server (mbox,
maildir, maildir++, database, stone tablets, etc) should be irrelevant;
it's not exposed to the IMAP client.
> then just sits there
> with the check($mail) function, which must be deep enough in the SA
> code that I'm not familiar with it and hoped someone had some ideas.
I'm afraid it took me some trial and error to work out the minimal
pieces of SA that I actually had to call to get this working - it was
still easier than rebuilding relay parsing and trust-path tracking, and
URI extraction from the message body myself. I've since forgotten most
of what I learned about the SA internals. :/
If you set a debug flag on the top-level SA object, you should get a
spew of standard SA debug output that will at least tell you where it
stalls.
A quick eyeball through the spamassassin command-line script shows you
should be able to change that line to:
my $spamtest = Mail::SpamAssassin->new({debug => 1});
to do this.
I wasn't sure before, but this script *does* run your live ruleset
against each message. I don't know if there's a way to separate the
message parse/deconstruction further from actually running the full
ruleset; for this script it would be better to *not* to have to run the
actual rules.
> This is for v3.3.2. I thought Kris was using this script regularly,
> and I have a production implementation of SA on this box, so I don't
> understand why it would be failing here.
Nor do I; I just diff'ed the live script against SVN and there are no
differences anywhere that might affect overall function. (Mostly in the
list of IP blocks used to trim the IP list output, plus a handful of
cosmetic output fiddles.)
-kgd
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi,
On Mon, Jun 10, 2013 at 8:40 PM, David B Funk
<db...@engineering.uiowa.edu> wrote:
> On Mon, 10 Jun 2013, Alex wrote:
>
>> Hi Kris,
>>
>> I'm trying to get your extract-data script running, and having some
>> difficulties. It's dying at the $spamtest->check($mail) call. It just
>> never returns. What does that function do?
>>
>> MSG: for (my $i=0; $i<$msgcount; $i++) {
>> my $msg = $imap->message_string($msgs[$i]);
>>
>> print ".";
>>
>> my $mail = $spamtest->parse($msg);
>> my $status = $spamtest->check($mail);
>>
>> My perl scripting abilities are somewhat limited, so I hoped you could
>> help me troubleshoot this. I can print out $mail and it contains the
>> full contents of the current message (there's only one in my imap mbox
>> test file).
>>
>> Ideas greatly appreciated.
>> Thanks,
>> Alex
>
> Alex, you do understand that script assumes you've got a full & properly
> configured installation of SA available on the machine you're running it on.
Yes, I'm sorry, I should have made it more clear that I understood
that. It correctly instantiates the new SA instance:
my $spamtest = Mail::SpamAssassin->new();
then logs in, opens the mbox folder, reads it in, then just sits there
with the check($mail) function, which must be deep enough in the SA
code that I'm not familiar with it and hoped someone had some ideas.
This is for v3.3.2. I thought Kris was using this script regularly,
and I have a production implementation of SA on this box, so I don't
understand why it would be failing here.
Thanks,
Alex
Re: Large # of Spam getting through all of a sudden.
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Mon, 10 Jun 2013, Alex wrote:
> Hi Kris,
>
> I'm trying to get your extract-data script running, and having some
> difficulties. It's dying at the $spamtest->check($mail) call. It just
> never returns. What does that function do?
>
> MSG: for (my $i=0; $i<$msgcount; $i++) {
> my $msg = $imap->message_string($msgs[$i]);
>
> print ".";
>
> my $mail = $spamtest->parse($msg);
> my $status = $spamtest->check($mail);
>
> My perl scripting abilities are somewhat limited, so I hoped you could
> help me troubleshoot this. I can print out $mail and it contains the
> full contents of the current message (there's only one in my imap mbox
> test file).
>
> Ideas greatly appreciated.
> Thanks,
> Alex
Alex, you do understand that script assumes you've got a full & properly
configured installation of SA available on the machine you're running it on.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi Kris,
I'm trying to get your extract-data script running, and having some
difficulties. It's dying at the $spamtest->check($mail) call. It just
never returns. What does that function do?
MSG: for (my $i=0; $i<$msgcount; $i++) {
my $msg = $imap->message_string($msgs[$i]);
print ".";
my $mail = $spamtest->parse($msg);
my $status = $spamtest->check($mail);
My perl scripting abilities are somewhat limited, so I hoped you could
help me troubleshoot this. I can print out $mail and it contains the
full contents of the current message (there's only one in my imap mbox
test file).
Ideas greatly appreciated.
Thanks,
Alex
Re: Large # of Spam getting through all of a sudden.
Posted by Kris Deugau <kd...@vianet.ca>.
Alex wrote:
> Do you have a method for collecting them, or is it done manually?
My process isn't specific to a given source. I get anywhere from 50 to
several hundred messages reported as spam by customers, daily. After
sorting, I feed the messages through
https://secure.deepnet.cx/trac/dnsbl/browser/trunk/dnsbl/extract-data
(modified slightly for local settings) to grab the IPs, URIs, and a
count for each. I go through the resulting IPs by hand to feed them
into the web form along with netblock owner data looked up by
one-at-a-time WHOIS lookups. Some lookups result in an abuse report
instead of adding an IP to the DNSBL.
> How do you calculate the netblock, or do you just block the specific
> IP or the whole class C?
"whois <ip>". Sometimes I have to do "whois </24 that the IP is in>" to
find the upstream allocation(s).
Fair warning - even at the relatively low volume of lookups I do, I've
seem indications that some providers' rWHOIS servers will ignore
requests after a ridiculously low threshold.
> Have you included those which are on the Spamhaus block list?
> http://www.spamhaus.org/drop/drop.txt
I haven't paid much attention to that; it's likely in use by our
backbone providers and our NOC may have added it to our core router
configurations as well. The IPs there are in the Spamhaus ZEN list
anyway (which we reject on), so they won't generally get through to
where I'll see them.
> Thanks for the work on your DNSBL. Definitely have to implement that.
> Perhaps I'll just start with blocking outright at the SMTP level.
I'd strongly advise against using my utility to generate DNSBL data to
block connections out of the box; you would run into far too much
collateral damage. However, when scored in SA it squeezes repeat
offenders harder and harder the more different IPs they use in their
leaf allocations. I *do* actually see noticeable shifts in the spam
reports occasionally as netblocks get flagged.
-kgd
Re: Large # of Spam getting through all of a sudden.
Posted by Alex <my...@gmail.com>.
Hi,
>> They all seem to be coming from IP's all by the same netblock owner.
>>
>> Here are some of them, but there are many many more.. It just started like 5 days ago.
>>
>> 66.96.253.241
>> 64.120.241.228
>> 66.197.142.29
>> 66.197.142.23
>> 66.197.207.152
>> 66.197.177.174
>> 64.191.61.25
>>
>> OrgName: Network Operations Center Inc.
>
> *nod* I recently flagged them as a nuisance netblock owner in the
> internal DNSBL[1] here. I've been seeing them for years.
Do you have a method for collecting them, or is it done manually?
How do you calculate the netblock, or do you just block the specific
IP or the whole class C?
Have you included those which are on the Spamhaus block list?
http://www.spamhaus.org/drop/drop.txt
Alas, the IPs mentioned above aren't on that list.
Thanks for the work on your DNSBL. Definitely have to implement that.
Perhaps I'll just start with blocking outright at the SMTP level.
Thanks,
Alex
RE: Large # of Spam getting through all of a sudden.
Posted by "Duncan, Brian M." <br...@kattenlaw.com>.
-----Original Message-----
From: Kris Deugau [mailto:kdeugau@vianet.ca]
Sent: Monday, June 10, 2013 2:21 PM
To: spamassassin-users
Subject: Re: Large # of Spam getting through all of a sudden.
>*nod* I recently flagged them as a nuisance netblock owner in the
>internal DNSBL[1] here. I've been seeing them for years.
>I have 54 netblocks of various sizes and distances from the regional IP
>registry on file for them, plus far more suballocations to their
>apparent customers.
>I would recommend scoring RP_MATCHES_RCVD to -0.001; it may be useful
>in combination with other factors, but as-is and with the default Bayes
>autolearn thresholds it can cause bad Bayes autolearn results. I'd also
>recommend dropping the Bayes autolearn-as-ham threshold below 0.
>-kgd
>[1] To maintain this local DNSBL, I feed IPs and whatever ARIN, RIPE, APNIC, AfriNIC or LACNIC allocation and reallocation data I can find into a somewhat rough-edged tool I wrote:
>https://secure.deepnet.cx/trac/dnsbl. It's set up to preemptively tag netblocks over time; if IPs keep getting reported in any given block, sooner or later it will cross a threshold >and IPs not actually reported will still have a bit set in the DNS result.
>In closing in on three years, I think I've removed netblocks for false-positives due to change in ownership of the block maybe twice.
Thanks for the suggestions and information on your experience with maintaining your own DNSBL, I have adjusted my autolearn-as-ham below 0 (-5 for now) I can see how in this scenario that I am in how that was not helping me.. I also set my RP_MATCHES_RCVD to -0.001, I was going to do that anyhow based on other reading I have done.
After this I am considering taking a look at building my own DNSBL, when I have more time later I will check out the tools you made, I took a quick look at the Perl scripts and they look like it makes it allot easier to do myself. Thanks for making that available to everyone.
Brian
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
Re: Large # of Spam getting through all of a sudden.
Posted by Kris Deugau <kd...@vianet.ca>.
(When creating a new thread, please create a new message instead of
replying to an existing message as your "new" thread will be buried
under that old thread for most people using a threading mail reader.)
Duncan, Brian M. wrote:
> Over the last 7 days I have seen a large # of Spam messages making it through our SpamAssassin 3.3.1 install. We use around 5 RBL's also.
>
> It looks like it is all from the same sender.
>
> They all seem to be coming from IP's all by the same netblock owner.
>
> Here are some of them, but there are many many more.. It just started like 5 days ago.
>
> 66.96.253.241
> 64.120.241.228
> 66.197.142.29
> 66.197.142.23
> 66.197.207.152
> 66.197.177.174
> 64.191.61.25
>
> OrgName: Network Operations Center Inc.
*nod* I recently flagged them as a nuisance netblock owner in the
internal DNSBL[1] here. I've been seeing them for years.
I have 54 netblocks of various sizes and distances from the regional IP
registry on file for them, plus far more suballocations to their
apparent customers.
> Each IP winds up on an RBL like 24 hours later, but then they have already moved on.. It looks like they have a system setup that sends like 10-20 messages on each host, then cycles to a new message and new host and repeats only during business hours. The removal text working in the bodies of the messages seems to randomly change also.
>
> I rarely have seen any SpamAssasin hits on the bodies of these messages.
>
> (cached, score=-0.125, required 6.5, autolearn=not spam, RP_MATCHES_RCVD -0.12)
I would recommend scoring RP_MATCHES_RCVD to -0.001; it may be useful
in combination with other factors, but as-is and with the default Bayes
autolearn thresholds it can cause bad Bayes autolearn results. I'd also
recommend dropping the Bayes autolearn-as-ham threshold below 0.
-kgd
[1] To maintain this local DNSBL, I feed IPs and whatever ARIN, RIPE,
APNIC, AfriNIC or LACNIC allocation and reallocation data I can find
into a somewhat rough-edged tool I wrote:
https://secure.deepnet.cx/trac/dnsbl. It's set up to preemptively tag
netblocks over time; if IPs keep getting reported in any given block,
sooner or later it will cross a threshold and IPs not actually reported
will still have a bit set in the DNS result.
In closing in on three years, I think I've removed netblocks for
false-positives due to change in ownership of the block maybe twice.
Re: Large # of Spam getting through all of a sudden.
Posted by RGB Camera <za...@gmail.com>.
On Mon, Jun 10, 2013 at 11:45 AM, Duncan, Brian M. <
brian.duncan@kattenlaw.com> wrote:
> Over the last 7 days I have seen a large # of Spam messages making it
> through our SpamAssassin 3.3.1 install. We use around 5 RBL's also.
>
> It looks like it is all from the same sender.
>
> They all seem to be coming from IP's all by the same netblock owner.
>
> Here are some of them, but there are many many more.. It just started
> like 5 days ago.
>
> 66.96.253.241
> 64.120.241.228
> 66.197.142.29
> 66.197.142.23
> 66.197.207.152
> 66.197.177.174
> 64.191.61.25
>
Yeah, 66.197.* got noticed here too. We're feeding messages from there to
Bayes in the hope that they get recognized when the spammers move to
another IP range, which will likely be in a few days...
RE: Large # of Spam getting through all of a sudden.
Posted by "Duncan, Brian M." <br...@kattenlaw.com>.
On 6/10/2013 2:45 PM, Duncan, Brian M. wrote:
> I rarely have seen any SpamAssasin hits on the bodies of these messages.
>
> (cached, score=-0.125, required 6.5, autolearn=not spam, RP_MATCHES_RCVD -0.12)
Do you train the Bayes database manually? Or via autolearn only?
I use SA via AMaViS, and the header changes look slightly different from
yours, but I see no evidence that Bayes scoring is being used in the
above header (if, in fact, that is a sample header with all SA markup
appended).
--Ben
Thanks for the reply,
We use Autolearn only.
I was thinking of starting some manual training after this bout of messages getting through, I just did not know how much of a benefit I would see given the behavior of the Spammer -And I have to setup IMAP2mbox so I can get these messages from Exchange over to my sendmail boxes first, hope to do that today..
All the hosts are winding up on Zen and Maps after 24 hours, but they only send like 20 messages in each set into my environment.. Then they switch to a new sending mail server, I figured they would have burned through their hosts by now. This sender seems to have amassed a large number of servers (not workstation botnets) before starting this last week. I can't recall the last time we had this happen.
Brian
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
Re: Large # of Spam getting through all of a sudden.
Posted by Ben Johnson <be...@indietorrent.org>.
On 6/10/2013 2:45 PM, Duncan, Brian M. wrote:
> I rarely have seen any SpamAssasin hits on the bodies of these messages.
>
> (cached, score=-0.125, required 6.5, autolearn=not spam, RP_MATCHES_RCVD -0.12)
Do you train the Bayes database manually? Or via autolearn only?
I use SA via AMaViS, and the header changes look slightly different from
yours, but I see no evidence that Bayes scoring is being used in the
above header (if, in fact, that is a sample header with all SA markup
appended).
--Ben