You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Bob Proulx <bo...@proulx.com> on 2007/12/13 02:07:00 UTC

Re: AWL: dont understand it

John D. Hardin wrote:
> peter pilsl wrote:
> > I need to turn off AWL by now.
> 
> Most people do... :)

The problem is that it is based upon the from address.  That is an
unreliable piece of data.  Spammers forge from addresses all of the
time.  Even valid senders will sometimes fabricate from addresses.  If
the input to the equation can't be guarenteed valid then the output of
the equation can't be guarenteed valid either.  GIGO.  The effect as
you see can be a denial of service against a valid from address.

Bob

Re: AWL: dont understand it

Posted by Mark Martinec <Ma...@ijs.si>.
On Thursday 13 December 2007 02:07:00 Bob Proulx wrote:
> The problem is that it is based upon the from address.  That is an
> unreliable piece of data.  Spammers forge from addresses all of the
> time.  Even valid senders will sometimes fabricate from addresses.  If
> the input to the equation can't be guarenteed valid then the output of
> the equation can't be guarenteed valid either.  GIGO.  The effect as
> you see can be a denial of service against a valid from address.

Right. And if someone is whitelisting some domains, things get quickly
much worse. Whithout whitelisting or other extreme scores AWL behaves
very well.

For about two months I'm running a modified version of SpamAssassin
with one additional field in an AWL SQL database, namely the DKIM
signer id. This effectively separates forged from nonforged authors
from domains such as gmail.com and yahoo.com. The AWL average is than
taken only across senders of the same signed domain as the current
message under investigation (or from unsigned messages in its own
separate group).

An interesting byproduct can be derived from the database: a long-term
score average for each DKIM signing id. It can be considered an
automatically derived signer reputation.

Here are some interesting spam score averages by signer id (just
loosely manually grouped for ease of reading and interpretation):

  ebay.fr                  -11.37
  ebay.ca                   -9.57
  ebay.co.uk                -9.10
  ebay.com                  -8.24
  ebay.at                   -6.80
  ebay.de                   -3.93
  reply3.ebay.com           -3.24
  reply.ebay.com            -1.90
  paypal.com                -6.95
  email.paypal.co.uk        -2.37

  gmail.com                 -4.28
  googlegroups.com          -3.94
  googlemail.com            -3.90
  google.com                -1.98

  yahoo-inc.com             -5.63
  yahoogroups.co.uk         -5.46
  yahoogroupes.fr           -5.05
  yahoogroups.com           -4.29
  yahoo.com.au              -4.72
  yahoo.se                  -2.19
  yahoo.com                 -2.08
  yahoo.co.uk               -0.31
  yahoo.es                   0.14
  yahoo.de                   0.20
  yahoo.com.cn               2.01
  yahoo.fr                   2.06
  yahoo.it                   2.34
  yahoo.ie                   4.23
  yahoo.gr                   4.28
  yahoo.ca                   4.77
  yahoo.co.nz                5.54
  yahoo.co.in                5.78
  yahoo.com.vn               5.87
  yahoo.com.hk               6.81
  yahoo.co.jp                7.04
  yahoo.com.sg               8.64
  yahoo.dk                  10.06
  yahoo.com.br              10.20
  yahoo.com.mx              11.13

  dostech.ca                -10.25
  kitterman.com             -9.80
  porcupine.org             -9.80
  megan.vbhcs.org           -9.40
  charite.de                -9.68
  state-of-mind.de          -9.36
  resistor.net              -9.29
  secnap.net                -8.54
  gmurray.org.uk            -8.10
  messiah.edu               -6.49

  cisco.com                 -5.63
  cern.ch                   -5.33
  skype.net                 -4.09
  welcome.skype.com         -2.07
  tugraz.at                 -4.91
  tu-graz.ac.at             -5.80
  uu.se                     -3.26
  aitech.ac.jp              -4.69
  hermes-softlab.com        -5.99
  amis.net                  -4.71
  ijs.si                    -4.62
  rogers.com                -4.20
  eurescom.eu               -3.98
  pacbell.net               -2.58
  newsletters.trendmicro.com  -2.42
  123greetings.com          -1.83
  amazon.com                -1.10
  youtube.com               -1.72
  alert.bankofamerica.com   -0.61
  m-w.com                    3.62
  astrology.com              3.67

  news.coleparmer.com       -1.54
  news.biomedcentral.com     0.22
  medcompare.com             0.40
  biocompare.com             1.01
  dentalcompare.com          1.62

  perfspot.com              -10.60
  hege.li                   -10.30
  prime.gushi.org           -9.86
  incertum.net              -9.80
  schetterer.org            -9.70
  nexaima.net               -9.23
  prodigy.net               -8.93
  unix-scripts.info         -8.92
  inetmsg.com               -8.79
  suedfactoring.de          -8.53
  izb.knu.ac.kr             -7.51
  arcamax.com               -6.20
  pd.infn.it                -5.75
  mtcc.com                  -5.72

  consulintel.es            -5.04
  geni.com                  -4.60
  ybb.ne.jp                 -4.37
  abv.bg                    -4.33
  journalexperts.com        -4.26
  vvm.com                   -4.03
  nagual.pp.ru              -4.00
  univ-tours.fr             -3.68
  btinternet.com            -3.61

  yousendit.com             -6.24
  springer.delivery.net     -3.55
  starwood.delivery.net     -1.19
  marriott.delivery.net      2.12
  alibris.i.delivery.net     2.29
  gap.delivery.net           2.70
  mail6.subscribermail.com    -2.97
  mail120.subscribermail.com  -5.17
  mail160.subscribermail.com  -0.55

  email.innocentive.com     -3.07
  meetup.com                -2.73
  themysterymethod.com      -2.18
  taggedmail.com            -1.89
  sbcglobal.net             -1.67
  delphij.net               -1.66
  rsys1.com                 -1.56
  dfxnews.com               -1.01
  mail-list.com             -0.40
  aweber.com                -0.31
  update.hallmark.com       -0.20
  emsnow-email.com          -0.20
  ipost.com                 -0.14
  care2.com                 -0.07
  email-beyond.com          0.02
  email.greenpeace.org      0.08
  c-f-1.com                 0.37
  skynet.be                 0.44
  getresponse.com           0.55
  hellgatelondon.com        0.63
  ophthalmologyweb.com      1.00
  neogen.ro                 1.00
  mail2fans.myreg.net       1.04
  qsnews.net                1.27
  tripadvisor.com           1.29
  info-aaas.org             1.31
  uk.update.ft.com          1.49
  avaaz.org                 1.49
  eletters.whatsnewnow.com  1.62
  email.powells.com         1.64
  yes.irislink.com          1.72
  sara-freder.com           1.77
  mailft.com                1.83
  e.drugstore.com           2.11
  reply.mb00.net            2.12
  e.bordersstores.com       2.19
  list2.bravenet.com        2.23
  webroot-email.com         2.35
  divx-newsletters.com      2.44
  reply.ms00.net            2.47
  email.bluemountain.com    2.49
  email.snapfish.com        2.53
  news.yousendit.com        2.54
  email.landsend.com        2.66
  mail.doctorstrust.com     2.70
  email.marquiswhoswho.com  2.79
  malv.co.uk                2.91
  dotmailer.co.uk           2.92
  zonealarm.zonelabs.com    2.92
  hotwire-travel.com        2.92
  pdamerica.org             3.13
  mercola.com               3.29
  photobucket.com           3.30
  mail.ivillage.com         3.32
  email.colonialwilliamsburg.com  3.41
  kazaapro.org              3.56
  nwnewsletters.com         3.58
  unparalleledproducts.com  3.60
  us.emaildirect.com        3.86
  news.virtualtourist.com   3.96
  bzene.com                 4.03
  1105info.com              4.13
  rediffmail.com            4.32
  threadless.com            4.72
  newlspromos2007.com       4.89
  newsletters.selfgrowth.com  5.31
  fazioquests.net           5.46
  sandisk-dm.com            6.38
  snapfish.p0.com           6.58
  netartsandculture.com     7.40
  earthlink.net             8.25
  quality-stocks.net        8.30
  atomidirect.com           8.36
  immensetradeblitz.com     8.82
  immensedispersalblitz.com    9.03
  immensecirculationblitz.com  9.08
  immensedispersalblasting.com 9.26
  immensedealingshoopla.com 10.41
  immensetradeblasting.com  11.08
  immensetradehoopla.com    11.47
  philreview.com            8.10
  residentialsreview.com    9.03
  specifiersreview.com      9.14
  nonprofitsreview.com      9.30
  beckreview.com            9.18
  intranetreviews.com       9.39
  vitareviews.com           9.63
  petsnewreviews.com        10.05
  oemreviews.com            11.55
  museumsreview.com         11.72
  vetnewreviews.com         12.36
  ironsreviews.com          12.77
  betterlivingdeals.com     9.16
  usafismail.net            9.30
  liberalture.com           9.30
  abundanttradeblasting.com    9.92
  flashydispersalpublicity.com 9.94
  tillinghastclassics.com   10.01
  priceypredictions.com     10.64
  enormoustrafficblitz.com  10.82
  middaybusiness.com        10.90
  myslotstips.com           10.94
  pintconduct.com           11.13
  flash.net                 11.46
  picoconduct.com           11.53
  tinyconduct.com           12.37
  corproute.com             12.37
  lspromos2007online.com    12.41
  toyspott.com              12.48
  atomiair.com              12.48
  atomexhibit.com           12.48
  pintquest.com             12.52
  pintreply.com             12.57
  liberalprint.com          12.60
  dwellroute.com            12.67
  choicemailslot.com        12.68
  barrysshipping.com        12.83
  derlitty.info             12.88
  corpvenue.com             12.90
  corpvard.com              12.94
  paintedsbulletins.com     12.97
  academicsbulletins.com    13.00
  psychsbulletins.com       13.10
  pintram.com               13.12
  seattlefresh.com          13.28
[etc]

Mark