getting 2 of some messages.

[Joe Acquisto-j4 <jo...@j4computers.com>]

Last few days, noticed getting two of some messages.   Been busy at my day job and brushed it off.   But now it appears to be happening with some (ir)regularity.

I can see from /var/log/mail that the repeat messages do have identical  message-id.  The only difference that caught my bleary eye was that for one
bayes=0.000000,autolearn=ham 
for the other
bayes=0.000000,autolearn=unavailable

Is the cause so obvious I should know already?   Hints? Ready fix?

spamassassin --version gives
SpamAssassin version 3.3.2
running on Perl version 5.16.0

Re: getting 2 of some messages.

[Bill Cole <sa...@billmail.scconsult.com>]

On 13 Aug 2015, at 19:59, Joe Acquisto-j4 wrote:

> Last few days, noticed getting two of some messages.   Been busy at my 
> day job and brushed it off.   But now it appears to be happening with 
> some (ir)regularity.
>
> I can see from /var/log/mail that the repeat messages do have 
> identical  message-id.  The only difference that caught my bleary eye 
> was that for one
> bayes=0.000000,autolearn=ham
> for the other
> bayes=0.000000,autolearn=unavailable
>
> Is the cause so obvious I should know already?   Hints? Ready fix?
>
> spamassassin --version gives
> SpamAssassin version 3.3.2
> running on Perl version 5.16.0

Sure, fine, but that's probably irrelevant. You don't mention how you're 
using SpamAssassin but it is almost never capable of causing message 
echoing, because it is almost always used in a consultative manner by 
something else that is responsible for storing and transporting mail: a 
MTA, a SMTP filtering proxy, etc. SA never has full custody of a 
message. You've mentioned no software that is responsible for transport 
or delivery of messages, so giving you anything but vague guesses is 
impossible

One common cause of receiving 2 copies of the same message is a network 
problem that occasionally drops outbound small packets. The last step in 
passing a message via SMTP is the receiving mail server sending an 
acknowledgement of having received the message OR an error reply telling 
the sender that it could not accept the message, either for a reason 
that should be deemed permanent or one that could be resolved in the 
near future. If that response is lost, the sender will eventually retry, 
while the receiver happily does its job of delivery. When the retry 
happens, the message is duplicated. This can also happen if your MTA is 
waiting for SpamAssassin to give its opinion before responding, and the 
sender times out in some way that your MTA can't detect. That can be 
caused by overaggressive firewalls, most notoriously Cisco gear with 
their spectacularly bad SMTP 'fixup' misfeature.

I recently had a spate of such duplicates when my ISP tried to roll out 
a change on their gateway devices that tried to enforce a 1:1 mapping of 
IPs to MAC addresses, even for business customers like me with multiple 
static IPs allocated. About a dozen times a day my MTA would get shunned 
by my next hop just long enough for the sender to be told it was 
unreachable while the MTA was waiting for something more after that last 
message that never came. It would eventually time out the SMTP session 
thinking it had received and acknowledged a piece of mail, and the 
sender woulds retry, thinking my MTA had died without receiving the 
mail.

The reason the two messages got two different autolearn results is 
possibly related (e.g. if there's a problem with your Bayes DB that is 
making SA take a long time to score messages or give up) but without 
knowing more details (Sendmail? Postfix? Amavis? MIMEDefang? BayesDB on 
a MySQL server halfway around the world? An MTA living behind a PIX that 
hasn't been touched in a decade?) anything along those lines is open to 
unrestricted imagination.