You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Robert Burrell Donkin <ro...@gmail.com> on 2009/04/16 19:53:53 UTC
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> I have a problem with a manufacturer of a certain anti-virus e-mail
> gateway. This particular mail receiver closes the TCP connection after
> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
> without waiting for a QUIT command to be sent by the client (James in
> this case).
>
> This is a clear violation of RFC 5321 which says that "the receiver
> MUST NOT intentionally close the transmission channel until it
> receives and replies to a QUIT command".
>
> Unfortunately the manufacturer is aware of this fact but still refuses
> to change the behavior of the software. They claim that this behavior
> is common for bigger providers in order to avoid DDOS attacks and save
> port numbers (which is BS in my opinion).
>
> Anyway, RemoteDelivery uses the Java Mail API to send the message,
> something like this:
>
> try { ...
> transport.sendMessage(message, addr);
> } finally {
> transport.close();
> }
>
> So sendMessage() succeeds but close() throws a MessagingException and
> James (correctly) thinks it has to send the message again. As a
> consequence the recipient receives the same message several times
> until James gives up permanently.
>
> Now, I do not ask for a modification in James since the problem
> clearly lies on the recipient's side...
i'm not sure that an unchecked close is correct in this case. i would
prefer to see an IO action in a finally wrapped in a exception
handling block eg
try { ...
transport.sendMessage(message, addr);
} finally {
try {
transport.close();
} catch (Exception e) {
getLog().warn("Failed to cleanly close connection to remote server", e);
}
}
if the action needs to be repeated then a boolean should be used. this
should ensure that both exceptions are logging and the appropriate one
processed.
FWIW i would support altering the code without the additional boolean
> But has anyone ever experienced something like this? And maybe is
> there a workaround?
(hopefully stefano or norman will jump in here)
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>> Robert Burrell Donkin ha scritto:
>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>> <ma...@gmail.com> wrote:
>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>> gateway. This particular mail receiver closes the TCP connection after
>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>> without waiting for a QUIT command to be sent by the client (James in
>>>> this case).
>>>>
>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>> MUST NOT intentionally close the transmission channel until it
>>>> receives and replies to a QUIT command".
>>>>
>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>> to change the behavior of the software. They claim that this behavior
>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>> port numbers (which is BS in my opinion).
>>>>
>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>> something like this:
>>>>
>>>> try { ...
>>>> transport.sendMessage(message, addr);
>>>> } finally {
>>>> transport.close();
>>>> }
>>>>
>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>> James (correctly) thinks it has to send the message again. As a
>>>> consequence the recipient receives the same message several times
>>>> until James gives up permanently.
>>>>
>>>> Now, I do not ask for a modification in James since the problem
>>>> clearly lies on the recipient's side...
>>>
>>> i'm not sure that an unchecked close is correct in this case. i would
>>> prefer to see an IO action in a finally wrapped in a exception
>>> handling block eg
>>>
>>> try { ...
>>> transport.sendMessage(message, addr);
>>> } finally {
>>> try {
>>> transport.close();
>>> } catch (Exception e) {
>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>> }
>>> }
>>>
>>> if the action needs to be repeated then a boolean should be used. this
>>> should ensure that both exceptions are logging and the appropriate one
>>> processed.
>>>
>>> FWIW i would support altering the code without the additional boolean
>>
>> +1
>
> That would solve the problem but I think the behavior of James would
> then become non-standard compliant.
>
> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
> experience a connection close, reset [...] SHOULD [...] treat the mail
> transaction as if a 451 response had been received and act
> accordingly."
<blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
SMTP clients that experience a connection close, reset, or other
communications failure due to circumstances not under their control
(in violation of the intent of this specification but sometimes
unavoidable) SHOULD, to maintain the robustness of the mail system,
treat the mail transaction as if a 451 response had been received and
act accordingly.
</blockquote>
AIUI transport.close() sends a QUIT and then forcably closes the
socket. the client should assume that an exception means that 451
'Requested action aborted: error in processing' has been returned by
the QUIT. in other words, that the QUIT failed and cannot be retried.
does this mean that the previous send should be assumed to be rolled back?
> Maybe a switch would be in order? But if nobody else ever had this
> problem I can also write a workaround locally without affecting the
> code base.
possibly
>>>> But has anyone ever experienced something like this? And maybe is
>>>> there a workaround?
>>>
>>> (hopefully stefano or norman will jump in here)
>>
>> I'm using a custom RemoteDelivery but (if I remember correctly) that
>> case should be handled the same. I never had a "duplicate delivery"
>> issue reported.
>>
>> I deliver a lot of mail nowadays, but mainly to italian recipients:
>> maybe the antivirus gateway Markus talks about is not so used in italy.
>> Markus, can you name the product?
>
> I would rather not; or at least not until I have spoken to my
> co-workers. The company is located in Austria but I have no idea how
> widely the product has been adopted.
you may want to take this off list to stefano directly or perhaps hook up on IRC
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Fri, Apr 17, 2009 at 12:38 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> On Fri, Apr 17, 2009 at 9:12 AM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>
> <snip>
>
>>> In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
>>> intentionally close the transmission channel until it receives and
>>> replies to a QUIT command".
>>
>> yes
>>
>> however, this behaviour is a real PITA for servers and enables DOS
>> attacks.
>
> I'm not an expert but wouldn't you run a DOS attack between DATA and
> the dot? Send a few bytes, wait a few seconds, send a few bytes...
easier coding just to open a connection and then not quit, but yes, i
see your point
> Besides, a reasonable timeout wouldn't be a problem. Timeouts are also
> specified in section 4.5.3.2. in the RFC.
yep (but IMHO 4.1.1.10 should have been clarified to include the
possibility of a timeout)
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Markus Wiederkehr <ma...@gmail.com>.
On Fri, Apr 17, 2009 at 9:12 AM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
<snip>
>> In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
>> intentionally close the transmission channel until it receives and
>> replies to a QUIT command".
>
> yes
>
> however, this behaviour is a real PITA for servers and enables DOS
> attacks.
I'm not an expert but wouldn't you run a DOS attack between DATA and
the dot? Send a few bytes, wait a few seconds, send a few bytes...
Besides, a reasonable timeout wouldn't be a problem. Timeouts are also
specified in section 4.5.3.2. in the RFC.
> i would expect most commercial SMTP servers to allow this
> part of the specification to be ignored. i expect most servers just
> set a timeout on the socket (AIUI james does this by default) so the
> connection will timeout after a suitable period of inactivity. the
> problem is that this server is particularly (and unusually) aggressive
> in terminating the connection.
>
> - robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Thu, Apr 16, 2009 at 10:41 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> On Thu, Apr 16, 2009 at 11:05 PM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
>> <ma...@gmail.com> wrote:
>>> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>>>> Robert Burrell Donkin ha scritto:
>>>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>>>> <ma...@gmail.com> wrote:
>>>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>>>> gateway. This particular mail receiver closes the TCP connection after
>>>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>>>> without waiting for a QUIT command to be sent by the client (James in
>>>>>> this case).
>>>>>>
>>>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>>>> MUST NOT intentionally close the transmission channel until it
>>>>>> receives and replies to a QUIT command".
>>>>>>
>>>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>>>> to change the behavior of the software. They claim that this behavior
>>>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>>>> port numbers (which is BS in my opinion).
>>>>>>
>>>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>>>> something like this:
>>>>>>
>>>>>> try { ...
>>>>>> transport.sendMessage(message, addr);
>>>>>> } finally {
>>>>>> transport.close();
>>>>>> }
>>>>>>
>>>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>>>> James (correctly) thinks it has to send the message again. As a
>>>>>> consequence the recipient receives the same message several times
>>>>>> until James gives up permanently.
>>>>>>
>>>>>> Now, I do not ask for a modification in James since the problem
>>>>>> clearly lies on the recipient's side...
>>>>>
>>>>> i'm not sure that an unchecked close is correct in this case. i would
>>>>> prefer to see an IO action in a finally wrapped in a exception
>>>>> handling block eg
>>>>>
>>>>> try { ...
>>>>> transport.sendMessage(message, addr);
>>>>> } finally {
>>>>> try {
>>>>> transport.close();
>>>>> } catch (Exception e) {
>>>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>>>> }
>>>>> }
>>>>>
>>>>> if the action needs to be repeated then a boolean should be used. this
>>>>> should ensure that both exceptions are logging and the appropriate one
>>>>> processed.
>>>>>
>>>>> FWIW i would support altering the code without the additional boolean
>>>>
>>>> +1
>>>
>>> That would solve the problem but I think the behavior of James would
>>> then become non-standard compliant.
>>>
>>> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
>>> experience a connection close, reset [...] SHOULD [...] treat the mail
>>> transaction as if a 451 response had been received and act
>>> accordingly."
>>
>> <blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
>> SMTP clients that experience a connection close, reset, or other
>> communications failure due to circumstances not under their control
>> (in violation of the intent of this specification but sometimes
>> unavoidable) SHOULD, to maintain the robustness of the mail system,
>> treat the mail transaction as if a 451 response had been received and
>> act accordingly.
>> </blockquote>
>>
>> AIUI transport.close() sends a QUIT and then forcably closes the
>> socket. the client should assume that an exception means that 451
>> 'Requested action aborted: error in processing' has been returned by
>> the QUIT. in other words, that the QUIT failed and cannot be retried.
>
> Yes; Transport.sendMessage() sends DATA and the terminating dot and
> Transport.close() sends QUIT. In this particular case the client
> cannot send QUIT because the server has already closed the connection.
agreed
>> does this mean that the previous send should be assumed to be rolled back?
>
> The RFC speaks of the "mail transaction" so I'd say yes. 451 is a
> Transient Negative Completion reply so the client should retry later
> (exactly what James does).
i don't have a deep understanding of the RFC so hopefully someone who
does will jump in here
otherwise, maybe we need to ask on the IEFT list...
> In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
> intentionally close the transmission channel until it receives and
> replies to a QUIT command".
yes
however, this behaviour is a real PITA for servers and enables DOS
attacks. i would expect most commercial SMTP servers to allow this
part of the specification to be ignored. i expect most servers just
set a timeout on the socket (AIUI james does this by default) so the
connection will timeout after a suitable period of inactivity. the
problem is that this server is particularly (and unusually) aggressive
in terminating the connection.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Fri, Apr 17, 2009 at 12:43 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> On Fri, Apr 17, 2009 at 10:22 AM, Stefano Bagnara <ap...@bago.org> wrote:
>> Markus Wiederkehr ha scritto:
>>> On Thu, Apr 16, 2009 at 11:05 PM, Robert Burrell Donkin
>>> <ro...@gmail.com> wrote:
>>>> On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
>>>> <ma...@gmail.com> wrote:
>>>>> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>>>>>> Robert Burrell Donkin ha scritto:
>>>>>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>>>>>> <ma...@gmail.com> wrote:
>>>>>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>>>>>> gateway. This particular mail receiver closes the TCP connection after
>>>>>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>>>>>> without waiting for a QUIT command to be sent by the client (James in
>>>>>>>> this case).
>>>>>>>>
>>>>>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>>>>>> MUST NOT intentionally close the transmission channel until it
>>>>>>>> receives and replies to a QUIT command".
>>>>>>>>
>>>>>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>>>>>> to change the behavior of the software. They claim that this behavior
>>>>>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>>>>>> port numbers (which is BS in my opinion).
>>>>>>>>
>>>>>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>>>>>> something like this:
>>>>>>>>
>>>>>>>> try { ...
>>>>>>>> transport.sendMessage(message, addr);
>>>>>>>> } finally {
>>>>>>>> transport.close();
>>>>>>>> }
>>>>>>>>
>>>>>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>>>>>> James (correctly) thinks it has to send the message again. As a
>>>>>>>> consequence the recipient receives the same message several times
>>>>>>>> until James gives up permanently.
>>>>>>>>
>>>>>>>> Now, I do not ask for a modification in James since the problem
>>>>>>>> clearly lies on the recipient's side...
>>>>>>> i'm not sure that an unchecked close is correct in this case. i would
>>>>>>> prefer to see an IO action in a finally wrapped in a exception
>>>>>>> handling block eg
>>>>>>>
>>>>>>> try { ...
>>>>>>> transport.sendMessage(message, addr);
>>>>>>> } finally {
>>>>>>> try {
>>>>>>> transport.close();
>>>>>>> } catch (Exception e) {
>>>>>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> if the action needs to be repeated then a boolean should be used. this
>>>>>>> should ensure that both exceptions are logging and the appropriate one
>>>>>>> processed.
>>>>>>>
>>>>>>> FWIW i would support altering the code without the additional boolean
>>>>>> +1
>>>>> That would solve the problem but I think the behavior of James would
>>>>> then become non-standard compliant.
>>>>>
>>>>> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
>>>>> experience a connection close, reset [...] SHOULD [...] treat the mail
>>>>> transaction as if a 451 response had been received and act
>>>>> accordingly."
>>>> <blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
>>>> SMTP clients that experience a connection close, reset, or other
>>>> communications failure due to circumstances not under their control
>>>> (in violation of the intent of this specification but sometimes
>>>> unavoidable) SHOULD, to maintain the robustness of the mail system,
>>>> treat the mail transaction as if a 451 response had been received and
>>>> act accordingly.
>>>> </blockquote>
>>>>
>>>> AIUI transport.close() sends a QUIT and then forcably closes the
>>>> socket. the client should assume that an exception means that 451
>>>> 'Requested action aborted: error in processing' has been returned by
>>>> the QUIT. in other words, that the QUIT failed and cannot be retried.
>>>
>>> Yes; Transport.sendMessage() sends DATA and the terminating dot and
>>> Transport.close() sends QUIT. In this particular case the client
>>> cannot send QUIT because the server has already closed the connection.
>>>
>>>> does this mean that the previous send should be assumed to be rolled back?
>>>
>>> The RFC speaks of the "mail transaction" so I'd say yes. 451 is a
>>> Transient Negative Completion reply so the client should retry later
>>> (exactly what James does).
>>
>> The rfc says that the time between "." and the reply to the "." should
>> be as short as possible to avoid duplicate send.
>>
>> The email is considered successfully SENT when the reply to "." is read.
>> What happen after this command is a new transaction.
>>
>> I have almost no doubts about this.
>
> Section 3.3. Mail Transactions seems to indicate that you are right
> about this. A mail transaction basically consists of MAIL, RCPT and
> DATA..
>
> So the current behaviour of RemoteDelivery indeed seems to be a bug.
want to open a JIRA and commit a fix?
(or shall i do it)
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Markus Wiederkehr <ma...@gmail.com>.
On Fri, Apr 17, 2009 at 10:22 AM, Stefano Bagnara <ap...@bago.org> wrote:
> Markus Wiederkehr ha scritto:
>> On Thu, Apr 16, 2009 at 11:05 PM, Robert Burrell Donkin
>> <ro...@gmail.com> wrote:
>>> On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
>>> <ma...@gmail.com> wrote:
>>>> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>>>>> Robert Burrell Donkin ha scritto:
>>>>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>>>>> <ma...@gmail.com> wrote:
>>>>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>>>>> gateway. This particular mail receiver closes the TCP connection after
>>>>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>>>>> without waiting for a QUIT command to be sent by the client (James in
>>>>>>> this case).
>>>>>>>
>>>>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>>>>> MUST NOT intentionally close the transmission channel until it
>>>>>>> receives and replies to a QUIT command".
>>>>>>>
>>>>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>>>>> to change the behavior of the software. They claim that this behavior
>>>>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>>>>> port numbers (which is BS in my opinion).
>>>>>>>
>>>>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>>>>> something like this:
>>>>>>>
>>>>>>> try { ...
>>>>>>> transport.sendMessage(message, addr);
>>>>>>> } finally {
>>>>>>> transport.close();
>>>>>>> }
>>>>>>>
>>>>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>>>>> James (correctly) thinks it has to send the message again. As a
>>>>>>> consequence the recipient receives the same message several times
>>>>>>> until James gives up permanently.
>>>>>>>
>>>>>>> Now, I do not ask for a modification in James since the problem
>>>>>>> clearly lies on the recipient's side...
>>>>>> i'm not sure that an unchecked close is correct in this case. i would
>>>>>> prefer to see an IO action in a finally wrapped in a exception
>>>>>> handling block eg
>>>>>>
>>>>>> try { ...
>>>>>> transport.sendMessage(message, addr);
>>>>>> } finally {
>>>>>> try {
>>>>>> transport.close();
>>>>>> } catch (Exception e) {
>>>>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> if the action needs to be repeated then a boolean should be used. this
>>>>>> should ensure that both exceptions are logging and the appropriate one
>>>>>> processed.
>>>>>>
>>>>>> FWIW i would support altering the code without the additional boolean
>>>>> +1
>>>> That would solve the problem but I think the behavior of James would
>>>> then become non-standard compliant.
>>>>
>>>> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
>>>> experience a connection close, reset [...] SHOULD [...] treat the mail
>>>> transaction as if a 451 response had been received and act
>>>> accordingly."
>>> <blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
>>> SMTP clients that experience a connection close, reset, or other
>>> communications failure due to circumstances not under their control
>>> (in violation of the intent of this specification but sometimes
>>> unavoidable) SHOULD, to maintain the robustness of the mail system,
>>> treat the mail transaction as if a 451 response had been received and
>>> act accordingly.
>>> </blockquote>
>>>
>>> AIUI transport.close() sends a QUIT and then forcably closes the
>>> socket. the client should assume that an exception means that 451
>>> 'Requested action aborted: error in processing' has been returned by
>>> the QUIT. in other words, that the QUIT failed and cannot be retried.
>>
>> Yes; Transport.sendMessage() sends DATA and the terminating dot and
>> Transport.close() sends QUIT. In this particular case the client
>> cannot send QUIT because the server has already closed the connection.
>>
>>> does this mean that the previous send should be assumed to be rolled back?
>>
>> The RFC speaks of the "mail transaction" so I'd say yes. 451 is a
>> Transient Negative Completion reply so the client should retry later
>> (exactly what James does).
>
> The rfc says that the time between "." and the reply to the "." should
> be as short as possible to avoid duplicate send.
>
> The email is considered successfully SENT when the reply to "." is read.
> What happen after this command is a new transaction.
>
> I have almost no doubts about this.
Section 3.3. Mail Transactions seems to indicate that you are right
about this. A mail transaction basically consists of MAIL, RCPT and
DATA..
So the current behaviour of RemoteDelivery indeed seems to be a bug.
Thanks,
Markus
>
>> In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
>> intentionally close the transmission channel until it receives and
>> replies to a QUIT command".
>>>> Maybe a switch would be in order? But if nobody else ever had this
>>>> problem I can also write a workaround locally without affecting the
>>>> code base.
>>> possibly
>>>
>>>>>>> But has anyone ever experienced something like this? And maybe is
>>>>>>> there a workaround?
>>>>>> (hopefully stefano or norman will jump in here)
>>>>> I'm using a custom RemoteDelivery but (if I remember correctly) that
>>>>> case should be handled the same. I never had a "duplicate delivery"
>>>>> issue reported.
>>>>>
>>>>> I deliver a lot of mail nowadays, but mainly to italian recipients:
>>>>> maybe the antivirus gateway Markus talks about is not so used in italy.
>>>>> Markus, can you name the product?
>>>> I would rather not; or at least not until I have spoken to my
>>>> co-workers. The company is located in Austria but I have no idea how
>>>> widely the product has been adopted.
>>> you may want to take this off list to stefano directly or perhaps hook up on IRC
>>
>> Good idea, thanks.
>>
>> Markus
>
> I'm often on #james@FreeNode, icq uin 16554317, skype bagovoip, aim
> instantbago .
>
> Stefano
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Stefano Bagnara <ap...@bago.org>.
Markus Wiederkehr ha scritto:
> On Thu, Apr 16, 2009 at 11:05 PM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
>> <ma...@gmail.com> wrote:
>>> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>>>> Robert Burrell Donkin ha scritto:
>>>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>>>> <ma...@gmail.com> wrote:
>>>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>>>> gateway. This particular mail receiver closes the TCP connection after
>>>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>>>> without waiting for a QUIT command to be sent by the client (James in
>>>>>> this case).
>>>>>>
>>>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>>>> MUST NOT intentionally close the transmission channel until it
>>>>>> receives and replies to a QUIT command".
>>>>>>
>>>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>>>> to change the behavior of the software. They claim that this behavior
>>>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>>>> port numbers (which is BS in my opinion).
>>>>>>
>>>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>>>> something like this:
>>>>>>
>>>>>> try { ...
>>>>>> transport.sendMessage(message, addr);
>>>>>> } finally {
>>>>>> transport.close();
>>>>>> }
>>>>>>
>>>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>>>> James (correctly) thinks it has to send the message again. As a
>>>>>> consequence the recipient receives the same message several times
>>>>>> until James gives up permanently.
>>>>>>
>>>>>> Now, I do not ask for a modification in James since the problem
>>>>>> clearly lies on the recipient's side...
>>>>> i'm not sure that an unchecked close is correct in this case. i would
>>>>> prefer to see an IO action in a finally wrapped in a exception
>>>>> handling block eg
>>>>>
>>>>> try { ...
>>>>> transport.sendMessage(message, addr);
>>>>> } finally {
>>>>> try {
>>>>> transport.close();
>>>>> } catch (Exception e) {
>>>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>>>> }
>>>>> }
>>>>>
>>>>> if the action needs to be repeated then a boolean should be used. this
>>>>> should ensure that both exceptions are logging and the appropriate one
>>>>> processed.
>>>>>
>>>>> FWIW i would support altering the code without the additional boolean
>>>> +1
>>> That would solve the problem but I think the behavior of James would
>>> then become non-standard compliant.
>>>
>>> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
>>> experience a connection close, reset [...] SHOULD [...] treat the mail
>>> transaction as if a 451 response had been received and act
>>> accordingly."
>> <blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
>> SMTP clients that experience a connection close, reset, or other
>> communications failure due to circumstances not under their control
>> (in violation of the intent of this specification but sometimes
>> unavoidable) SHOULD, to maintain the robustness of the mail system,
>> treat the mail transaction as if a 451 response had been received and
>> act accordingly.
>> </blockquote>
>>
>> AIUI transport.close() sends a QUIT and then forcably closes the
>> socket. the client should assume that an exception means that 451
>> 'Requested action aborted: error in processing' has been returned by
>> the QUIT. in other words, that the QUIT failed and cannot be retried.
>
> Yes; Transport.sendMessage() sends DATA and the terminating dot and
> Transport.close() sends QUIT. In this particular case the client
> cannot send QUIT because the server has already closed the connection.
>
>> does this mean that the previous send should be assumed to be rolled back?
>
> The RFC speaks of the "mail transaction" so I'd say yes. 451 is a
> Transient Negative Completion reply so the client should retry later
> (exactly what James does).
The rfc says that the time between "." and the reply to the "." should
be as short as possible to avoid duplicate send.
The email is considered successfully SENT when the reply to "." is read.
What happen after this command is a new transaction.
I have almost no doubts about this.
> In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
> intentionally close the transmission channel until it receives and
> replies to a QUIT command".
>>> Maybe a switch would be in order? But if nobody else ever had this
>>> problem I can also write a workaround locally without affecting the
>>> code base.
>> possibly
>>
>>>>>> But has anyone ever experienced something like this? And maybe is
>>>>>> there a workaround?
>>>>> (hopefully stefano or norman will jump in here)
>>>> I'm using a custom RemoteDelivery but (if I remember correctly) that
>>>> case should be handled the same. I never had a "duplicate delivery"
>>>> issue reported.
>>>>
>>>> I deliver a lot of mail nowadays, but mainly to italian recipients:
>>>> maybe the antivirus gateway Markus talks about is not so used in italy.
>>>> Markus, can you name the product?
>>> I would rather not; or at least not until I have spoken to my
>>> co-workers. The company is located in Austria but I have no idea how
>>> widely the product has been adopted.
>> you may want to take this off list to stefano directly or perhaps hook up on IRC
>
> Good idea, thanks.
>
> Markus
I'm often on #james@FreeNode, icq uin 16554317, skype bagovoip, aim
instantbago .
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Markus Wiederkehr <ma...@gmail.com>.
On Thu, Apr 16, 2009 at 11:05 PM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
> <ma...@gmail.com> wrote:
>> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>>> Robert Burrell Donkin ha scritto:
>>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>>> <ma...@gmail.com> wrote:
>>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>>> gateway. This particular mail receiver closes the TCP connection after
>>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>>> without waiting for a QUIT command to be sent by the client (James in
>>>>> this case).
>>>>>
>>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>>> MUST NOT intentionally close the transmission channel until it
>>>>> receives and replies to a QUIT command".
>>>>>
>>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>>> to change the behavior of the software. They claim that this behavior
>>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>>> port numbers (which is BS in my opinion).
>>>>>
>>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>>> something like this:
>>>>>
>>>>> try { ...
>>>>> transport.sendMessage(message, addr);
>>>>> } finally {
>>>>> transport.close();
>>>>> }
>>>>>
>>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>>> James (correctly) thinks it has to send the message again. As a
>>>>> consequence the recipient receives the same message several times
>>>>> until James gives up permanently.
>>>>>
>>>>> Now, I do not ask for a modification in James since the problem
>>>>> clearly lies on the recipient's side...
>>>>
>>>> i'm not sure that an unchecked close is correct in this case. i would
>>>> prefer to see an IO action in a finally wrapped in a exception
>>>> handling block eg
>>>>
>>>> try { ...
>>>> transport.sendMessage(message, addr);
>>>> } finally {
>>>> try {
>>>> transport.close();
>>>> } catch (Exception e) {
>>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>>> }
>>>> }
>>>>
>>>> if the action needs to be repeated then a boolean should be used. this
>>>> should ensure that both exceptions are logging and the appropriate one
>>>> processed.
>>>>
>>>> FWIW i would support altering the code without the additional boolean
>>>
>>> +1
>>
>> That would solve the problem but I think the behavior of James would
>> then become non-standard compliant.
>>
>> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
>> experience a connection close, reset [...] SHOULD [...] treat the mail
>> transaction as if a 451 response had been received and act
>> accordingly."
>
> <blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
> SMTP clients that experience a connection close, reset, or other
> communications failure due to circumstances not under their control
> (in violation of the intent of this specification but sometimes
> unavoidable) SHOULD, to maintain the robustness of the mail system,
> treat the mail transaction as if a 451 response had been received and
> act accordingly.
> </blockquote>
>
> AIUI transport.close() sends a QUIT and then forcably closes the
> socket. the client should assume that an exception means that 451
> 'Requested action aborted: error in processing' has been returned by
> the QUIT. in other words, that the QUIT failed and cannot be retried.
Yes; Transport.sendMessage() sends DATA and the terminating dot and
Transport.close() sends QUIT. In this particular case the client
cannot send QUIT because the server has already closed the connection.
> does this mean that the previous send should be assumed to be rolled back?
The RFC speaks of the "mail transaction" so I'd say yes. 451 is a
Transient Negative Completion reply so the client should retry later
(exactly what James does).
In section 4.1.1.10. the RFC also says that "The receiver MUST NOT
intentionally close the transmission channel until it receives and
replies to a QUIT command".
>> Maybe a switch would be in order? But if nobody else ever had this
>> problem I can also write a workaround locally without affecting the
>> code base.
>
> possibly
>
>>>>> But has anyone ever experienced something like this? And maybe is
>>>>> there a workaround?
>>>>
>>>> (hopefully stefano or norman will jump in here)
>>>
>>> I'm using a custom RemoteDelivery but (if I remember correctly) that
>>> case should be handled the same. I never had a "duplicate delivery"
>>> issue reported.
>>>
>>> I deliver a lot of mail nowadays, but mainly to italian recipients:
>>> maybe the antivirus gateway Markus talks about is not so used in italy.
>>> Markus, can you name the product?
>>
>> I would rather not; or at least not until I have spoken to my
>> co-workers. The company is located in Austria but I have no idea how
>> widely the product has been adopted.
>
> you may want to take this off list to stefano directly or perhaps hook up on IRC
Good idea, thanks.
Markus
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Thu, Apr 16, 2009 at 8:12 PM, Markus Wiederkehr
<ma...@gmail.com> wrote:
> On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
>> Robert Burrell Donkin ha scritto:
>>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>>> <ma...@gmail.com> wrote:
>>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>>> gateway. This particular mail receiver closes the TCP connection after
>>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>>> without waiting for a QUIT command to be sent by the client (James in
>>>> this case).
>>>>
>>>> This is a clear violation of RFC 5321 which says that "the receiver
>>>> MUST NOT intentionally close the transmission channel until it
>>>> receives and replies to a QUIT command".
>>>>
>>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>>> to change the behavior of the software. They claim that this behavior
>>>> is common for bigger providers in order to avoid DDOS attacks and save
>>>> port numbers (which is BS in my opinion).
>>>>
>>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>>> something like this:
>>>>
>>>> try { ...
>>>> transport.sendMessage(message, addr);
>>>> } finally {
>>>> transport.close();
>>>> }
>>>>
>>>> So sendMessage() succeeds but close() throws a MessagingException and
>>>> James (correctly) thinks it has to send the message again. As a
>>>> consequence the recipient receives the same message several times
>>>> until James gives up permanently.
>>>>
>>>> Now, I do not ask for a modification in James since the problem
>>>> clearly lies on the recipient's side...
>>>
>>> i'm not sure that an unchecked close is correct in this case. i would
>>> prefer to see an IO action in a finally wrapped in a exception
>>> handling block eg
>>>
>>> try { ...
>>> transport.sendMessage(message, addr);
>>> } finally {
>>> try {
>>> transport.close();
>>> } catch (Exception e) {
>>> getLog().warn("Failed to cleanly close connection to remote server", e);
>>> }
>>> }
>>>
>>> if the action needs to be repeated then a boolean should be used. this
>>> should ensure that both exceptions are logging and the appropriate one
>>> processed.
>>>
>>> FWIW i would support altering the code without the additional boolean
>>
>> +1
>
> That would solve the problem but I think the behavior of James would
> then become non-standard compliant.
>
> See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
> experience a connection close, reset [...] SHOULD [...] treat the mail
> transaction as if a 451 response had been received and act
> accordingly."
<blockquote cite='http://tools.ietf.org/html/rfc5321#section-3.8'>
SMTP clients that experience a connection close, reset, or other
communications failure due to circumstances not under their control
(in violation of the intent of this specification but sometimes
unavoidable) SHOULD, to maintain the robustness of the mail system,
treat the mail transaction as if a 451 response had been received and
act accordingly.
</blockquote>
AIUI transport.close() sends a QUIT and then forcably closes the
socket. the client should assume that an exception means that 451
'Requested action aborted: error in processing' has been returned by
the QUIT. in other words, that the QUIT failed and cannot be retried.
does this mean that the previous send should be assumed to be rolled back?
> Maybe a switch would be in order? But if nobody else ever had this
> problem I can also write a workaround locally without affecting the
> code base.
possibly
>>>> But has anyone ever experienced something like this? And maybe is
>>>> there a workaround?
>>>
>>> (hopefully stefano or norman will jump in here)
>>
>> I'm using a custom RemoteDelivery but (if I remember correctly) that
>> case should be handled the same. I never had a "duplicate delivery"
>> issue reported.
>>
>> I deliver a lot of mail nowadays, but mainly to italian recipients:
>> maybe the antivirus gateway Markus talks about is not so used in italy.
>> Markus, can you name the product?
>
> I would rather not; or at least not until I have spoken to my
> co-workers. The company is located in Austria but I have no idea how
> widely the product has been adopted.
you may want to take this off list to stefano directly or perhaps hook up on IRC
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Markus Wiederkehr <ma...@gmail.com>.
On Thu, Apr 16, 2009 at 8:40 PM, Stefano Bagnara <ap...@bago.org> wrote:
> Robert Burrell Donkin ha scritto:
>> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
>> <ma...@gmail.com> wrote:
>>> I have a problem with a manufacturer of a certain anti-virus e-mail
>>> gateway. This particular mail receiver closes the TCP connection after
>>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>>> without waiting for a QUIT command to be sent by the client (James in
>>> this case).
>>>
>>> This is a clear violation of RFC 5321 which says that "the receiver
>>> MUST NOT intentionally close the transmission channel until it
>>> receives and replies to a QUIT command".
>>>
>>> Unfortunately the manufacturer is aware of this fact but still refuses
>>> to change the behavior of the software. They claim that this behavior
>>> is common for bigger providers in order to avoid DDOS attacks and save
>>> port numbers (which is BS in my opinion).
>>>
>>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>>> something like this:
>>>
>>> try { ...
>>> transport.sendMessage(message, addr);
>>> } finally {
>>> transport.close();
>>> }
>>>
>>> So sendMessage() succeeds but close() throws a MessagingException and
>>> James (correctly) thinks it has to send the message again. As a
>>> consequence the recipient receives the same message several times
>>> until James gives up permanently.
>>>
>>> Now, I do not ask for a modification in James since the problem
>>> clearly lies on the recipient's side...
>>
>> i'm not sure that an unchecked close is correct in this case. i would
>> prefer to see an IO action in a finally wrapped in a exception
>> handling block eg
>>
>> try { ...
>> transport.sendMessage(message, addr);
>> } finally {
>> try {
>> transport.close();
>> } catch (Exception e) {
>> getLog().warn("Failed to cleanly close connection to remote server", e);
>> }
>> }
>>
>> if the action needs to be repeated then a boolean should be used. this
>> should ensure that both exceptions are logging and the appropriate one
>> processed.
>>
>> FWIW i would support altering the code without the additional boolean
>
> +1
That would solve the problem but I think the behavior of James would
then become non-standard compliant.
See http://tools.ietf.org/html/rfc5321#section-3.8: "SMTP clients that
experience a connection close, reset [...] SHOULD [...] treat the mail
transaction as if a 451 response had been received and act
accordingly."
Maybe a switch would be in order? But if nobody else ever had this
problem I can also write a workaround locally without affecting the
code base.
>>> But has anyone ever experienced something like this? And maybe is
>>> there a workaround?
>>
>> (hopefully stefano or norman will jump in here)
>
> I'm using a custom RemoteDelivery but (if I remember correctly) that
> case should be handled the same. I never had a "duplicate delivery"
> issue reported.
>
> I deliver a lot of mail nowadays, but mainly to italian recipients:
> maybe the antivirus gateway Markus talks about is not so used in italy.
> Markus, can you name the product?
I would rather not; or at least not until I have spoken to my
co-workers. The company is located in Austria but I have no idea how
widely the product has been adopted.
Markus
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Problem with RemoteDelivery with non standard compliant mail
receiver
Posted by Stefano Bagnara <ap...@bago.org>.
Robert Burrell Donkin ha scritto:
> On Thu, Apr 16, 2009 at 3:32 PM, Markus Wiederkehr
> <ma...@gmail.com> wrote:
>> I have a problem with a manufacturer of a certain anti-virus e-mail
>> gateway. This particular mail receiver closes the TCP connection after
>> receiving the terminating <CRLF>.<CRLF> indicator of the DATA command
>> without waiting for a QUIT command to be sent by the client (James in
>> this case).
>>
>> This is a clear violation of RFC 5321 which says that "the receiver
>> MUST NOT intentionally close the transmission channel until it
>> receives and replies to a QUIT command".
>>
>> Unfortunately the manufacturer is aware of this fact but still refuses
>> to change the behavior of the software. They claim that this behavior
>> is common for bigger providers in order to avoid DDOS attacks and save
>> port numbers (which is BS in my opinion).
>>
>> Anyway, RemoteDelivery uses the Java Mail API to send the message,
>> something like this:
>>
>> try { ...
>> transport.sendMessage(message, addr);
>> } finally {
>> transport.close();
>> }
>>
>> So sendMessage() succeeds but close() throws a MessagingException and
>> James (correctly) thinks it has to send the message again. As a
>> consequence the recipient receives the same message several times
>> until James gives up permanently.
>>
>> Now, I do not ask for a modification in James since the problem
>> clearly lies on the recipient's side...
>
> i'm not sure that an unchecked close is correct in this case. i would
> prefer to see an IO action in a finally wrapped in a exception
> handling block eg
>
> try { ...
> transport.sendMessage(message, addr);
> } finally {
> try {
> transport.close();
> } catch (Exception e) {
> getLog().warn("Failed to cleanly close connection to remote server", e);
> }
> }
>
> if the action needs to be repeated then a boolean should be used. this
> should ensure that both exceptions are logging and the appropriate one
> processed.
>
> FWIW i would support altering the code without the additional boolean
+1
>> But has anyone ever experienced something like this? And maybe is
>> there a workaround?
>
> (hopefully stefano or norman will jump in here)
I'm using a custom RemoteDelivery but (if I remember correctly) that
case should be handled the same. I never had a "duplicate delivery"
issue reported.
I deliver a lot of mail nowadays, but mainly to italian recipients:
maybe the antivirus gateway Markus talks about is not so used in italy.
Markus, can you name the product?
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org