You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Soeren Hilmer <so...@tietoenator.com> on 2003/10/02 10:14:13 UTC
RemoteDelivery progressively increasing delay-time
Hi,
I have been thinking about extending RemoteDelivery to take a list of
delaytimes.
Something like:
<delayTime> 600 </delayTime>
<delayTime> 1600000 </delayTime>
<delayTime> 21600000 </delayTime>
<delayTime> 21600000 </delayTime>
The purpose being that I have often observed that when a server does not reply
it often replys just after (probably due to some timeout on the first
request). The proposed change will thus allow one to progressively degrade
one's faith in the mail ever getting delivered.
If the method is used the maxRetries of course will not have effect.
Should I go ahead with such a patch?
--Søren
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax: +45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer@tietoenator.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: RemoteDelivery progressively increasing delay-time
Posted by Serge Knystautas <se...@lokitech.com>.
Soeren Hilmer wrote:
> I do not really know how this process works, who has the final word on which
> of the proposed ways to do this, we go for?
The person who does the work gets by far the most say. The community
can suggest designs and may in special cases veto it.
> I like Serge's "attempts" version best, but in this case are maxRetries then
> to be disregarded? Or is it to be used on the last delaytime entry, if the
> attempts attribute is missing? (assuming that attempts defaults to 1 if not
> specified) As this last version is the most versatile, I believe this is the
> route to go.
Yeah, I had forgotten we couldn't use attributes, so perhaps an optional
delimiter.
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: RemoteDelivery progressively increasing delay-time
Posted by "Richard O. Hammer" <RO...@EarthLink.net>.
Regarding the timing of subsequent tries to overcome network failures,
I have seen the phrase "exponential backoff strategy" used. I thought
I had seen that in RFC 2821. But if it were there I expect Noel would
have reported it.
As I understand it, the delays in an exponential backoff strategy
would be computed as in the following example.
If the goal is to have 5 retries which end after 5 days, then my
calculator says the first delay should be about 13.4 seconds (5
days)**(1/5). (Where ** means "raised to the power of".)
13.4**1 = 13 ~= 13 seconds
13.4**2 = 180 ~= 3 minutes
13.4**3 = 2,406 ~= 40 minutes
13.4**4 = 32,242 ~= 9 hours
13.4**5 = 432,040 ~= 5 days
Rich Hammer
Hillsborough, N.C.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by Steve Brewin <sb...@synsys.com>.
Noel J. Bergman wrote:
> An reasonable approach, but wrong execution. Whenever you
> have multiple
> <tag> entries, the resulting value provided by the Mailet API
> is going to be
> a CSV (comma separated value) String whose order reflects the
> order of the
> <tag> elements. Therefore using a comma to delimit optional
> values within a
> tag will be ambiguous with having multiple tags.
OK, I was having a Baldrick momement, aka "I have a cunning plan, my
Lord..." (see http://www.mgnet.karoo.net/blackadder2characters.htm if you
don't have a clue what I'm on about and really want to know). Using commas
was stupid, other delimiters such as * should work fine.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by "Noel J. Bergman" <no...@devtech.com>.
> You could work around this by optionally comma delimiting an attempts
> value in the <delayTime> tag...
An reasonable approach, but wrong execution. Whenever you have multiple
<tag> entries, the resulting value provided by the Mailet API is going to be
a CSV (comma separated value) String whose order reflects the order of the
<tag> elements. Therefore using a comma to delimit optional values within a
tag will be ambiguous with having multiple tags.
To use your example:
> <delayTime> msecs [, attempts]</delayTime>
> <delayTime> 600000 </delayTime>
> <delayTime> 600000, 3</delayTime>
That won't work because the resulting CSV would be "600000,600000,3", which
is ambiguous. I don't like the idea that the ambiguity is resolved by
saying that 3 is too low a number to actually be a delay value.
Some other format expressing repetition and/or units of time would be
acceptable, so long as the resulting CSV remains unambiguous. Since I'm not
writing the parser, I'll leave the format up to the parser author, so long
as it is reasonable. :-) As a mere suggestion, something like:
<delayTime> [attempts*]value[unit]</delayTime>
<delayTime> 600000 </delayTime>
<delayTime> 3*600000</delayTime>
would be unambiguous: "600000,3*600000". Attempts defaults to 1, unit
defaults to milliseconds. We could easily handle the attempts today, and
add support for other types of time [sec, min, hours, days, whatever]
whenever someone gets an itch to scratch.
I agree that the last value should be used for all retries up to the max for
which there isn't a specified delay.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by Steve Brewin <sb...@synsys.com>.
Soeren Hilmer wrote:
> I like Serge's "attempts" version best,
But as Noel noted in his reponse to Serge, the Mailet API does not support
it. You could work around this by optionally comma delimiting an attempts
value in the <delayTime> tag...
<delayTime> msecs [, attempts]</delayTime>
<delayTime> 600000 </delayTime>
<delayTime> 600000, 3</delayTime>
> but in this case are
> maxRetries then
> to be disregarded? Or is it to be used on the last delaytime
> entry, if the
> attempts attribute is missing? (assuming that attempts
> defaults to 1 if not
> specified)
I do not believe the maxRetries value should ever be disregarded.
(1) When maxRetries > #delayTimes * #attempts the value of the last delay
time should be used after #delayTimes * #attempts. It does not matter if
this last delay time has an attempts value or not.
(2) When maxRetries < #delayTimes * #attempts we have a conflict!
Personally, I would honour the maxRetries value which is symetrical with
(1).
The nice thing about this is the current syntax still works and you get the
versatility you require.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: RemoteDelivery progressively increasing delay-time
Posted by Soeren Hilmer <so...@tietoenator.com>.
Hi,
I do not really know how this process works, who has the final word on which
of the proposed ways to do this, we go for?
I like Serge's "attempts" version best, but in this case are maxRetries then
to be disregarded? Or is it to be used on the last delaytime entry, if the
attempts attribute is missing? (assuming that attempts defaults to 1 if not
specified) As this last version is the most versatile, I believe this is the
route to go.
I would like to start on this monday, so ...
--Søren
On Thursday 02 October 2003 19:38, Serge Knystautas wrote:
> Soeren Hilmer wrote:
> > I have been thinking about extending RemoteDelivery to take a list of
> > delaytimes.
> >
> > Something like:
> > <delayTime> 600 </delayTime>
> > <delayTime> 1600000 </delayTime>
> > <delayTime> 21600000 </delayTime>
> > <delayTime> 21600000 </delayTime>
>
> I like this, but would suggest an attribute named "attempts" to each
> element. For example, <delayTime attempts="3"> 600 </delayTime> would
> try 3 times at 600ms delays before moving to the next delayTime element.
--
Søren Hilmer, M.Sc.
R&D manager Phone: +45 70 27 64 00
TietoEnator IT+ A/S Fax: +45 70 27 64 40
Ved Lunden 12 Direct: +45 87 46 64 57
DK-8230 Åbyhøj Email: soren.hilmer@tietoenator.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by Steve Brewin <sb...@synsys.com>.
Noel J. Bergman wrote:
>
> This is not exposed to Mailet API components. We only get
> [name,value]. As
> noted in RemoteDelivery.init(), the resulting value is an
> ordered CSV (which
> answers a question Steve just asked).
As noted in org.apache.james.transport.mailets.RemoteDelivery.init()? I
don't see comments there relating to this!
What I do see is org.apache.james.core.MailetConfigImpl wherein method
getInitParameter(String name) iterates through occurences of the named child
of the passed Configuration to create a CSV delimited String. Most
importantly, the javadoc for
org.apache.avalon.framework.configuration.Configuration.getChildren(String)
says "The array order will reflect the order in the source config file."
So, the name/value pairs in the CSV String answered by
getInitParameter(String name) will be in the order the tags were encountered
in the config. file. We can just use that order and disregard any thoughts I
had about needing to impose a sort order.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I like this, but would suggest an attribute named "attempts" to each
> element. For example, <delayTime attempts="3"> 600 </delayTime> would
> try 3 times at 600ms delays before moving to the next delayTime element.
This is not exposed to Mailet API components. We only get [name,value]. As
noted in RemoteDelivery.init(), the resulting value is an ordered CSV (which
answers a question Steve just asked).
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: RemoteDelivery progressively increasing delay-time
Posted by Serge Knystautas <se...@lokitech.com>.
Soeren Hilmer wrote:
> I have been thinking about extending RemoteDelivery to take a list of
> delaytimes.
>
> Something like:
> <delayTime> 600 </delayTime>
> <delayTime> 1600000 </delayTime>
> <delayTime> 21600000 </delayTime>
> <delayTime> 21600000 </delayTime>
I like this, but would suggest an attribute named "attempts" to each
element. For example, <delayTime attempts="3"> 600 </delayTime> would
try 3 times at 600ms delays before moving to the next delayTime element.
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Still, I would retain <maxRetrys> tag and use the last of the
> <delayTime> tags as the time to use where maxRetrys > number
> of delayTime tags.
Agreed, as I noted in my response to Soren.
> As I am unsure that same named tags are returned in the sequence they
> are entered, I might enforce successively longer delay times and define
> 'last' as the longest.
We could, but I do believe that order is guaranteed by Avalon. Stephen
McConnell ought to be able to let us know. If so, I'd prefer to leave it
ordered and not sort.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by Steve Brewin <sb...@synsys.com>.
Noel J. Bergman wrote:
>
> > Personally, if the change was to be made, I would prefer
> something like...
>
> I understand why, but it would require a change to the data
> model we expose
> via the Mailet API. The current API supports neither child
> elements nor
> attributes. The proposed change is compatible with the
> current data model.
Fair point! Still, I would retain <maxRetrys> tag and use the last of the
<delayTime> tags as the time to use where maxRetrys > number of delayTime
tags. As I am unsure that same named tags are returned in the sequence they
are entered, I might enforce successively longer delay times and define
'last' as the longest.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Personally, if the change was to be made, I would prefer something like...
I understand why, but it would require a change to the data model we expose
via the Mailet API. The current API supports neither child elements nor
attributes. The proposed change is compatible with the current data model.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by Steve Brewin <sb...@synsys.com>.
Soeren Hilmer wrote:
>
> I have been thinking about extending RemoteDelivery to take a list of
> delaytimes.
Seems like a fair idea.
> Something like:
> <delayTime> 600 </delayTime>
> <delayTime> 1600000 </delayTime>
> <delayTime> 21600000 </delayTime>
> <delayTime> 21600000 </delayTime>
Personally, if the change was to be made, I would prefer something like...
<delayTimes>
<!-- First delay -->
<msecs> 600 </msecs>
<!-- Second delay -->
<secs> 1600 </secs>
<!-- Third delay -->
<mins> 360 </mins>
<!-- Delay applied up to maxRetries -->
<hours> 6 </hours>
</delayTimes>
I would keep the <maxRetries> tag and apply the final value in the
<delayTimes> list as the delay to use up to the value of <maxRetries>.
The <delayTime> and <delayTimes> tags would have to be mutually exclusive.
Just my thoughts.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: RemoteDelivery progressively increasing delay-time
Posted by "Noel J. Bergman" <no...@devtech.com>.
Søren,
It has been discussed before, but no one ever implemented it. RFC 2821
section 4.5.4.1 discusses a similar strategy:
Experience suggests that failures are typically transient (the target
system or its connection has crashed), favoring a policy of two
connection attempts in the first hour the message is in the queue,
and then backing off to one every two or three hours.
It also says that:
Retries continue until the message is transmitted or the sender gives
up; the give-up time generally needs to be at least 4-5 days. The
parameters to the retry algorithm MUST be configurable.
So I agree on the idea to support progressive delay times. The last time
should be applied for all further retries. An RFC compliant configuration
might look like:
<!-- 5 minutes -->
<delayTime> 300000 </delayTime>
<!-- 10 minutes -->
<delayTime> 600000 </delayTime>
<!-- 45 minutes -->
<delayTime> 2700000 </delayTime>
<!-- 2 hours -->
<delayTime> 7200000 </delayTime>
<!-- 3 hours -->
<delayTime> 10800000 </delayTime>
<!-- 6 hours -->
<delayTime> 21600000 </delayTime>
<maxRetries> 25 </maxRetries>
That provides a 5 day retry period, with 4 attempts in the first hour, two
more within the first 6 hours, and then every 6 hours for the rest of the
period.
That is quite likely the initial configuration I would test on my server.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org