You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Nick Dunkin <Ni...@vecima.com.INVALID> on 2023/02/20 19:02:25 UTC
Configuring sleep times between parent connection retires?
Hi,
If a parent cache is offline for a brief period of time (say < 3 seconds) I would like to know if there is way to configure a “sleep” between connection retries on a particular parent?
The ability to configure the “connection attempts” is not useful here, as when the parent is completely offline the algorithm can run through many hundreds of connection attempts in milliseconds (confirmed from logs).
The use case here is that, for us, it is preferable to make the client occasionally wait 1-3 seconds, than to return them a HTTP 502.
I’ve started looking at the code, but I’d like to confirm that this is not already supported somehow.
Many thanks
Nick
Nick Dunkin
Director, Software Architecture
Manager – Architecture and New Product Introduction
o: +1 678.258.4071
e: nick.dunkin@vecima.com<ma...@vecima.com>
[cidimage001.png@01D6CC8C.6FC5A580]
Re: Configuring sleep times between parent connection retires?
Posted by Nick Dunkin <Ni...@vecima.com.INVALID>.
Hi John,
Many thanks for the clarification.
Cheers,
Nick
Sent from my iPhone
> On Feb 20, 2023, at 7:20 PM, John Rushford <jj...@gmail.com> wrote:
>
> Yeah, there is no mechanism that would wait within a transaction for a parent to become available again. Within a transaction when a parent is marked down, ATS will try to select another available parent to complete the transaction or go direct to the origin if the config allows it. The closest thing to what you’re asking for is the proxy.config.http.parent_proxy.retry_time. A down parent will be retried once that time has expired and will be marked up if the next transaction succeeds
>
> John.
>
>> On Feb 20, 2023, at 4:18 PM, Nick Dunkin <Ni...@vecima.com.INVALID> wrote:
>>
>> Hi John,
>>
>> Ok, that’s what I thought.
>>
>> It’s the 502s that I’m trying to avoid in this instance.
>>
>> As far as I can tell, the retry mechanisms on a parent do not support any kind of “back off” mechanism, so you can have 100+ retries, but with an offline parent those “connection attempts” are exhausted in milliseconds.
>>
>> I would like to be able to allow a parent to be temporarily “unavailable” for anything up to ~2 seconds, and make the downstream client wait for the parent resource to be available again. It’s a rare edge case, but it’s a real one.
>>
>> The context here Is ABR video streaming, where a client has enough buffer to very occasionally wait a couple of extra seconds for its content, but an HTTP 502 would cause it to abort its stream.
>>
>> Thanks for your assistance,
>>
>> Nick
>>
>> From: John Rushford <jj...@gmail.com>
>> Date: Monday, February 20, 2023 at 6:03 PM
>> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>
>> Cc: Nick Dunkin <Ni...@vecima.com.invalid>
>> Subject: Re: Configuring sleep times between parent connection retires?
>> No a parent is not retried within the same transaction. If the configured number of retry counts is not exceeded, another parent would be chosen if there are other available parents. If there are no other available parents and the configs allow it, the transaction may go direct to the origin, go_direct in parent.config. If there are no other available parents and go_direct == false, the transaction fails and the client sees a 502
>>
>> Sent from my iPhone
>>
>>>> On Feb 20, 2023, at 3:18 PM, Nick Dunkin <Ni...@vecima.com.invalid> wrote:
>>>
>>> Hi John,
>>>
>>> Thank you for the reply.
>>>
>>> Are you suggesting that I could have a parent marked down, with a very short retry_time (~1 second), and it would be retried within the same transaction?
>>>
>>> I would have thought that at the point it’s marked down, the transaction would end?
>>>
>>> Thanks
>>>
>>> Nick
>>>
>>> From: John J. Rushford <jj...@gmail.com>
>>> Date: Monday, February 20, 2023 at 3:21 PM
>>> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>, Nick Dunkin <Ni...@vecima.com.INVALID>
>>> Subject: Re: Configuring sleep times between parent connection retires?
>>> Nick,
>>>
>>> See the Parent Proxy Configurations,
>>> https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
>>>
>>> You may need to do some tweaking. There is a sliding window,
>>> proxy.config.http.parent_proxy.retry_time. When the failure threshold
>>> ,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
>>> the parent is marked down and will not be used until the retry window
>>> has elapsed. when the window has elapsed, a failed parent will be
>>> retried but, if the retry fails, no further attempts will be made to
>>> that parent until the retry window has elapsed again. If the retry
>>> succeeds, the parent is marked up.
>>>
>>> thanks
>>>
>>> John
>>>
>>>> On 2/20/23 12:02, Nick Dunkin wrote:
>>>>
>>>> Hi,
>>>>
>>>> If a parent cache is offline for a brief period of time (say < 3
>>>> seconds) I would like to know if there is way to configure a “sleep”
>>>> between connection retries on a particular parent?
>>>>
>>>> The ability to configure the “connection attempts” is not useful here,
>>>> as when the parent is completely offline the algorithm can run through
>>>> many hundreds of connection attempts in milliseconds (confirmed from
>>>> logs).
>>>>
>>>> The use case here is that, for us, it is preferable to make the client
>>>> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>>>>
>>>> I’ve started looking at the code, but I’d like to confirm that this is
>>>> not already supported somehow.
>>>>
>>>> Many thanks
>>>>
>>>> Nick
>>>>
>>>> *Nick Dunkin*
>>>>
>>>> Director, Software Architecture
>>>>
>>>> Manager – Architecture and New Product Introduction
>>>>
>>>> *o: **+1 678.258.4071*
>>>>
>>>> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>>>>
>>>> cidimage001.png@01D6CC8C.6FC5A580
>>>>
>
Re: Configuring sleep times between parent connection retires?
Posted by John Rushford <jj...@gmail.com>.
Yeah, there is no mechanism that would wait within a transaction for a parent to become available again. Within a transaction when a parent is marked down, ATS will try to select another available parent to complete the transaction or go direct to the origin if the config allows it. The closest thing to what you’re asking for is the proxy.config.http.parent_proxy.retry_time. A down parent will be retried once that time has expired and will be marked up if the next transaction succeeds
John.
> On Feb 20, 2023, at 4:18 PM, Nick Dunkin <Ni...@vecima.com.INVALID> wrote:
>
> Hi John,
>
> Ok, that’s what I thought.
>
> It’s the 502s that I’m trying to avoid in this instance.
>
> As far as I can tell, the retry mechanisms on a parent do not support any kind of “back off” mechanism, so you can have 100+ retries, but with an offline parent those “connection attempts” are exhausted in milliseconds.
>
> I would like to be able to allow a parent to be temporarily “unavailable” for anything up to ~2 seconds, and make the downstream client wait for the parent resource to be available again. It’s a rare edge case, but it’s a real one.
>
> The context here Is ABR video streaming, where a client has enough buffer to very occasionally wait a couple of extra seconds for its content, but an HTTP 502 would cause it to abort its stream.
>
> Thanks for your assistance,
>
> Nick
>
> From: John Rushford <jj...@gmail.com>
> Date: Monday, February 20, 2023 at 6:03 PM
> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>
> Cc: Nick Dunkin <Ni...@vecima.com.invalid>
> Subject: Re: Configuring sleep times between parent connection retires?
> No a parent is not retried within the same transaction. If the configured number of retry counts is not exceeded, another parent would be chosen if there are other available parents. If there are no other available parents and the configs allow it, the transaction may go direct to the origin, go_direct in parent.config. If there are no other available parents and go_direct == false, the transaction fails and the client sees a 502
>
> Sent from my iPhone
>
>> On Feb 20, 2023, at 3:18 PM, Nick Dunkin <Ni...@vecima.com.invalid> wrote:
>>
>> Hi John,
>>
>> Thank you for the reply.
>>
>> Are you suggesting that I could have a parent marked down, with a very short retry_time (~1 second), and it would be retried within the same transaction?
>>
>> I would have thought that at the point it’s marked down, the transaction would end?
>>
>> Thanks
>>
>> Nick
>>
>> From: John J. Rushford <jj...@gmail.com>
>> Date: Monday, February 20, 2023 at 3:21 PM
>> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>, Nick Dunkin <Ni...@vecima.com.INVALID>
>> Subject: Re: Configuring sleep times between parent connection retires?
>> Nick,
>>
>> See the Parent Proxy Configurations,
>> https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
>>
>> You may need to do some tweaking. There is a sliding window,
>> proxy.config.http.parent_proxy.retry_time. When the failure threshold
>> ,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
>> the parent is marked down and will not be used until the retry window
>> has elapsed. when the window has elapsed, a failed parent will be
>> retried but, if the retry fails, no further attempts will be made to
>> that parent until the retry window has elapsed again. If the retry
>> succeeds, the parent is marked up.
>>
>> thanks
>>
>> John
>>
>>> On 2/20/23 12:02, Nick Dunkin wrote:
>>>
>>> Hi,
>>>
>>> If a parent cache is offline for a brief period of time (say < 3
>>> seconds) I would like to know if there is way to configure a “sleep”
>>> between connection retries on a particular parent?
>>>
>>> The ability to configure the “connection attempts” is not useful here,
>>> as when the parent is completely offline the algorithm can run through
>>> many hundreds of connection attempts in milliseconds (confirmed from
>>> logs).
>>>
>>> The use case here is that, for us, it is preferable to make the client
>>> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>>>
>>> I’ve started looking at the code, but I’d like to confirm that this is
>>> not already supported somehow.
>>>
>>> Many thanks
>>>
>>> Nick
>>>
>>> *Nick Dunkin*
>>>
>>> Director, Software Architecture
>>>
>>> Manager – Architecture and New Product Introduction
>>>
>>> *o: **+1 678.258.4071*
>>>
>>> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>>>
>>> cidimage001.png@01D6CC8C.6FC5A580
>>>
Re: Configuring sleep times between parent connection retires?
Posted by Nick Dunkin <Ni...@vecima.com.INVALID>.
Hi John,
Ok, that’s what I thought.
It’s the 502s that I’m trying to avoid in this instance.
As far as I can tell, the retry mechanisms on a parent do not support any kind of “back off” mechanism, so you can have 100+ retries, but with an offline parent those “connection attempts” are exhausted in milliseconds.
I would like to be able to allow a parent to be temporarily “unavailable” for anything up to ~2 seconds, and make the downstream client wait for the parent resource to be available again. It’s a rare edge case, but it’s a real one.
The context here Is ABR video streaming, where a client has enough buffer to very occasionally wait a couple of extra seconds for its content, but an HTTP 502 would cause it to abort its stream.
Thanks for your assistance,
Nick
From: John Rushford <jj...@gmail.com>
Date: Monday, February 20, 2023 at 6:03 PM
To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>
Cc: Nick Dunkin <Ni...@vecima.com.invalid>
Subject: Re: Configuring sleep times between parent connection retires?
No a parent is not retried within the same transaction. If the configured number of retry counts is not exceeded, another parent would be chosen if there are other available parents. If there are no other available parents and the configs allow it, the transaction may go direct to the origin, go_direct in parent.config. If there are no other available parents and go_direct == false, the transaction fails and the client sees a 502
Sent from my iPhone
> On Feb 20, 2023, at 3:18 PM, Nick Dunkin <Ni...@vecima.com.invalid> wrote:
>
> Hi John,
>
> Thank you for the reply.
>
> Are you suggesting that I could have a parent marked down, with a very short retry_time (~1 second), and it would be retried within the same transaction?
>
> I would have thought that at the point it’s marked down, the transaction would end?
>
> Thanks
>
> Nick
>
> From: John J. Rushford <jj...@gmail.com>
> Date: Monday, February 20, 2023 at 3:21 PM
> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>, Nick Dunkin <Ni...@vecima.com.INVALID>
> Subject: Re: Configuring sleep times between parent connection retires?
> Nick,
>
> See the Parent Proxy Configurations,
> https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
>
> You may need to do some tweaking. There is a sliding window,
> proxy.config.http.parent_proxy.retry_time. When the failure threshold
> ,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
> the parent is marked down and will not be used until the retry window
> has elapsed. when the window has elapsed, a failed parent will be
> retried but, if the retry fails, no further attempts will be made to
> that parent until the retry window has elapsed again. If the retry
> succeeds, the parent is marked up.
>
> thanks
>
> John
>
>> On 2/20/23 12:02, Nick Dunkin wrote:
>>
>> Hi,
>>
>> If a parent cache is offline for a brief period of time (say < 3
>> seconds) I would like to know if there is way to configure a “sleep”
>> between connection retries on a particular parent?
>>
>> The ability to configure the “connection attempts” is not useful here,
>> as when the parent is completely offline the algorithm can run through
>> many hundreds of connection attempts in milliseconds (confirmed from
>> logs).
>>
>> The use case here is that, for us, it is preferable to make the client
>> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>>
>> I’ve started looking at the code, but I’d like to confirm that this is
>> not already supported somehow.
>>
>> Many thanks
>>
>> Nick
>>
>> *Nick Dunkin*
>>
>> Director, Software Architecture
>>
>> Manager – Architecture and New Product Introduction
>>
>> *o: **+1 678.258.4071*
>>
>> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>>
>> cidimage001.png@01D6CC8C.6FC5A580
>>
Re: Configuring sleep times between parent connection retires?
Posted by John Rushford <jj...@gmail.com>.
No a parent is not retried within the same transaction. If the configured number of retry counts is not exceeded, another parent would be chosen if there are other available parents. If there are no other available parents and the configs allow it, the transaction may go direct to the origin, go_direct in parent.config. If there are no other available parents and go_direct == false, the transaction fails and the client sees a 502
Sent from my iPhone
> On Feb 20, 2023, at 3:18 PM, Nick Dunkin <Ni...@vecima.com.invalid> wrote:
>
> Hi John,
>
> Thank you for the reply.
>
> Are you suggesting that I could have a parent marked down, with a very short retry_time (~1 second), and it would be retried within the same transaction?
>
> I would have thought that at the point it’s marked down, the transaction would end?
>
> Thanks
>
> Nick
>
> From: John J. Rushford <jj...@gmail.com>
> Date: Monday, February 20, 2023 at 3:21 PM
> To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>, Nick Dunkin <Ni...@vecima.com.INVALID>
> Subject: Re: Configuring sleep times between parent connection retires?
> Nick,
>
> See the Parent Proxy Configurations,
> https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
>
> You may need to do some tweaking. There is a sliding window,
> proxy.config.http.parent_proxy.retry_time. When the failure threshold
> ,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
> the parent is marked down and will not be used until the retry window
> has elapsed. when the window has elapsed, a failed parent will be
> retried but, if the retry fails, no further attempts will be made to
> that parent until the retry window has elapsed again. If the retry
> succeeds, the parent is marked up.
>
> thanks
>
> John
>
>> On 2/20/23 12:02, Nick Dunkin wrote:
>>
>> Hi,
>>
>> If a parent cache is offline for a brief period of time (say < 3
>> seconds) I would like to know if there is way to configure a “sleep”
>> between connection retries on a particular parent?
>>
>> The ability to configure the “connection attempts” is not useful here,
>> as when the parent is completely offline the algorithm can run through
>> many hundreds of connection attempts in milliseconds (confirmed from
>> logs).
>>
>> The use case here is that, for us, it is preferable to make the client
>> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>>
>> I’ve started looking at the code, but I’d like to confirm that this is
>> not already supported somehow.
>>
>> Many thanks
>>
>> Nick
>>
>> *Nick Dunkin*
>>
>> Director, Software Architecture
>>
>> Manager – Architecture and New Product Introduction
>>
>> *o: **+1 678.258.4071*
>>
>> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>>
>> cidimage001.png@01D6CC8C.6FC5A580
>>
Re: Configuring sleep times between parent connection retires?
Posted by Nick Dunkin <Ni...@vecima.com.INVALID>.
Hi John,
Thank you for the reply.
Are you suggesting that I could have a parent marked down, with a very short retry_time (~1 second), and it would be retried within the same transaction?
I would have thought that at the point it’s marked down, the transaction would end?
Thanks
Nick
From: John J. Rushford <jj...@gmail.com>
Date: Monday, February 20, 2023 at 3:21 PM
To: dev@trafficserver.apache.org <de...@trafficserver.apache.org>, Nick Dunkin <Ni...@vecima.com.INVALID>
Subject: Re: Configuring sleep times between parent connection retires?
Nick,
See the Parent Proxy Configurations,
https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
You may need to do some tweaking. There is a sliding window,
proxy.config.http.parent_proxy.retry_time. When the failure threshold
,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
the parent is marked down and will not be used until the retry window
has elapsed. when the window has elapsed, a failed parent will be
retried but, if the retry fails, no further attempts will be made to
that parent until the retry window has elapsed again. If the retry
succeeds, the parent is marked up.
thanks
John
On 2/20/23 12:02, Nick Dunkin wrote:
>
> Hi,
>
> If a parent cache is offline for a brief period of time (say < 3
> seconds) I would like to know if there is way to configure a “sleep”
> between connection retries on a particular parent?
>
> The ability to configure the “connection attempts” is not useful here,
> as when the parent is completely offline the algorithm can run through
> many hundreds of connection attempts in milliseconds (confirmed from
> logs).
>
> The use case here is that, for us, it is preferable to make the client
> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>
> I’ve started looking at the code, but I’d like to confirm that this is
> not already supported somehow.
>
> Many thanks
>
> Nick
>
> *Nick Dunkin*
>
> Director, Software Architecture
>
> Manager – Architecture and New Product Introduction
>
> *o: **+1 678.258.4071*
>
> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>
> cidimage001.png@01D6CC8C.6FC5A580
>
Re: Configuring sleep times between parent connection retires?
Posted by "John J. Rushford" <jj...@gmail.com>.
Nick,
See the Parent Proxy Configurations,
https://docs.trafficserver.apache.org/admin-guide/files/records.config.en.html#parent-proxy-configuration
You may need to do some tweaking. There is a sliding window,
proxy.config.http.parent_proxy.retry_time. When the failure threshold
,proxy.config.http.parent_proxy.fail_threshold¶, on a parent is reached
the parent is marked down and will not be used until the retry window
has elapsed. when the window has elapsed, a failed parent will be
retried but, if the retry fails, no further attempts will be made to
that parent until the retry window has elapsed again. If the retry
succeeds, the parent is marked up.
thanks
John
On 2/20/23 12:02, Nick Dunkin wrote:
>
> Hi,
>
> If a parent cache is offline for a brief period of time (say < 3
> seconds) I would like to know if there is way to configure a “sleep”
> between connection retries on a particular parent?
>
> The ability to configure the “connection attempts” is not useful here,
> as when the parent is completely offline the algorithm can run through
> many hundreds of connection attempts in milliseconds (confirmed from
> logs).
>
> The use case here is that, for us, it is preferable to make the client
> occasionally wait 1-3 seconds, than to return them a HTTP 502.
>
> I’ve started looking at the code, but I’d like to confirm that this is
> not already supported somehow.
>
> Many thanks
>
> Nick
>
> *Nick Dunkin*
>
> Director, Software Architecture
>
> Manager – Architecture and New Product Introduction
>
> *o: **+1 678.258.4071*
>
> *e:*nick.dunkin@vecima.com <ma...@vecima.com>
>
> cidimage001.png@01D6CC8C.6FC5A580
>