Understanding request chains into multi-tier ATS

[Nick Dunkin via users <us...@trafficserver.apache.org>]

Hi,

We’ve been doing some experiments to understand how “slow clients” (ie. user agents on slow connections) can affect the response times in ATS hierarchies.

[cid:image002.png@01DB6E85.801D9650]

Keeping it simple for starters. Everything here is about a CACHE MISS. All networks are assumed perfect, except for the network towards the client. All caches are ATS 9.2.x.

We find that a client on a slow connection, requesting a file (a few MB), will take TIME E to receive that file. That time is dependent on the bandwidth of their “slow connection”.   That time is correctly reflected in the client response time in the ATS log of the Edge Cache.

We see that TIME E becomes a driver of:

  *   TIME M, the time it took to receive that content from the Mid-Tier Cache, and also
  *   TIME O, the time, reported at the Mid-Tier, it took to receive the object from the Origin.

When this happens, it can be misinterpreted as a slow Origin connection, but the timings are actually being dictated by the slow connection to the client.

Could someone point me to a document that explains the network stack of ATS, and could help me understand how these “cache miss” chains are handled internally?

I’d also like to understand how configuration of any internal buffers could reduce the time it takes to bring objects into the Mid-Tier cache. I appreciate that may come with other headaches around memory, and maybe this is all operating exactly as it should? Any insight would be appreciated.

Many thanks,

Nick

Nick Dunkin
Director, Software Architecture
Manager – Architecture and New Product Introduction
o:  +1 678.258.4071<tel:+16782584071>
e: nick.dunkin@vecima.com<ma...@vecima.com>

[cidimage001.png@01D6CC8C.6FC5A580]

Re: Understanding request chains into multi-tier ATS

[Evan Zelkowitz <ez...@apache.org>]

I dont believe there are any internal buffer adjustments you can make to this, it's just a problem inherent to how ATS works where a slow client ends up pacing the entire request as ATS is writing it back to the client on the fly and so depends on how fast they are ingesting it (this can also affect any other clients that land on a read-while-writer for the same content if the writer requester is slow).

One possible solution that has been created is the cache-fill plugin:
https://docs.trafficserver.apache.org/en/9.2.x/admin-guide/plugins/cache_fill.en.html

This will kick off a background request, that is handled by ATS itself and so not dependent on a client, to put the object in cache. The downside to this is stated in the docs, you end up with an extra connection to the upstream for each MISS as both the original request from the client continues as well as the new ATS owned request

You can also look at adding extra logging field values:
https://docs.trafficserver.apache.org/en/9.2.x/admin-guide/logging/formatting.en.html#admin-logging-fields-time
adding ones such as `stms` which is time spent just with the upstream transaction

On 2025/01/24 22:37:03 Nick Dunkin via users wrote:
> Hi,
> 
> We’ve been doing some experiments to understand how “slow clients” (ie. user agents on slow connections) can affect the response times in ATS hierarchies.
> 
> [cid:image002.png@01DB6E85.801D9650]
> 
> Keeping it simple for starters. Everything here is about a CACHE MISS. All networks are assumed perfect, except for the network towards the client. All caches are ATS 9.2.x.
> 
> We find that a client on a slow connection, requesting a file (a few MB), will take TIME E to receive that file. That time is dependent on the bandwidth of their “slow connection”.   That time is correctly reflected in the client response time in the ATS log of the Edge Cache.
> 
> We see that TIME E becomes a driver of:
> 
>   *   TIME M, the time it took to receive that content from the Mid-Tier Cache, and also
>   *   TIME O, the time, reported at the Mid-Tier, it took to receive the object from the Origin.
> 
> When this happens, it can be misinterpreted as a slow Origin connection, but the timings are actually being dictated by the slow connection to the client.
> 
> Could someone point me to a document that explains the network stack of ATS, and could help me understand how these “cache miss” chains are handled internally?
> 
> I’d also like to understand how configuration of any internal buffers could reduce the time it takes to bring objects into the Mid-Tier cache. I appreciate that may come with other headaches around memory, and maybe this is all operating exactly as it should? Any insight would be appreciated.
> 
> Many thanks,
> 
> Nick
> 
> Nick Dunkin
> Director, Software Architecture
> Manager – Architecture and New Product Introduction
> o:  +1 678.258.4071<tel:+16782584071>
> e: nick.dunkin@vecima.com<ma...@vecima.com>
> 
> [cidimage001.png@01D6CC8C.6FC5A580]
>