You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by "vishwas k.n." <vi...@gmail.com> on 2019/07/28 06:40:10 UTC
Few Questions
Hello,
I am new to ATS and had a few questions after having installed it and
played around with it a bit.
I am primarily interested in the reverse-proxy config mode of operation of
ATS.
1. When an origin server responds with a 304 to a request from ATS, then
ATS goes on to write the entire content and not just the headers into a new
location on the disk. This seems to be a "general space management
technique". In case of large files, wouldn't it make sense for the
directory to point to the earlier location on the disk and just update the
response headers ?. That way a whole chunk of write could be avoided.
2. Is there a configuration to enable the AIO sync/write to the disk happen
immediately after the response is received from the origin server ?
3. There are a few threads on the list discussing performance, however I
havent been able to zero in on any which discuss performance in detail.
We are trying to derive ATS performance benchmark using wrk as a client.
The initial tests suggest the below results:
*Iteration * *Connections/Threads (wrk)* *File Size* *Duration Of Test*
*ATS*
1 1/1 2MB 5min 8.83 Gbps
2 1/1 3MB 5min 8.27 Gbps
3 1/1 4MB 5min 3.81 Gbps
4 1/1 10MB 5min 3.80 Gbps
5 1/1 1MB 5min 8.58 Gbps
The ATS configuration is used more or less the defaults available once the
ATS is installed.
The ATS version being used is 7.1.4
Is there a cheatsheet which indicates the tunable parameters to get better
performance with ATS ?
NOTE: I am using file cache.
thanks,
-vishwas.
Re: Few Questions
Posted by "vishwas k.n." <vi...@gmail.com>.
thank you Masaori. I did tweak the value of ram_cache_cutoff but didnt see
much of an improvement. I will also try Alan's suggestions and post back .
regards,
-vishwas.
On Tue, Jul 30, 2019 at 10:32 AM Masaori Koshiba <ma...@apache.org> wrote:
> As for 3), the drop of throughput might come from
> proxy.config.cache.ram_cache_cutoff. The default size is 4MB.
>
>
> https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/files/records.config.en.html#proxy-config-cache-ram-cache-cutoff
>
> - Masaori
>
> On Tue, Jul 30, 2019 at 3:48 AM Alan Carroll <
> solidwallofcode@verizonmedia.com> wrote:
>
>> 1) It's a bit more subtle than that. It depends on the size of the
>> object. ATS stores objects in "fragments" and if the object fits in a
>> single fragment, it's all written together. Objects larger than a fragment
>> keep just the headers in the first fragment and that is what get rewritten,
>> not the entire object.
>>
>> 2) No. The performance hit is very high from that and it's actually quite
>> difficult at the cache writing point to do this for a single object, as
>> multiple smaller writes get aggregated.
>>
>> 3) I'm not one of the performance guys, but it looks like the change in
>> throughput may be due to TCP kernel socket buffer size or possibly ICW
>> size. If so, you'd see higher aggregate bandwidth with parallel requests,
>> which is expected usage. We have ATS doing 40+G/s in productionn for large
>> files, where the limit is the network bandwidth, not ATS.
>>
>> On Sun, Jul 28, 2019 at 1:40 AM vishwas k.n. <vi...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I am new to ATS and had a few questions after having installed it and
>>> played around with it a bit.
>>> I am primarily interested in the reverse-proxy config mode of operation
>>> of ATS.
>>>
>>> 1. When an origin server responds with a 304 to a request from ATS, then
>>> ATS goes on to write the entire content and not just the headers into a new
>>> location on the disk. This seems to be a "general space management
>>> technique". In case of large files, wouldn't it make sense for the
>>> directory to point to the earlier location on the disk and just update the
>>> response headers ?. That way a whole chunk of write could be avoided.
>>>
>>> 2. Is there a configuration to enable the AIO sync/write to the disk
>>> happen immediately after the response is received from the origin server ?
>>>
>>> 3. There are a few threads on the list discussing performance, however I
>>> havent been able to zero in on any which discuss performance in detail.
>>> We are trying to derive ATS performance benchmark using wrk as a client.
>>> The initial tests suggest the below results:
>>>
>>> *Iteration * *Connections/Threads (wrk)* *File Size* *Duration Of Test*
>>> *ATS*
>>> 1 1/1 2MB 5min 8.83 Gbps
>>> 2 1/1 3MB 5min 8.27 Gbps
>>> 3 1/1 4MB 5min 3.81 Gbps
>>> 4 1/1 10MB 5min 3.80 Gbps
>>> 5 1/1 1MB 5min 8.58 Gbps
>>>
>>> The ATS configuration is used more or less the defaults available once
>>> the ATS is installed.
>>> The ATS version being used is 7.1.4
>>> Is there a cheatsheet which indicates the tunable parameters to get
>>> better performance with ATS ?
>>> NOTE: I am using file cache.
>>>
>>> thanks,
>>> -vishwas.
>>>
>>
Re: Few Questions
Posted by Masaori Koshiba <ma...@apache.org>.
As for 3), the drop of throughput might come from
proxy.config.cache.ram_cache_cutoff. The default size is 4MB.
https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/files/records.config.en.html#proxy-config-cache-ram-cache-cutoff
- Masaori
On Tue, Jul 30, 2019 at 3:48 AM Alan Carroll <
solidwallofcode@verizonmedia.com> wrote:
> 1) It's a bit more subtle than that. It depends on the size of the object.
> ATS stores objects in "fragments" and if the object fits in a single
> fragment, it's all written together. Objects larger than a fragment keep
> just the headers in the first fragment and that is what get rewritten, not
> the entire object.
>
> 2) No. The performance hit is very high from that and it's actually quite
> difficult at the cache writing point to do this for a single object, as
> multiple smaller writes get aggregated.
>
> 3) I'm not one of the performance guys, but it looks like the change in
> throughput may be due to TCP kernel socket buffer size or possibly ICW
> size. If so, you'd see higher aggregate bandwidth with parallel requests,
> which is expected usage. We have ATS doing 40+G/s in productionn for large
> files, where the limit is the network bandwidth, not ATS.
>
> On Sun, Jul 28, 2019 at 1:40 AM vishwas k.n. <vi...@gmail.com> wrote:
>
>> Hello,
>>
>> I am new to ATS and had a few questions after having installed it and
>> played around with it a bit.
>> I am primarily interested in the reverse-proxy config mode of operation
>> of ATS.
>>
>> 1. When an origin server responds with a 304 to a request from ATS, then
>> ATS goes on to write the entire content and not just the headers into a new
>> location on the disk. This seems to be a "general space management
>> technique". In case of large files, wouldn't it make sense for the
>> directory to point to the earlier location on the disk and just update the
>> response headers ?. That way a whole chunk of write could be avoided.
>>
>> 2. Is there a configuration to enable the AIO sync/write to the disk
>> happen immediately after the response is received from the origin server ?
>>
>> 3. There are a few threads on the list discussing performance, however I
>> havent been able to zero in on any which discuss performance in detail.
>> We are trying to derive ATS performance benchmark using wrk as a client.
>> The initial tests suggest the below results:
>>
>> *Iteration * *Connections/Threads (wrk)* *File Size* *Duration Of Test*
>> *ATS*
>> 1 1/1 2MB 5min 8.83 Gbps
>> 2 1/1 3MB 5min 8.27 Gbps
>> 3 1/1 4MB 5min 3.81 Gbps
>> 4 1/1 10MB 5min 3.80 Gbps
>> 5 1/1 1MB 5min 8.58 Gbps
>>
>> The ATS configuration is used more or less the defaults available once
>> the ATS is installed.
>> The ATS version being used is 7.1.4
>> Is there a cheatsheet which indicates the tunable parameters to get
>> better performance with ATS ?
>> NOTE: I am using file cache.
>>
>> thanks,
>> -vishwas.
>>
>
Re: Few Questions
Posted by "vishwas k.n." <vi...@gmail.com>.
thank you Alan for your quick response. Let me reiterate the tests with the
suggested changes.
regards,
-vishwas.
On Tue, Jul 30, 2019 at 12:18 AM Alan Carroll <
solidwallofcode@verizonmedia.com> wrote:
> 1) It's a bit more subtle than that. It depends on the size of the object.
> ATS stores objects in "fragments" and if the object fits in a single
> fragment, it's all written together. Objects larger than a fragment keep
> just the headers in the first fragment and that is what get rewritten, not
> the entire object.
>
> 2) No. The performance hit is very high from that and it's actually quite
> difficult at the cache writing point to do this for a single object, as
> multiple smaller writes get aggregated.
>
> 3) I'm not one of the performance guys, but it looks like the change in
> throughput may be due to TCP kernel socket buffer size or possibly ICW
> size. If so, you'd see higher aggregate bandwidth with parallel requests,
> which is expected usage. We have ATS doing 40+G/s in productionn for large
> files, where the limit is the network bandwidth, not ATS.
>
> On Sun, Jul 28, 2019 at 1:40 AM vishwas k.n. <vi...@gmail.com> wrote:
>
>> Hello,
>>
>> I am new to ATS and had a few questions after having installed it and
>> played around with it a bit.
>> I am primarily interested in the reverse-proxy config mode of operation
>> of ATS.
>>
>> 1. When an origin server responds with a 304 to a request from ATS, then
>> ATS goes on to write the entire content and not just the headers into a new
>> location on the disk. This seems to be a "general space management
>> technique". In case of large files, wouldn't it make sense for the
>> directory to point to the earlier location on the disk and just update the
>> response headers ?. That way a whole chunk of write could be avoided.
>>
>> 2. Is there a configuration to enable the AIO sync/write to the disk
>> happen immediately after the response is received from the origin server ?
>>
>> 3. There are a few threads on the list discussing performance, however I
>> havent been able to zero in on any which discuss performance in detail.
>> We are trying to derive ATS performance benchmark using wrk as a client.
>> The initial tests suggest the below results:
>>
>> *Iteration * *Connections/Threads (wrk)* *File Size* *Duration Of Test*
>> *ATS*
>> 1 1/1 2MB 5min 8.83 Gbps
>> 2 1/1 3MB 5min 8.27 Gbps
>> 3 1/1 4MB 5min 3.81 Gbps
>> 4 1/1 10MB 5min 3.80 Gbps
>> 5 1/1 1MB 5min 8.58 Gbps
>>
>> The ATS configuration is used more or less the defaults available once
>> the ATS is installed.
>> The ATS version being used is 7.1.4
>> Is there a cheatsheet which indicates the tunable parameters to get
>> better performance with ATS ?
>> NOTE: I am using file cache.
>>
>> thanks,
>> -vishwas.
>>
>
Re: Few Questions
Posted by Alan Carroll <so...@verizonmedia.com>.
1) It's a bit more subtle than that. It depends on the size of the object.
ATS stores objects in "fragments" and if the object fits in a single
fragment, it's all written together. Objects larger than a fragment keep
just the headers in the first fragment and that is what get rewritten, not
the entire object.
2) No. The performance hit is very high from that and it's actually quite
difficult at the cache writing point to do this for a single object, as
multiple smaller writes get aggregated.
3) I'm not one of the performance guys, but it looks like the change in
throughput may be due to TCP kernel socket buffer size or possibly ICW
size. If so, you'd see higher aggregate bandwidth with parallel requests,
which is expected usage. We have ATS doing 40+G/s in productionn for large
files, where the limit is the network bandwidth, not ATS.
On Sun, Jul 28, 2019 at 1:40 AM vishwas k.n. <vi...@gmail.com> wrote:
> Hello,
>
> I am new to ATS and had a few questions after having installed it and
> played around with it a bit.
> I am primarily interested in the reverse-proxy config mode of operation of
> ATS.
>
> 1. When an origin server responds with a 304 to a request from ATS, then
> ATS goes on to write the entire content and not just the headers into a new
> location on the disk. This seems to be a "general space management
> technique". In case of large files, wouldn't it make sense for the
> directory to point to the earlier location on the disk and just update the
> response headers ?. That way a whole chunk of write could be avoided.
>
> 2. Is there a configuration to enable the AIO sync/write to the disk
> happen immediately after the response is received from the origin server ?
>
> 3. There are a few threads on the list discussing performance, however I
> havent been able to zero in on any which discuss performance in detail.
> We are trying to derive ATS performance benchmark using wrk as a client.
> The initial tests suggest the below results:
>
> *Iteration * *Connections/Threads (wrk)* *File Size* *Duration Of Test*
> *ATS*
> 1 1/1 2MB 5min 8.83 Gbps
> 2 1/1 3MB 5min 8.27 Gbps
> 3 1/1 4MB 5min 3.81 Gbps
> 4 1/1 10MB 5min 3.80 Gbps
> 5 1/1 1MB 5min 8.58 Gbps
>
> The ATS configuration is used more or less the defaults available once the
> ATS is installed.
> The ATS version being used is 7.1.4
> Is there a cheatsheet which indicates the tunable parameters to get better
> performance with ATS ?
> NOTE: I am using file cache.
>
> thanks,
> -vishwas.
>