You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Luís Antonio Pereira <lu...@gmail.com> on 2014/07/07 20:59:42 UTC
Best practices for hit ratio improve
Hi all
I would like to improve the performance of my webcache, for example
for example implementing the pining/cache.config:
On records.config:
CONFIG proxy.config.cache.permit.pinning INT 1
On cache.config:
# agressive cache rule
url_regex=\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http pin-in-cache=360d
url_regex=\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http ttl-in-cache=360d
url_regex=\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http revalidate=5m
url_regex=\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http action=ignore-no-cache
# facebook rule
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http pin-in-cache=360d
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http ttl-in-cache=360d
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http revalidate=8h
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http action=ignore-no-cache
any sugestions??? another configuration item???
Thank's
Luís Antonio Pereira
Re: Best practices for hit ratio improve
Posted by "Adam W. Dace" <co...@gmail.com>.
Huh. I'll try not to do that long-term, this really is just an experiment.
Maybe this once more has to do with me using Firefox not Chrome in the past.
I've just never been able to get past 64MB of "Ram Used" in
traffic_top...hoping to give Traffic Server a nice gigabyte or two of RAM
in the future.
On Fri, Jul 11, 2014 at 8:27 PM, Leif Hedstrom <zw...@apache.org> wrote:
>
>
> On Jul 11, 2014, at 2:18 PM, "Adam W. Dace" <co...@gmail.com>
> wrote:
>
> I've actually been experimenting with this a bit myself.
>
> Over time I've been frustrated by the fact that Traffic Server doesn't
> seem to want to use its RAM cache much, on a system with very little load.
>
> Here's my current cache.config:
> url_regex=\.(js|gif|jpg|jpeg|png|svg|css|ico|bmp|swf|flv|html|pdf)$
> scheme=http ttl-in-cache=60d
> url_regex=.* scheme=http revalidate=1d
>
> Does this really help? I'm not really the one to ask, though it does seem
> to make things a bit faster...and my RAM usage is up.
>
>
>
> I'm sure this helps, but a forward proxy you are clearly violating the
> protocol :).
>
> As for RAM cache, it's simply a mapping over disk blocks. I don't
> understand why it would not use all RAM you give it.
>
>
>
> I've also been documenting how I tune Traffic Server on a personal basis,
> feel free to check it out at WebProxyCacheTuning
> <https://cwiki.apache.org/confluence/display/TS/WebProxyCacheTuning>.
> At this point I'm pretty sure I'm done tweaking Traffic Server v5.0.
>
> Regards,
>
> Adam
>
>
> On Mon, Jul 7, 2014 at 2:25 PM, Leif Hedstrom <zw...@apache.org> wrote:
>
>>
>> On Jul 7, 2014, at 12:59 PM, Luís Antonio Pereira <
>> luis.a.pereira@gmail.com> wrote:
>>
>> > Hi all
>> >
>> > I would like to improve the performance of my webcache, for example
>> > for example implementing the pining/cache.config:
>> >
>> > On records.config:
>> > CONFIG proxy.config.cache.permit.pinning INT 1
>>
>>
>> Before you go there, I’d seriously investigate your cache and content.
>> Some thoughts
>>
>> 1. Is your cache not large enough? How large is your active data content?
>> Disks are fairly cheap, pinning is not (looking at your rules, I think
>> you’ll kill your performance completely).
>>
>> 2. Do you have adequate RAM ?
>>
>> 3. What evidence is there to make you think pinning actually makes things
>> better? Maybe what you are really looking for is to allocate some amount of
>> the disks for Facebook content only ? This can be done with volume.config /
>> hosting.config.
>>
>> 4. Are you sure you are not mixing up the notion of Cache-Control:
>> max-age with pinning?
>>
>>
>>
>> In almost all cases I’ve seen ATS use, the cache does a great job on its
>> own. Messing with things like cache pinning ought to be a last resort IMO,
>> for some very specific use cases. I don’t think this is one of those.
>>
>> Cheers,
>>
>> — Leif
>>
>>
>>
>
>
> --
> ____________________________________________________________
> Adam W. Dace <co...@gmail.com>
>
> Phone: (815) 355-5848
> Instant Messenger: AIM & Yahoo! IM - colonelforbin74 | ICQ - #39374451
> Microsoft Messenger - colonelforbin74@live.com <ad...@turing.com>
>
> Google Profile: https://plus.google.com/u/0/109309036874332290399/about
>
>
--
____________________________________________________________
Adam W. Dace <co...@gmail.com>
Phone: (815) 355-5848
Instant Messenger: AIM & Yahoo! IM - colonelforbin74 | ICQ - #39374451
Microsoft Messenger - colonelforbin74@live.com <ad...@turing.com>
Google Profile: https://plus.google.com/u/0/109309036874332290399/about
Re: Best practices for hit ratio improve
Posted by Luís Antonio Pereira <lu...@gmail.com>.
I try this configurations and my bandwidth savings increase to 29%
url_regex=(.*)\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http ttl-in-cache=360d
url_regex=(.*)\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http revalidate=5m
url_regex=(.*)\.(js|gif|jpg|jpeg|png|css|ico|bmp|mp3|mp4|avi|mov|swf|flv|pdf)$
scheme=http action=ignore-no-cache
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http ttl-in-cache=360d
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http revalidate=8h
url_regex=^http://([a-z][0-9]\.){0,1}(profile|sphotos|photos-[a-z])\.ak\.fbcdn\.net/hp(rofile|hotos)-ak-
scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.(youtube|googlevideo).com/(get_video|videoplayback|videodownload)\?
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.(youtube|googlevideo).com/(get_video|videoplayback|videodownload)\?
scheme=http revalidate=8h
url_regex=^http://[^/]*\.(youtube|googlevideo).com/(get_video|videoplayback|videodownload)\?
scheme=http action=ignore-no-cache
url_regex=^http://[a-z]{1,}\.dl\.sourceforge\.net\/project/
scheme=http ttl-in-cache=360d
url_regex=^http://[a-z]{1,}\.dl\.sourceforge\.net\/project/
scheme=http revalidate=8h
url_regex=^http://[a-z]{1,}\.dl\.sourceforge\.net\/project/
scheme=http action=ignore-no-cache
url_regex=^http://(.*)\.windowsupdate.com/.*\.(cab|exe|dll|ms)
scheme=http ttl-in-cache=360d
url_regex=^http://(.*)\.windowsupdate.com/.*\.(cab|exe|dll|ms)
scheme=http revalidate=8h
url_regex=^http://(.*)\.windowsupdate.com/.*\.(cab|exe|dll|ms)
scheme=http action=ignore-no-cache
url_regex=^http://(.*)\.microsoft.com/.*\.(cab|exe|dll|ms) scheme=http
ttl-in-cache=360d
url_regex=^http://(.*)\.microsoft.com/.*\.(cab|exe|dll|ms) scheme=http
revalidate=8h
url_regex=^http://(.*)\.microsoft.com/.*\.(cab|exe|dll|ms) scheme=http
action=ignore-no-cache
url_regex=^http://[^/]*\.apersonal.avira-update.com/.*\.(cab|exe|dll|msi|gz)
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.apersonal.avira-update.com/.*\.(cab|exe|dll|msi|gz)
scheme=http revalidate=8h
url_regex=^http://[^/]*\.apersonal.avira-update.com/.*\.(cab|exe|dll|msi|gz)
scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http revalidate=8h
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http revalidate=8h
url_regex=^http://[^/]*\.aliveupdate.symantecliveupdate.com/.*\.(cab|exe|dll|msi)
scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.avast.com/.*\.(vpu|cab|stamp|exe) scheme=http
ttl-in-cache=360d
url_regex=^http://[^/]*\.avast.com/.*\.(vpu|cab|stamp|exe) scheme=http
revalidate=8h
url_regex=^http://[^/]*\.avast.com/.*\.(vpu|cab|stamp|exe) scheme=http
action=ignore-no-cache
url_regex=^http://[^/]*\.avg.com/.*\.(bin|gz) scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.avg.com/.*\.(bin|gz) scheme=http revalidate=8h
url_regex=^http://[^/]*\.avg.com/.*\.(bin|gz) scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.avgate.net/.*\.(bin|gz) scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.avgate.net/.*\.(bin|gz) scheme=http revalidate=8h
url_regex=^http://[^/]*\.avgate.net/.*\.(bin|gz) scheme=http
action=ignore-no-cache
url_regex=^http://[^/]*\.kaspersky\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.kaspersky\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http revalidate=8h
url_regex=^http://[^/]*\.kaspersky\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http action=ignore-no-cache
url_regex=^http://[^/]*\.kaspersky-labs\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http ttl-in-cache=360d
url_regex=^http://[^/]*\.kaspersky-labs\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http revalidate=8h
url_regex=^http://[^/]*\.kaspersky-labs\.com/.*\.(xml|stt|dll|dat|avc|dif|diff|exe|cab|fad|zip|kdc|klz)
scheme=http action=ignore-no-cache
2014-07-11 21:27 GMT-04:00 Leif Hedstrom <zw...@apache.org>:
>
>
> On Jul 11, 2014, at 2:18 PM, "Adam W. Dace" <co...@gmail.com>
> wrote:
>
> I've actually been experimenting with this a bit myself.
>
> Over time I've been frustrated by the fact that Traffic Server doesn't seem
> to want to use its RAM cache much, on a system with very little load.
>
> Here's my current cache.config:
> url_regex=\.(js|gif|jpg|jpeg|png|svg|css|ico|bmp|swf|flv|html|pdf)$
> scheme=http ttl-in-cache=60d
> url_regex=.* scheme=http revalidate=1d
>
> Does this really help? I'm not really the one to ask, though it does seem
> to make things a bit faster...and my RAM usage is up.
>
>
>
> I'm sure this helps, but a forward proxy you are clearly violating the
> protocol :).
>
> As for RAM cache, it's simply a mapping over disk blocks. I don't understand
> why it would not use all RAM you give it.
>
>
>
> I've also been documenting how I tune Traffic Server on a personal basis,
> feel free to check it out at WebProxyCacheTuning.
> At this point I'm pretty sure I'm done tweaking Traffic Server v5.0.
>
> Regards,
>
> Adam
>
>
> On Mon, Jul 7, 2014 at 2:25 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>
>>
>> On Jul 7, 2014, at 12:59 PM, Luís Antonio Pereira
>> <lu...@gmail.com> wrote:
>>
>> > Hi all
>> >
>> > I would like to improve the performance of my webcache, for example
>> > for example implementing the pining/cache.config:
>> >
>> > On records.config:
>> > CONFIG proxy.config.cache.permit.pinning INT 1
>>
>>
>> Before you go there, I’d seriously investigate your cache and content.
>> Some thoughts
>>
>> 1. Is your cache not large enough? How large is your active data content?
>> Disks are fairly cheap, pinning is not (looking at your rules, I think
>> you’ll kill your performance completely).
>>
>> 2. Do you have adequate RAM ?
>>
>> 3. What evidence is there to make you think pinning actually makes things
>> better? Maybe what you are really looking for is to allocate some amount of
>> the disks for Facebook content only ? This can be done with volume.config /
>> hosting.config.
>>
>> 4. Are you sure you are not mixing up the notion of Cache-Control: max-age
>> with pinning?
>>
>>
>>
>> In almost all cases I’ve seen ATS use, the cache does a great job on its
>> own. Messing with things like cache pinning ought to be a last resort IMO,
>> for some very specific use cases. I don’t think this is one of those.
>>
>> Cheers,
>>
>> — Leif
>>
>>
>
>
>
> --
> ____________________________________________________________
> Adam W. Dace <co...@gmail.com>
>
> Phone: (815) 355-5848
> Instant Messenger: AIM & Yahoo! IM - colonelforbin74 | ICQ - #39374451
> Microsoft Messenger - colonelforbin74@live.com
>
> Google Profile: https://plus.google.com/u/0/109309036874332290399/about
Re: Best practices for hit ratio improve
Posted by Leif Hedstrom <zw...@apache.org>.
> On Jul 11, 2014, at 2:18 PM, "Adam W. Dace" <co...@gmail.com> wrote:
>
> I've actually been experimenting with this a bit myself.
>
> Over time I've been frustrated by the fact that Traffic Server doesn't seem to want to use its RAM cache much, on a system with very little load.
>
> Here's my current cache.config:
> url_regex=\.(js|gif|jpg|jpeg|png|svg|css|ico|bmp|swf|flv|html|pdf)$ scheme=http ttl-in-cache=60d
> url_regex=.* scheme=http revalidate=1d
>
> Does this really help? I'm not really the one to ask, though it does seem to make things a bit faster...and my RAM usage is up.
I'm sure this helps, but a forward proxy you are clearly violating the protocol :).
As for RAM cache, it's simply a mapping over disk blocks. I don't understand why it would not use all RAM you give it.
>
> I've also been documenting how I tune Traffic Server on a personal basis, feel free to check it out at WebProxyCacheTuning.
> At this point I'm pretty sure I'm done tweaking Traffic Server v5.0.
>
> Regards,
>
> Adam
>
>
>> On Mon, Jul 7, 2014 at 2:25 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>
>> On Jul 7, 2014, at 12:59 PM, Luís Antonio Pereira <lu...@gmail.com> wrote:
>>
>> > Hi all
>> >
>> > I would like to improve the performance of my webcache, for example
>> > for example implementing the pining/cache.config:
>> >
>> > On records.config:
>> > CONFIG proxy.config.cache.permit.pinning INT 1
>>
>>
>> Before you go there, I’d seriously investigate your cache and content. Some thoughts
>>
>> 1. Is your cache not large enough? How large is your active data content? Disks are fairly cheap, pinning is not (looking at your rules, I think you’ll kill your performance completely).
>>
>> 2. Do you have adequate RAM ?
>>
>> 3. What evidence is there to make you think pinning actually makes things better? Maybe what you are really looking for is to allocate some amount of the disks for Facebook content only ? This can be done with volume.config / hosting.config.
>>
>> 4. Are you sure you are not mixing up the notion of Cache-Control: max-age with pinning?
>>
>>
>>
>> In almost all cases I’ve seen ATS use, the cache does a great job on its own. Messing with things like cache pinning ought to be a last resort IMO, for some very specific use cases. I don’t think this is one of those.
>>
>> Cheers,
>>
>> — Leif
>
>
>
> --
> ____________________________________________________________
> Adam W. Dace <co...@gmail.com>
>
> Phone: (815) 355-5848
> Instant Messenger: AIM & Yahoo! IM - colonelforbin74 | ICQ - #39374451
> Microsoft Messenger - colonelforbin74@live.com
>
> Google Profile: https://plus.google.com/u/0/109309036874332290399/about
Re: Best practices for hit ratio improve
Posted by "Adam W. Dace" <co...@gmail.com>.
I've actually been experimenting with this a bit myself.
Over time I've been frustrated by the fact that Traffic Server doesn't seem
to want to use its RAM cache much, on a system with very little load.
Here's my current cache.config:
url_regex=\.(js|gif|jpg|jpeg|png|svg|css|ico|bmp|swf|flv|html|pdf)$
scheme=http ttl-in-cache=60d
url_regex=.* scheme=http revalidate=1d
Does this really help? I'm not really the one to ask, though it does seem
to make things a bit faster...and my RAM usage is up.
I've also been documenting how I tune Traffic Server on a personal basis,
feel free to check it out at WebProxyCacheTuning
<https://cwiki.apache.org/confluence/display/TS/WebProxyCacheTuning>.
At this point I'm pretty sure I'm done tweaking Traffic Server v5.0.
Regards,
Adam
On Mon, Jul 7, 2014 at 2:25 PM, Leif Hedstrom <zw...@apache.org> wrote:
>
> On Jul 7, 2014, at 12:59 PM, Luís Antonio Pereira <
> luis.a.pereira@gmail.com> wrote:
>
> > Hi all
> >
> > I would like to improve the performance of my webcache, for example
> > for example implementing the pining/cache.config:
> >
> > On records.config:
> > CONFIG proxy.config.cache.permit.pinning INT 1
>
>
> Before you go there, I’d seriously investigate your cache and content.
> Some thoughts
>
> 1. Is your cache not large enough? How large is your active data content?
> Disks are fairly cheap, pinning is not (looking at your rules, I think
> you’ll kill your performance completely).
>
> 2. Do you have adequate RAM ?
>
> 3. What evidence is there to make you think pinning actually makes things
> better? Maybe what you are really looking for is to allocate some amount of
> the disks for Facebook content only ? This can be done with volume.config /
> hosting.config.
>
> 4. Are you sure you are not mixing up the notion of Cache-Control: max-age
> with pinning?
>
>
>
> In almost all cases I’ve seen ATS use, the cache does a great job on its
> own. Messing with things like cache pinning ought to be a last resort IMO,
> for some very specific use cases. I don’t think this is one of those.
>
> Cheers,
>
> — Leif
>
>
>
--
____________________________________________________________
Adam W. Dace <co...@gmail.com>
Phone: (815) 355-5848
Instant Messenger: AIM & Yahoo! IM - colonelforbin74 | ICQ - #39374451
Microsoft Messenger - colonelforbin74@live.com <ad...@turing.com>
Google Profile: https://plus.google.com/u/0/109309036874332290399/about
Re: Best practices for hit ratio improve
Posted by Leif Hedstrom <zw...@apache.org>.
On Jul 7, 2014, at 12:59 PM, Luís Antonio Pereira <lu...@gmail.com> wrote:
> Hi all
>
> I would like to improve the performance of my webcache, for example
> for example implementing the pining/cache.config:
>
> On records.config:
> CONFIG proxy.config.cache.permit.pinning INT 1
Before you go there, I’d seriously investigate your cache and content. Some thoughts
1. Is your cache not large enough? How large is your active data content? Disks are fairly cheap, pinning is not (looking at your rules, I think you’ll kill your performance completely).
2. Do you have adequate RAM ?
3. What evidence is there to make you think pinning actually makes things better? Maybe what you are really looking for is to allocate some amount of the disks for Facebook content only ? This can be done with volume.config / hosting.config.
4. Are you sure you are not mixing up the notion of Cache-Control: max-age with pinning?
In almost all cases I’ve seen ATS use, the cache does a great job on its own. Messing with things like cache pinning ought to be a last resort IMO, for some very specific use cases. I don’t think this is one of those.
Cheers,
— Leif