You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by ge...@free.fr on 2014/06/10 11:18:12 UTC
header modification
Hi all,
3 questions :
- could a "regex_suffix" rule be added into future version of ATS, someone can use it to match in one rule all kind of image like "suffix_regex=(png|bmp|jpg|jpeg|gif)" instead of writing one line by suffix ?
- what is the best place to modify header for a group of document type like images, remap.config (regex_map+header_rewrite) or cache.config ?
- is or will it possible to use header_rewrite plugin into cache.config to modify header ?
nice job done with ATS, thank you all.
Regards,
Denis
Re: header modification
Posted by Sudheer Vinukonda <su...@yahoo-inc.com.INVALID>.
On 6/11/14, 12:29 AM, "geodni@free.fr" <ge...@free.fr> wrote:
>>> I want to keep objects like media (avi|jpg|mp3) for 7 days in cache
>>>while telling the client "max-age=3d", keep office file for 3
>>>days in cache while telling the client 1 day (using ATS 4.1.2 TTL are
>>>in seconds)
>>
>>I think you are overcomplicating things a bit though, use the protocol
>>:). http://tools.ietf.org/html/rfc7234#section-5.2.2.9, e.g.
>>
>> Cache-Control: max-age=259200, s-maxage=604800
>>
>>This would be so easy if you could just configure your origin servers to
>>behave and send the above. Barring that, you could use
>>header_rewrite to modify the origin server response to just that. I just
>>cringe when people have to fight poorly configured origin
>>servers, this stuff belongs on the origin, and not on the caches.
>
>Leif,
>thanks for the reminder but I don't really have this choice by now.
>Still trying to use regex_map with header_rewrite plugin with
>cond %{SEND_RESPONSE_HDR_HOOK}
>cond %{PATH} "/(avi|mp3|jpg)$/"
>rm-header Cache-Control
>add-header Cache-Control max-age=3d
>cond %{PATH} "/(doc|xls|ppt)$/"
>rm-header Cache-Control
>add-header Cache-Control max-age=1d
>
>I got this diag which shows only second %{PATH} condition is tested but
>also ending with a strange message "PATH requires remap initialization!
>Evaluating to false!". Even with a file named test.doc, no header rewrite
>action is done :
>
>(header_rewrite) activated transaction hook via remap.config: new hook=5
>(header_rewrite_dbg) Calling CTOR for Resources (RemapAPI)
>(header_rewrite) rri: 0x7fffffffb8e0
>(header_rewrite) Building resource structure for hook (17)
>(header_rewrite) Adding TXN client request header buffers
>(header_rewrite) Adding TXN client request header buffers for remap
>instance
>(header_rewrite) Appending PATH to evaluation value: test.avi
>(header_rewrite) Evaluating PATH
>(header_rewrite) Test regular expression (doc|xls|ppt)$ : test.avi
>(header_rewrite) returing with status: 0
>(header_rewrite_dbg) Calling CTOR for Resources (InkAPI)
>(header_rewrite) Building resource structure for hook (5)
>(header_rewrite) Adding TXN client request header buffers
>(header_rewrite) Adding TXN client response header buffers
>(header_rewrite) PATH requires remap initialization! Evaluating to false!
>
>
>(header_rewrite) activated transaction hook via remap.config: new hook=5
>(header_rewrite_dbg) Calling CTOR for Resources (RemapAPI)
>(header_rewrite) rri: 0x7fffff6f90b0
>(header_rewrite) Building resource structure for hook (17)
>(header_rewrite) Adding TXN client request header buffers
>(header_rewrite) Adding TXN client request header buffers for remap
>instance
>(header_rewrite) Appending PATH to evaluation value: test.doc
>(header_rewrite) Evaluating PATH
>(header_rewrite) Test regular expression (doc|xls|ppt)$ : test.doc
>(header_rewrite) Successfully found regular expression match
>(header_rewrite) OperatorRMHeader::exec() invoked on header Cache-Control
>(header_rewrite) OperatorAddHeader::exec() invoked on header
>Cache-Control: max-age=1d
>(header_rewrite) adding header Cache-Control
>(header_rewrite) returing with status: 0
>(header_rewrite_dbg) Calling CTOR for Resources (InkAPI)
>(header_rewrite) Building resource structure for hook (5)
>(header_rewrite) Adding TXN client request header buffers
>(header_rewrite) Adding TXN client response header buffers
>(header_rewrite) PATH requires remap initialization! Evaluating to false!
>
You may want to upgrade to ats-5.0.x, if that¹s a possibility - I see this
error too on ats-4.0, but, it works fine in 5.0 version.
>
>regards,
>Denis
Re: header modification : partially solved
Posted by ge...@free.fr.
> Besides using the old version of ATS, which is not supported, your
> config acts slightly oddly. I think it might even be a bug (it
> doesn’t run the rules in the order you’d expect, i.e. it matches the
> second regex first).
OK, I successfully ported ATS 4.2.x branch based on GIT 0c8d513 which contains patch TS-2784 (FreeBSD 10 header_rewrite compilation problem) for FreeBSD 10 using GCC 4.7 intead of native CLANG 3.3, adapting 4.1.2 patches :
header_rewrite rules are well applied and fully working !!!
I went back to ATS 4.1.2 to find that it does not work at all, no rewrite of header at all, look at the last message repeated twice (see "condition" in source) :
(header_rewrite) activated transaction hook via remap.config: new hook=5
(header_rewrite_dbg) Calling CTOR for Resources (RemapAPI)
(header_rewrite) rri: 0x7fffff7fa0b0
(header_rewrite) Building resource structure for hook (17)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client request header buffers for remap instance
(header_rewrite) returing with status: 0
(header_rewrite_dbg) Calling CTOR for Resources (InkAPI)
(header_rewrite) Building resource structure for hook (5)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client response header buffers
(header_rewrite) PATH requires remap initialization! Evaluating to false!
(header_rewrite) PATH requires remap initialization! Evaluating to false!
Now, I just hope that 4.2.2 will be released soon and ported to FreeBSD ports !
> > Leif,
> > thanks for the reminder but I don't really have this choice by now.
> > Still trying to use regex_map with header_rewrite plugin with
> > cond %{SEND_RESPONSE_HDR_HOOK}
> > cond %{PATH} "/(avi|mp3|jpg)$/"
> > rm-header Cache-Control
> > add-header Cache-Control max-age=3d
> > cond %{PATH} "/(doc|xls|ppt)$/"
> > rm-header Cache-Control
> > add-header Cache-Control max-age=1d
>
>
> The “expected” syntax for this rule is / was e.g.
>
>
> cond %{SEND_RESPONSE_HDR_HOOK}
> cond %{PATH} /(avi|mp3|jpg)$/
> rm-header Cache-Control
> add-header Cache-Control max-age=3d [L]
>
> cond %{SEND_RESPONSE_HDR_HOOK}
> cond %{PATH} /(doc|xls|ppt)$/
> rm-header Cache-Control
> add-header Cache-Control max-age=1d [L]
>
>
> (note that you shouldn’t have to use “” around the regexes). I’ll
> file a bug on the other issue in a bit, your syntax technically
> works, but ordering seems wrong.
I still need double quote around regexes in 4.1.2. Thanks again for the tip, it works in 4.2.2 !
Regards,
Denis
Re: header modification
Posted by Leif Hedstrom <zw...@apache.org>.
On Jun 11, 2014, at 1:29 AM, geodni@free.fr wrote:
>>> I want to keep objects like media (avi|jpg|mp3) for 7 days in cache while telling the client "max-age=3d", keep office file for 3
>>> days in cache while telling the client 1 day (using ATS 4.1.2 TTL are in seconds)
>>
>> I think you are overcomplicating things a bit though, use the protocol :). http://tools.ietf.org/html/rfc7234#section-5.2.2.9, e.g.
>>
>> Cache-Control: max-age=259200, s-maxage=604800
>>
>> This would be so easy if you could just configure your origin servers to behave and send the above. Barring that, you could use
>> header_rewrite to modify the origin server response to just that. I just cringe when people have to fight poorly configured origin
>> servers, this stuff belongs on the origin, and not on the caches.
>
Besides using the old version of ATS, which is not supported, your config acts slightly oddly. I think it might even be a bug (it doesn’t run the rules in the order you’d expect, i.e. it matches the second regex first).
> Leif,
> thanks for the reminder but I don't really have this choice by now.
> Still trying to use regex_map with header_rewrite plugin with
> cond %{SEND_RESPONSE_HDR_HOOK}
> cond %{PATH} "/(avi|mp3|jpg)$/"
> rm-header Cache-Control
> add-header Cache-Control max-age=3d
> cond %{PATH} "/(doc|xls|ppt)$/"
> rm-header Cache-Control
> add-header Cache-Control max-age=1d
The “expected” syntax for this rule is / was e.g.
cond %{SEND_RESPONSE_HDR_HOOK}
cond %{PATH} /(avi|mp3|jpg)$/
rm-header Cache-Control
add-header Cache-Control max-age=3d [L]
cond %{SEND_RESPONSE_HDR_HOOK}
cond %{PATH} /(doc|xls|ppt)$/
rm-header Cache-Control
add-header Cache-Control max-age=1d [L]
(note that you shouldn’t have to use “” around the regexes). I’ll file a bug on the other issue in a bit, your syntax technically works, but ordering seems wrong.
— Leif
Re: header modification
Posted by ge...@free.fr.
>> I want to keep objects like media (avi|jpg|mp3) for 7 days in cache while telling the client "max-age=3d", keep office file for 3
>>days in cache while telling the client 1 day (using ATS 4.1.2 TTL are in seconds)
>
>I think you are overcomplicating things a bit though, use the protocol :). http://tools.ietf.org/html/rfc7234#section-5.2.2.9, e.g.
>
> Cache-Control: max-age=259200, s-maxage=604800
>
>This would be so easy if you could just configure your origin servers to behave and send the above. Barring that, you could use
>header_rewrite to modify the origin server response to just that. I just cringe when people have to fight poorly configured origin
>servers, this stuff belongs on the origin, and not on the caches.
Leif,
thanks for the reminder but I don't really have this choice by now.
Still trying to use regex_map with header_rewrite plugin with
cond %{SEND_RESPONSE_HDR_HOOK}
cond %{PATH} "/(avi|mp3|jpg)$/"
rm-header Cache-Control
add-header Cache-Control max-age=3d
cond %{PATH} "/(doc|xls|ppt)$/"
rm-header Cache-Control
add-header Cache-Control max-age=1d
I got this diag which shows only second %{PATH} condition is tested but also ending with a strange message "PATH requires remap initialization! Evaluating to false!". Even with a file named test.doc, no header rewrite action is done :
(header_rewrite) activated transaction hook via remap.config: new hook=5
(header_rewrite_dbg) Calling CTOR for Resources (RemapAPI)
(header_rewrite) rri: 0x7fffffffb8e0
(header_rewrite) Building resource structure for hook (17)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client request header buffers for remap instance
(header_rewrite) Appending PATH to evaluation value: test.avi
(header_rewrite) Evaluating PATH
(header_rewrite) Test regular expression (doc|xls|ppt)$ : test.avi
(header_rewrite) returing with status: 0
(header_rewrite_dbg) Calling CTOR for Resources (InkAPI)
(header_rewrite) Building resource structure for hook (5)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client response header buffers
(header_rewrite) PATH requires remap initialization! Evaluating to false!
(header_rewrite) activated transaction hook via remap.config: new hook=5
(header_rewrite_dbg) Calling CTOR for Resources (RemapAPI)
(header_rewrite) rri: 0x7fffff6f90b0
(header_rewrite) Building resource structure for hook (17)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client request header buffers for remap instance
(header_rewrite) Appending PATH to evaluation value: test.doc
(header_rewrite) Evaluating PATH
(header_rewrite) Test regular expression (doc|xls|ppt)$ : test.doc
(header_rewrite) Successfully found regular expression match
(header_rewrite) OperatorRMHeader::exec() invoked on header Cache-Control
(header_rewrite) OperatorAddHeader::exec() invoked on header Cache-Control: max-age=1d
(header_rewrite) adding header Cache-Control
(header_rewrite) returing with status: 0
(header_rewrite_dbg) Calling CTOR for Resources (InkAPI)
(header_rewrite) Building resource structure for hook (5)
(header_rewrite) Adding TXN client request header buffers
(header_rewrite) Adding TXN client response header buffers
(header_rewrite) PATH requires remap initialization! Evaluating to false!
regards,
Denis
Re: header modification
Posted by Leif Hedstrom <zw...@apache.org>.
On Jun 10, 2014, at 10:14 AM, geodni@free.fr wrote:
>
> On Jun 10, 2014, at 3:18 AM, geodni@free.fr wrote:
>
>>> - what is the best place to modify header for a group of document type like images, remap.config (regex_map+header_rewrite) or cache.config ?
>>
>> Best place? On the origin for sure. If you for some reason can’t do that, header_rewrite ought to suffice for many (but not all) >rules. I’m pretty sure it can do regex matching on the path.
>
> I tried using ATS 4.1.2 but it doesn't work, maybe it's implemented into later versions like 4.2. I am waiting for its availability in the ports tree under FreeBSD.
>
>>> - is or will it possible to use header_rewrite plugin into cache.config to modify header ?
>>
>>
>> No, cache.config is orthoganol from header_rewrite. The former controls caching behavior, the latter modifies request and/or response >headers.
> OK.
> I want to keep objects like media (avi|jpg|mp3) for 7 days in cache while telling the client "max-age=3d", keep office file for 3 days in cache while telling the client 1 day (using ATS 4.1.2 TTL are in seconds)
I think you are overcomplicating things a bit though, use the protocol :). http://tools.ietf.org/html/rfc7234#section-5.2.2.9, e.g.
Cache-Control: max-age=259200, s-maxage=604800
This would be so easy if you could just configure your origin servers to behave and send the above. Barring that, you could use header_rewrite to modify the origin server response to just that. I just cringe when people have to fight poorly configured origin servers, this stuff belongs on the origin, and not on the caches.
Cheers,
— Leif
Re: header modification
Posted by ge...@free.fr.
On Jun 10, 2014, at 3:18 AM, geodni@free.fr wrote:
>> - what is the best place to modify header for a group of document type like images, remap.config (regex_map+header_rewrite) or cache.config ?
>
>Best place? On the origin for sure. If you for some reason can’t do that, header_rewrite ought to suffice for many (but not all) >rules. I’m pretty sure it can do regex matching on the path.
I tried using ATS 4.1.2 but it doesn't work, maybe it's implemented into later versions like 4.2. I am waiting for its availability in the ports tree under FreeBSD.
>> - is or will it possible to use header_rewrite plugin into cache.config to modify header ?
>
>
>No, cache.config is orthoganol from header_rewrite. The former controls caching behavior, the latter modifies request and/or response >headers.
OK.
I want to keep objects like media (avi|jpg|mp3) for 7 days in cache while telling the client "max-age=3d", keep office file for 3 days in cache while telling the client 1 day (using ATS 4.1.2 TTL are in seconds)
First I use cache.config
url_regex=.* suffix=avi,mp3,jpg ttl-in-cache=7d
url_regex=.* suffix=doc,xls,ppt ttl_in-cache=3d
Then I want to use remap.config
regex_map www.domain.com http://10.10.16.3 @plugin=header_rewrite.so @pparam=/usr/local/etc/trafficserver/header_rewrite_media.config
And header_rewrite_media.config
cond %{SEND_RESPONSE_HDR_HOOK}
cond %{PATH} "/(avi|mp3|jpg)$/"
rm-header Cache-Control
add-header Cache-Control max-age=3d
cond %{PATH} "/(doc|xls|ppt)$/"
rm-header Cache-Control
add-header Cache-Control max-age=1d
I can't get it to work. Most of CDN offer this kind of granularity but I don't find how to it with ATS.
Regards,
Denis
>Cheers,
>
>— Leif
Re: header modification
Posted by Leif Hedstrom <zw...@apache.org>.
On Jun 10, 2014, at 3:18 AM, geodni@free.fr wrote:
> - what is the best place to modify header for a group of document type like images, remap.config (regex_map+header_rewrite) or cache.config ?
Best place? On the origin for sure. If you for some reason can’t do that, header_rewrite ought to suffice for many (but not all) rules. I’m pretty sure it can do regex matching on the path.
> - is or will it possible to use header_rewrite plugin into cache.config to modify header ?
No, cache.config is orthoganol from header_rewrite. The former controls caching behavior, the latter modifies request and/or response headers.
Cheers,
— Leif
Re: header modification
Posted by James Peach <jp...@apache.org>.
On Jun 10, 2014, at 2:18 AM, geodni@free.fr wrote:
> Hi all,
> 3 questions :
> - could a "regex_suffix" rule be added into future version of ATS, someone can use it to match in one rule all kind of image like "suffix_regex=(png|bmp|jpg|jpeg|gif)" instead of writing one line by suffix ?
I think that you can already do this using comma as a separator, eg. suffix=png,bmp,jpg,jpeg,gif
> - what is the best place to modify header for a group of document type like images, remap.config (regex_map+header_rewrite) or cache.config ?
> - is or will it possible to use header_rewrite plugin into cache.config to modify header ?
>
> nice job done with ATS, thank you all.
>
> Regards,
> Denis