You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Hiroaki Nakamura <hn...@gmail.com> on 2016/03/08 13:03:18 UTC
How to set the Host header of the request to the origin server in
hierachical caching
Hi all,
I tried the example using the Hierarchical Caching
https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/hierachical-caching.en.html#example
I found the Host header of the request to the origin server become
origin.example.com.
I would like to this to be example.com.
How can I set the Host header value of the request to the origin
server to be the same as the request to the
child cache in hierarchical cashing settings?
Thanks,
Hiroaki
Re: How to set the Host header of the request to the origin server in
hierachical caching
Posted by Jeremy Payne <jp...@gmail.com>.
So to be clear, the important piece changed here was to change the HOST of
the destination URL. The HOST header was not changed.
The HOST header plays no significant role other than setting the value of
the HOST in the destination URL.
With your rule in place the GET to the parent should look like the below.
GET http://example.com/blah HTTP/1.1
HOST: example.com
On Wed, Mar 9, 2016 at 12:22 AM, Hiroaki Nakamura <hn...@gmail.com>
wrote:
> Hi Jeremy,
> Thanks for your reply.
>
> Now I solved the problem. I believe we need to set the Host header manually
> on the child cache, not the parent cache.
>
> My current settings are something like below:
>
> # child remap.config
> map http://example.com/ http://origin.example.com/
>
> # child parent.config
> dest_domain=. parent="parent1.example.com:80; parent2.example.com:80"
> round_robin=consistent_hash
>
> # child plugin.config
> header_rewrite.so header_rewrite.config
>
> # child header_rewrite.config
> cond %{SEND_REQUEST_HDR_HOOK}
> set-destination HOST %{HEADER:Host}
>
> # parent remap.config
> map http://example.com/ http://origin.example.com/
>
> Thanks all for your help!
> Hiroaki
>
> 2016-03-09 12:13 GMT+09:00 Jeremy Payne <jp...@gmail.com>:
> > at the parent layer, you will have to re-set the HOST header upon origin
> > communication.
> >
> > you can use the header rewrite plugin or the lua plugin to do this.
> >
> > header_rewrite:
> >
> >
> https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/header_rewrite.en.html
> >
> > ts_lua:
> >
> >
> https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/ts_lua.en.html
> >
> > function send_request()
> > ts.server_request.header['Host'] = 'example.com'
> > end
> >
> > function do_remap()
> > ts.hook(TS_LUA_HOOK_SEND_REQUEST_HDR, send_request)
> > return 0
> > end
> >
> >
> >
> > On Tue, Mar 8, 2016 at 8:47 PM, Hiroaki Nakamura <hn...@gmail.com>
> wrote:
> >>
> >> Thanks for replies.
> >> However I still cannot solve the issue.
> >>
> >> I reviewed my settings and found out I already had set
> >> proxy.config.url_remap.pristine_host_hdr to 1
> >> in both the child and the parent cache.
> >>
> >> I set all these configs to 1 in both the child and the parent cache.
> >> CONFIG proxy.config.url_remap.remap_required INT 1
> >> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
> >> CONFIG proxy.config.reverse_proxy.enabled INT 1
> >>
> >> My remap and parent configs were like this:
> >>
> >> child remap.config
> >> map http://example.com/ http://origin.example.com/
> >>
> >> child parent.config
> >> dest_domain=. parent="parent1.example.com:9090;
> >> parent2.example.com:9090" round_robin=consistent_hash
> >>
> >> parent remap.config
> >> map http://origin.example.com/ http://origin.example.com/
> >>
> >> I changed parent remap.config like
> >> map http://example.com/ http://origin.example.com/
> >>
> >> Still I got the Header value origin.example.com in the request to the
> >> origin server.
> >>
> >> Also I add %<{Host}cqh> to the logs_xml.config of the child and the
> >> parent cache.
> >> And I found out the Host value is origin.example.com, not example.com
> in
> >> the
> >> parent cache proxy log.
> >> In the child cache proxy log, the Host value is example.com, which is
> >> correct.
> >>
> >> So I think the child cache does not preserve the host header.
> >> I'm using trafficserver version 6.1.1 on CentOS 7.
> >> And my proxy chain is like below.
> >>
> >> nginx -> ATS child -> ATS parent
> >>
> >> Regards,
> >> Hiroaki
> >>
> >> 2016-03-09 7:05 GMT+09:00 Leif Hedstrom <zw...@apache.org>:
> >> >
> >> > On Mar 8, 2016, at 2:46 PM, Jeremy Payne <jp...@gmail.com> wrote:
> >> >
> >> > The parent remap would then have to match on:
> >> >
> >> > map http://origin.example.com/ http://origin.example.com/
> >> >
> >> > Last time I checked this is how it worked. I know there was talks of
> >> > having
> >> > a parent.config option which forced the child to send a relative URL
> >> > request. Instead of the full URL GET currently sent to the parent.
> >> >
> >> >
> >> >
> >> > That used to be an option in the remap configuration setting, but was
> >> > removed (because it was crippled). John Rushford is working on a
> >> > replacement
> >> > for this feature, such that parent request follow normal (“origin”)
> >> > request
> >> > semantics, and not the way parent proxying works today. I’m hoping
> that
> >> > it’ll be configurable per rule in parent.config (and not a global
> >> > setting
> >> > as it was before). Alternatively, maybe it could be an overridable
> >> > records.config setting.
> >> >
> >> > That much said, I believe Jeremy is right, in that pristine host
> headers
> >> > don’t apply to parent selection. It looks like parent selection
> happens
> >> > right after remapping, and it does not consider pristine host headers.
> >> > John
> >> > and Phil, can you confirm?
> >> >
> >> > Note that the parent.config selection happens on the remapped host
> name
> >> > (I’m
> >> > pretty sure, I’m sure someone will correct me :).
> >> >
> >> > Cheers,
> >> >
> >> > — Leif
> >> >
> >
> >
>
Re: How to set the Host header of the request to the origin server in
hierachical caching
Posted by Hiroaki Nakamura <hn...@gmail.com>.
Hi Jeremy,
Thanks for your reply.
Now I solved the problem. I believe we need to set the Host header manually
on the child cache, not the parent cache.
My current settings are something like below:
# child remap.config
map http://example.com/ http://origin.example.com/
# child parent.config
dest_domain=. parent="parent1.example.com:80; parent2.example.com:80"
round_robin=consistent_hash
# child plugin.config
header_rewrite.so header_rewrite.config
# child header_rewrite.config
cond %{SEND_REQUEST_HDR_HOOK}
set-destination HOST %{HEADER:Host}
# parent remap.config
map http://example.com/ http://origin.example.com/
Thanks all for your help!
Hiroaki
2016-03-09 12:13 GMT+09:00 Jeremy Payne <jp...@gmail.com>:
> at the parent layer, you will have to re-set the HOST header upon origin
> communication.
>
> you can use the header rewrite plugin or the lua plugin to do this.
>
> header_rewrite:
>
> https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/header_rewrite.en.html
>
> ts_lua:
>
> https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/ts_lua.en.html
>
> function send_request()
> ts.server_request.header['Host'] = 'example.com'
> end
>
> function do_remap()
> ts.hook(TS_LUA_HOOK_SEND_REQUEST_HDR, send_request)
> return 0
> end
>
>
>
> On Tue, Mar 8, 2016 at 8:47 PM, Hiroaki Nakamura <hn...@gmail.com> wrote:
>>
>> Thanks for replies.
>> However I still cannot solve the issue.
>>
>> I reviewed my settings and found out I already had set
>> proxy.config.url_remap.pristine_host_hdr to 1
>> in both the child and the parent cache.
>>
>> I set all these configs to 1 in both the child and the parent cache.
>> CONFIG proxy.config.url_remap.remap_required INT 1
>> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
>> CONFIG proxy.config.reverse_proxy.enabled INT 1
>>
>> My remap and parent configs were like this:
>>
>> child remap.config
>> map http://example.com/ http://origin.example.com/
>>
>> child parent.config
>> dest_domain=. parent="parent1.example.com:9090;
>> parent2.example.com:9090" round_robin=consistent_hash
>>
>> parent remap.config
>> map http://origin.example.com/ http://origin.example.com/
>>
>> I changed parent remap.config like
>> map http://example.com/ http://origin.example.com/
>>
>> Still I got the Header value origin.example.com in the request to the
>> origin server.
>>
>> Also I add %<{Host}cqh> to the logs_xml.config of the child and the
>> parent cache.
>> And I found out the Host value is origin.example.com, not example.com in
>> the
>> parent cache proxy log.
>> In the child cache proxy log, the Host value is example.com, which is
>> correct.
>>
>> So I think the child cache does not preserve the host header.
>> I'm using trafficserver version 6.1.1 on CentOS 7.
>> And my proxy chain is like below.
>>
>> nginx -> ATS child -> ATS parent
>>
>> Regards,
>> Hiroaki
>>
>> 2016-03-09 7:05 GMT+09:00 Leif Hedstrom <zw...@apache.org>:
>> >
>> > On Mar 8, 2016, at 2:46 PM, Jeremy Payne <jp...@gmail.com> wrote:
>> >
>> > The parent remap would then have to match on:
>> >
>> > map http://origin.example.com/ http://origin.example.com/
>> >
>> > Last time I checked this is how it worked. I know there was talks of
>> > having
>> > a parent.config option which forced the child to send a relative URL
>> > request. Instead of the full URL GET currently sent to the parent.
>> >
>> >
>> >
>> > That used to be an option in the remap configuration setting, but was
>> > removed (because it was crippled). John Rushford is working on a
>> > replacement
>> > for this feature, such that parent request follow normal (“origin”)
>> > request
>> > semantics, and not the way parent proxying works today. I’m hoping that
>> > it’ll be configurable per rule in parent.config (and not a global
>> > setting
>> > as it was before). Alternatively, maybe it could be an overridable
>> > records.config setting.
>> >
>> > That much said, I believe Jeremy is right, in that pristine host headers
>> > don’t apply to parent selection. It looks like parent selection happens
>> > right after remapping, and it does not consider pristine host headers.
>> > John
>> > and Phil, can you confirm?
>> >
>> > Note that the parent.config selection happens on the remapped host name
>> > (I’m
>> > pretty sure, I’m sure someone will correct me :).
>> >
>> > Cheers,
>> >
>> > — Leif
>> >
>
>
Re: How to set the Host header of the request to the origin server in
hierachical caching
Posted by Jeremy Payne <jp...@gmail.com>.
at the parent layer, you will have to re-set the HOST header upon origin
communication.
you can use the header rewrite plugin or the lua plugin to do this.
header_rewrite:
https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/header_rewrite.en.html
ts_lua:
https://trafficserver.readthedocs.org/en/6.0.x/reference/plugins/ts_lua.en.html
function send_request()
ts.server_request.header['Host'] = 'example.com'
end
function do_remap()
ts.hook(TS_LUA_HOOK_SEND_REQUEST_HDR, send_request)
return 0
end
On Tue, Mar 8, 2016 at 8:47 PM, Hiroaki Nakamura <hn...@gmail.com> wrote:
> Thanks for replies.
> However I still cannot solve the issue.
>
> I reviewed my settings and found out I already had set
> proxy.config.url_remap.pristine_host_hdr to 1
> in both the child and the parent cache.
>
> I set all these configs to 1 in both the child and the parent cache.
> CONFIG proxy.config.url_remap.remap_required INT 1
> CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
> CONFIG proxy.config.reverse_proxy.enabled INT 1
>
> My remap and parent configs were like this:
>
> child remap.config
> map http://example.com/ http://origin.example.com/
>
> child parent.config
> dest_domain=. parent="parent1.example.com:9090;
> parent2.example.com:9090" round_robin=consistent_hash
>
> parent remap.config
> map http://origin.example.com/ http://origin.example.com/
>
> I changed parent remap.config like
> map http://example.com/ http://origin.example.com/
>
> Still I got the Header value origin.example.com in the request to the
> origin server.
>
> Also I add %<{Host}cqh> to the logs_xml.config of the child and the
> parent cache.
> And I found out the Host value is origin.example.com, not example.com in
> the
> parent cache proxy log.
> In the child cache proxy log, the Host value is example.com, which is
> correct.
>
> So I think the child cache does not preserve the host header.
> I'm using trafficserver version 6.1.1 on CentOS 7.
> And my proxy chain is like below.
>
> nginx -> ATS child -> ATS parent
>
> Regards,
> Hiroaki
>
> 2016-03-09 7:05 GMT+09:00 Leif Hedstrom <zw...@apache.org>:
> >
> > On Mar 8, 2016, at 2:46 PM, Jeremy Payne <jp...@gmail.com> wrote:
> >
> > The parent remap would then have to match on:
> >
> > map http://origin.example.com/ http://origin.example.com/
> >
> > Last time I checked this is how it worked. I know there was talks of
> having
> > a parent.config option which forced the child to send a relative URL
> > request. Instead of the full URL GET currently sent to the parent.
> >
> >
> >
> > That used to be an option in the remap configuration setting, but was
> > removed (because it was crippled). John Rushford is working on a
> replacement
> > for this feature, such that parent request follow normal (“origin”)
> request
> > semantics, and not the way parent proxying works today. I’m hoping that
> > it’ll be configurable per rule in parent.config (and not a global
> setting
> > as it was before). Alternatively, maybe it could be an overridable
> > records.config setting.
> >
> > That much said, I believe Jeremy is right, in that pristine host headers
> > don’t apply to parent selection. It looks like parent selection happens
> > right after remapping, and it does not consider pristine host headers.
> John
> > and Phil, can you confirm?
> >
> > Note that the parent.config selection happens on the remapped host name
> (I’m
> > pretty sure, I’m sure someone will correct me :).
> >
> > Cheers,
> >
> > — Leif
> >
>
Re: How to set the Host header of the request to the origin server in
hierachical caching
Posted by Hiroaki Nakamura <hn...@gmail.com>.
Thanks for replies.
However I still cannot solve the issue.
I reviewed my settings and found out I already had set
proxy.config.url_remap.pristine_host_hdr to 1
in both the child and the parent cache.
I set all these configs to 1 in both the child and the parent cache.
CONFIG proxy.config.url_remap.remap_required INT 1
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
CONFIG proxy.config.reverse_proxy.enabled INT 1
My remap and parent configs were like this:
child remap.config
map http://example.com/ http://origin.example.com/
child parent.config
dest_domain=. parent="parent1.example.com:9090;
parent2.example.com:9090" round_robin=consistent_hash
parent remap.config
map http://origin.example.com/ http://origin.example.com/
I changed parent remap.config like
map http://example.com/ http://origin.example.com/
Still I got the Header value origin.example.com in the request to the
origin server.
Also I add %<{Host}cqh> to the logs_xml.config of the child and the
parent cache.
And I found out the Host value is origin.example.com, not example.com in the
parent cache proxy log.
In the child cache proxy log, the Host value is example.com, which is correct.
So I think the child cache does not preserve the host header.
I'm using trafficserver version 6.1.1 on CentOS 7.
And my proxy chain is like below.
nginx -> ATS child -> ATS parent
Regards,
Hiroaki
2016-03-09 7:05 GMT+09:00 Leif Hedstrom <zw...@apache.org>:
>
> On Mar 8, 2016, at 2:46 PM, Jeremy Payne <jp...@gmail.com> wrote:
>
> The parent remap would then have to match on:
>
> map http://origin.example.com/ http://origin.example.com/
>
> Last time I checked this is how it worked. I know there was talks of having
> a parent.config option which forced the child to send a relative URL
> request. Instead of the full URL GET currently sent to the parent.
>
>
>
> That used to be an option in the remap configuration setting, but was
> removed (because it was crippled). John Rushford is working on a replacement
> for this feature, such that parent request follow normal (“origin”) request
> semantics, and not the way parent proxying works today. I’m hoping that
> it’ll be configurable per rule in parent.config (and not a global setting
> as it was before). Alternatively, maybe it could be an overridable
> records.config setting.
>
> That much said, I believe Jeremy is right, in that pristine host headers
> don’t apply to parent selection. It looks like parent selection happens
> right after remapping, and it does not consider pristine host headers. John
> and Phil, can you confirm?
>
> Note that the parent.config selection happens on the remapped host name (I’m
> pretty sure, I’m sure someone will correct me :).
>
> Cheers,
>
> — Leif
>
Re: How to set the Host header of the request to the origin server in hierachical caching
Posted by Leif Hedstrom <zw...@apache.org>.
> On Mar 8, 2016, at 2:46 PM, Jeremy Payne <jp...@gmail.com> wrote:
>
> The parent remap would then have to match on:
>
> map http://origin.example.com/ <http://origin.example.com/> http://origin.example.com/ <http://origin.example.com/>
>
> Last time I checked this is how it worked. I know there was talks of having a parent.config option which forced the child to send a relative URL request. Instead of the full URL GET currently sent to the parent.
That used to be an option in the remap configuration setting, but was removed (because it was crippled). John Rushford is working on a replacement for this feature, such that parent request follow normal (“origin”) request semantics, and not the way parent proxying works today. I’m hoping that it’ll be configurable per rule in parent.config (and not a global setting as it was before). Alternatively, maybe it could be an overridable records.config setting.
That much said, I believe Jeremy is right, in that pristine host headers don’t apply to parent selection. It looks like parent selection happens right after remapping, and it does not consider pristine host headers. John and Phil, can you confirm?
Note that the parent.config selection happens on the remapped host name (I’m pretty sure, I’m sure someone will correct me :).
Cheers,
— Leif
Re: How to set the Host header of the request to the origin server in
hierachical caching
Posted by Jeremy Payne <jp...@gmail.com>.
In this pristine configuration, isnt the HOST header seen at the parent
layer still not considered by remap due to the fact the child=>parent GET
is of the absolute URL ?
So at the parent layer you would still need a remap.config entry to match
on http://origin.example.com
In other word, user request at the child looks like this
GET /foo.bar HTTP/1.1
HOST: example.com
The request is translated by the child upon sending to the parent. This
translation is done regardless of pristine or not. So
even if you did change the HOST header, the host in the GET url takes
precedence. At least with respect to remap and given my testing.
GET http://origin.example.com/foo.bar HTTP/1.1
HOST:example.com
The parent remap would then have to match on:
map http://origin.example.com/ http://origin.example.com/
Last time I checked this is how it worked. I know there was talks of having
a parent.config option which forced the child to send a relative URL
request. Instead of the full URL GET currently sent to the parent.
On Tue, Mar 8, 2016 at 2:08 PM, Miles Libbey <ml...@apache.org> wrote:
> I think you'd set
> proxy.config.url_remap.pristine_host_hdr (
> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/records.config.en.html#proxy-config-url-remap-pristine-host-hdr
> )
> to 1 (either for that remap rule using the conf_remap plugin (
> https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/conf_remap.en.html)
> or globally).
>
> Then you'd have
> child remap:
> map http://example.com http://origin.example.com (perhaps
> with @plugin=conf_remap.so
> @pparam=proxy.config.url_remap.pristine_host_hdr=1)
> child parent.config
> dest_domain=example.com method=get parent="parent1.example.com:80
> and parent remap
> map http://example.com http://origin.example.com
>
>
> miles
>
>
>
> On Tuesday, March 8, 2016 4:04 AM, Hiroaki Nakamura <hn...@gmail.com>
> wrote:
>
>
> Hi all,
>
> I tried the example using the Hierarchical Caching
>
> https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/hierachical-caching.en.html#example
>
> I found the Host header of the request to the origin server become
> origin.example.com.
> I would like to this to be example.com.
> How can I set the Host header value of the request to the origin
> server to be the same as the request to the
> child cache in hierarchical cashing settings?
>
> Thanks,
> Hiroaki
>
>
>
Re: How to set the Host header of the request to the origin server
in hierachical caching
Posted by Miles Libbey <ml...@apache.org>.
I think you'd setproxy.config.url_remap.pristine_host_hdr (https://docs.trafficserver.apache.org/en/latest/admin-guide/files/records.config.en.html#proxy-config-url-remap-pristine-host-hdr)
to 1 (either for that remap rule using the conf_remap plugin (https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/conf_remap.en.html) or globally).
Then you'd havechild remap: map http://example.com http://origin.example.com (perhaps with @plugin=conf_remap.so @pparam=proxy.config.url_remap.pristine_host_hdr=1)child parent.configdest_domain=example.com method=get parent="parent1.example.com:80
and parent remapmap http://example.com http://origin.example.com
miles
On Tuesday, March 8, 2016 4:04 AM, Hiroaki Nakamura <hn...@gmail.com> wrote:
Hi all,
I tried the example using the Hierarchical Caching
https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/hierachical-caching.en.html#example
I found the Host header of the request to the origin server become
origin.example.com.
I would like to this to be example.com.
How can I set the Host header value of the request to the origin
server to be the same as the request to the
child cache in hierarchical cashing settings?
Thanks,
Hiroaki