You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@trafficserver.apache.org by Josh Gitlin <jg...@pinnacle21.com> on 2020/05/07 17:51:07 UTC

Remap causing parent servers to not have a match

Hello,

Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.

I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.

I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.

Contents of remap.config now:

map http://www.proxy.example.com http://www.example.com/
map https://www.proxy.example.com https://www.example.com/


Proposed fix to my config:

map http://www.proxy.example.com http://www.example.com/
map https://www.proxy.example.com https://www.example.com/
map http://www.example.com http://www.example.com/
map https://www.example.com https://www.example.com/

Is this the "right" way to fix this issue? The duplication feels like there must be a better way...

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

Re: Remap causing parent servers to not have a match

Posted by Josh Gitlin <jg...@pinnacle21.com>.

Thanks Miles, that's helpful

I think the issue here is that what I want isn't parent-child caches but peer caches. I'm looking into the header rewrite plugin to see if I can do something there to break the loop.

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

On May 7, 2020, at 3:31 PM, Miles Libbey <ml...@apache.org>> wrote:

For our cache hierarchy we do:
child:
map inbound inbound
parent.config: dest_domain=inbound scheme=http parent="..." go_direct=false

parent:
map inbound origin

We do it this way because:
- we can keep https throughout the hierarchy-- the children and
parents can use the same certificates. We sometimes go to external
sources for origin -- for instance a cloud storage provider. We
wouldn't be able to get a cert for those domains.
- We can have as many/as few layers of hierarchy as you want. Want 4
layers? Make the first 3 children, the last a parent.

But, we have distinct hardware for children and parents -- we've not
tried to have a machine act both as a child and a parent at the same
time. I suppose you'd turn off loop detection, list the other machine
in the parent.config's parent section, and "self" in the secondary
ring?
miles

On Thu, May 7, 2020 at 12:05 PM Josh Gitlin <jg...@pinnacle21.com>> wrote:

The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:

Hello,

Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.

I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.

I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.

Contents of remap.config now:

map http://www.proxy.example.com http://www.example.com/
map https://www.proxy.example.com https://www.example.com/

Proposed fix to my config:

map http://www.proxy.example.com http://www.example.com/
map https://www.proxy.example.com https://www.example.com/
map http://www.example.com http://www.example.com/
map https://www.example.com https://www.example.com/

Is this the "right" way to fix this issue? The duplication feels like there must be a better way...

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com

PINNACLE 21
www.pinnacle21.com

Re: Remap causing parent servers to not have a match

Posted by Miles Libbey <ml...@apache.org>.

For our cache hierarchy we do:
child:
map inbound inbound
parent.config: dest_domain=inbound scheme=http parent="..." go_direct=false

parent:
map inbound origin

We do it this way because:
- we can keep https throughout the hierarchy-- the children and
parents can use the same certificates. We sometimes go to external
sources for origin -- for instance a cloud storage provider. We
wouldn't be able to get a cert for those domains.
- We can have as many/as few layers of hierarchy as you want. Want 4
layers? Make the first 3 children, the last a parent.

But, we have distinct hardware for children and parents -- we've not
tried to have a machine act both as a child and a parent at the same
time. I suppose you'd turn off loop detection, list the other machine
in the parent.config's parent section, and "self" in the secondary
ring?
miles

On Thu, May 7, 2020 at 12:05 PM Josh Gitlin <jg...@pinnacle21.com> wrote:
>
> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)
>
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com
>
> PINNACLE 21
> www.pinnacle21.com
>
> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>
> Hello,
>
> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>
> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>
> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>
> Contents of remap.config now:
>
> map http://www.proxy.example.com http://www.example.com/
> map https://www.proxy.example.com https://www.example.com/
>
>
> Proposed fix to my config:
>
> map http://www.proxy.example.com http://www.example.com/
> map https://www.proxy.example.com https://www.example.com/
> map http://www.example.com http://www.example.com/
> map https://www.example.com https://www.example.com/
>
> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com
>
> PINNACLE 21
> www.pinnacle21.com
>
>

Re: Remap causing parent servers to not have a match

Posted by John Rushford <jj...@gmail.com>.

I’ll get a back port for this to 8.1

Sent from my iPhone

> On May 7, 2020, at 2:38 PM, Leif Hedstrom <zw...@apache.org> wrote:
> 
> 
> 
>> On May 7, 2020, at 2:21 PM, Miles Libbey <ml...@apache.org> wrote:
>> 
>> Unfortunately, 9 is neither released nor stable yet :/
> 
> Correct. Possibly someone can back port these changes to their 8.x tree  (I know others have  :-) ).
> 
>> 
>> Though we've not done it, I was thinking that you could use
>> parent.config's "primary ring" to get the traffic to the peer, but,
>> when down, sends it to itself through the secondary ring, and hitting
>> its parent rule.
>> 
>> I'm also not entirely sure how the 9 feature works, but, I think the
>> primary benefit is that you wouldn't have to generate different
>> parent.config for each peer. (Whereas in my described version, peer1
>> would need peer2 in the parent list, and vice-versa for peer2. Both
>> could have 127.0.0.1 as their secondary ring)
> 
> Right. That’s the point of this feature, one parent.config for the “cluster”, and magic happens.
> 
> — Leif
> 
>> 
>>> On Thu, May 7, 2020 at 12:53 PM Josh Gitlin <jg...@pinnacle21.com> wrote:
>>> 
>>> Thanks Leif! I am on 8.0.7, so maybe I should switch to 9
>>> 
>>> Josh Gitlin
>>> Principal DevOps Engineer
>>> jgitlin@pinnacle21.com
>>> 
>>> PINNACLE 21
>>> www.pinnacle21.com
>>> 
>>>> On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org> wrote:
>>> 
>>> 
>>> 
>>>> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>>> 
>>> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)
>>> 
>>> 
>>> 
>>> In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.
>>> 
>>> — Leif
>>> 
>>> https://github.com/apache/trafficserver/pull/5544
>>> 
>>> 
>>> Josh Gitlin
>>> Principal DevOps Engineer
>>> jgitlin@pinnacle21.com
>>> 
>>> PINNACLE 21
>>> www.pinnacle21.com
>>> 
>>>> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>>> 
>>> Hello,
>>> 
>>> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>>> 
>>> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>>> 
>>> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>>> 
>>> Contents of remap.config now:
>>> 
>>> map http://www.proxy.example.com http://www.example.com/
>>> map https://www.proxy.example.com https://www.example.com/
>>> 
>>> 
>>> Proposed fix to my config:
>>> 
>>> map http://www.proxy.example.com http://www.example.com/
>>> map https://www.proxy.example.com https://www.example.com/
>>> map http://www.example.com http://www.example.com/
>>> map https://www.example.com https://www.example.com/
>>> 
>>> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>>> 
>>> Josh Gitlin
>>> Principal DevOps Engineer
>>> jgitlin@pinnacle21.com
>>> 
>>> PINNACLE 21
>>> www.pinnacle21.com
>>> 
>>> 
>>> 
>>> 
>

Re: Remap causing parent servers to not have a match

Posted by Josh Gitlin <jg...@pinnacle21.com>.

Thanks for everyone's input!

Just managed to get my desired configuration working by going direct when the src_ip matches one of the other parents, and having the remap rules "doubled", as follows:

parent.config:

# Bypass peers if the request came from a peer already
dest_domain=. src_ip="10.21.14.25" go_direct=true
dest_domain=. src_ip="10.21.14.4" go_direct=true

# Otherwsie, send to our peers
dest_domain=. scheme=http  parent="10.21.14.25:8080;10.21.14.4:8080"  round_robin=strict  go_direct=true
dest_domain=. scheme=https parent="10.21.14.25:8443;10.21.14.4:8443"  round_robin=strict  go_direct=true

remap.config:

map http://www.proxy.example.com http://www.example.com/
map http://www.example.com http://www.example.com/

Since I'm using Chef to build this, it was trivial to make different parent.confs for each node:

# Bypass peers if the request came from a peer already
<% @peers.each do |peer| -%>
dest_domain=. src_ip="<%= peer %>" go_direct=true
<% end -%>

# Otherwsie, send to our peers
dest_domain=. scheme=http  parent="<%= @peers.map{|p| "#{p}:8080"}.join ";" -%>"  round_robin=strict  go_direct=true
dest_domain=. scheme=https parent="<%= @peers.map{|p| "#{p}:8443"}.join ";" -%>"  round_robin=strict  go_direct=true

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

On May 7, 2020, at 5:03 PM, John Rushford <jj...@gmail.com>> wrote:

In 9.0 when the parent.config is loaded, the parents for each config line are examined to see if the fqdn/ip matches the host itself or is localhost, if so that host is marked as unavailable for use by ParentSelection and NextHopSelection so prevent looping.  However if you need to use the the host and you're sure that you won't loop because the combination of remap rules and the matching parent.config will not cause any problem, you can disable the self detection by adding 'ignore_self_detect=true' in the parent.config line and the the host will allow itself to be used as a parent.  8.1 has the self detection markdown as well but the 'ignore_self_detect' flag is not available there but should be backported.

On Thu, May 7, 2020 at 2:38 PM Leif Hedstrom <zw...@apache.org>> wrote:

> On May 7, 2020, at 2:21 PM, Miles Libbey <ml...@apache.org>> wrote:
>
> Unfortunately, 9 is neither released nor stable yet :/

Correct. Possibly someone can back port these changes to their 8.x tree  (I know others have  :-) ).

>
> Though we've not done it, I was thinking that you could use
> parent.config's "primary ring" to get the traffic to the peer, but,
> when down, sends it to itself through the secondary ring, and hitting
> its parent rule.
>
> I'm also not entirely sure how the 9 feature works, but, I think the
> primary benefit is that you wouldn't have to generate different
> parent.config for each peer. (Whereas in my described version, peer1
> would need peer2 in the parent list, and vice-versa for peer2. Both
> could have 127.0.0.1 as their secondary ring)

Right. That’s the point of this feature, one parent.config for the “cluster”, and magic happens.

— Leif

>
> On Thu, May 7, 2020 at 12:53 PM Josh Gitlin <jg...@pinnacle21.com>> wrote:
>>
>> Thanks Leif! I am on 8.0.7, so maybe I should switch to 9
>>
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com<ma...@pinnacle21.com>
>>
>> PINNACLE 21
>> www.pinnacle21.com<http://www.pinnacle21.com/>
>>
>> On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org>> wrote:
>>
>>
>>
>> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com>> wrote:
>>
>> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)
>>
>>
>>
>> In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.
>>
>> — Leif
>>
>> https://github.com/apache/trafficserver/pull/5544
>>
>>
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com<ma...@pinnacle21.com>
>>
>> PINNACLE 21
>> www.pinnacle21.com<http://www.pinnacle21.com/>
>>
>> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com>> wrote:
>>
>> Hello,
>>
>> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>>
>> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>>
>> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>>
>> Contents of remap.config now:
>>
>> map http://www.proxy.example.com<http://www.proxy.example.com/> http://www.example.com/
>> map https://www.proxy.example.com<https://www.proxy.example.com/> https://www.example.com/
>>
>>
>> Proposed fix to my config:
>>
>> map http://www.proxy.example.com<http://www.proxy.example.com/> http://www.example.com/
>> map https://www.proxy.example.com<https://www.proxy.example.com/> https://www.example.com/
>> map http://www.example.com<http://www.example.com/> http://www.example.com/
>> map https://www.example.com<https://www.example.com/> https://www.example.com/
>>
>> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>>
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com<ma...@pinnacle21.com>
>>
>> PINNACLE 21
>> www.pinnacle21.com<http://www.pinnacle21.com/>
>>
>>
>>
>>

--
John Rushford
jjrushford@gmail.com<ma...@gmail.com>

Re: Remap causing parent servers to not have a match

Posted by John Rushford <jj...@gmail.com>.

In 9.0 when the parent.config is loaded, the parents for each config line
are examined to see if the fqdn/ip matches the host itself or is localhost,
if so that host is marked as unavailable for use by ParentSelection and
NextHopSelection so prevent looping.  However if you need to use the the
host and you're sure that you won't loop because the combination of remap
rules and the matching parent.config will not cause any problem, you can
disable the self detection by adding 'ignore_self_detect=true' in the
parent.config line and the the host will allow itself to be used as a
parent.  8.1 has the self detection markdown as well but the
'ignore_self_detect' flag is not available there but should be backported.

On Thu, May 7, 2020 at 2:38 PM Leif Hedstrom <zw...@apache.org> wrote:

>
>
> > On May 7, 2020, at 2:21 PM, Miles Libbey <ml...@apache.org> wrote:
> >
> > Unfortunately, 9 is neither released nor stable yet :/
>
> Correct. Possibly someone can back port these changes to their 8.x tree
> (I know others have  :-) ).
>
> >
> > Though we've not done it, I was thinking that you could use
> > parent.config's "primary ring" to get the traffic to the peer, but,
> > when down, sends it to itself through the secondary ring, and hitting
> > its parent rule.
> >
> > I'm also not entirely sure how the 9 feature works, but, I think the
> > primary benefit is that you wouldn't have to generate different
> > parent.config for each peer. (Whereas in my described version, peer1
> > would need peer2 in the parent list, and vice-versa for peer2. Both
> > could have 127.0.0.1 as their secondary ring)
>
> Right. That’s the point of this feature, one parent.config for the
> “cluster”, and magic happens.
>
> — Leif
>
> >
> > On Thu, May 7, 2020 at 12:53 PM Josh Gitlin <jg...@pinnacle21.com>
> wrote:
> >>
> >> Thanks Leif! I am on 8.0.7, so maybe I should switch to 9
> >>
> >> Josh Gitlin
> >> Principal DevOps Engineer
> >> jgitlin@pinnacle21.com
> >>
> >> PINNACLE 21
> >> www.pinnacle21.com
> >>
> >> On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org> wrote:
> >>
> >>
> >>
> >> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
> >>
> >> The more I dig into this, the more I realize I have gone horribly wrong
> somewhere, as I seem to have just created an infinite parent proxy loop. So
> I may need to RTFM again to fix this broken design! :)
> >>
> >>
> >>
> >> In ATS 9.x, there is a “self” detection mechanism for this exact
> purpose (“cache cluster”). It prevents a box to parent a request that is
> hashing to itself.
> >>
> >> — Leif
> >>
> >> https://github.com/apache/trafficserver/pull/5544
> >>
> >>
> >> Josh Gitlin
> >> Principal DevOps Engineer
> >> jgitlin@pinnacle21.com
> >>
> >> PINNACLE 21
> >> www.pinnacle21.com
> >>
> >> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
> >>
> >> Hello,
> >>
> >> Apologies if this was covered in the docs or a previous message; I
> couldn't find an answer in my search.
> >>
> >> I am having an issue with remapping and parent caching. I have two
> Apache Traffic Server instances for HA, and each one has the other
> configured as its parent cache. The goal being a shared cache, because the
> two instances are behind a load balancer with leastconn distribution.
> >>
> >> I am seeing an issue where cache misses on server B get forwarded to
> server A with the remapped URL and server A refuses to serve because it
> does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL")
> I know I can resolve this by simply adding the original URL to the remap
> config, but that felt like the wrong fix.
> >>
> >> Contents of remap.config now:
> >>
> >> map http://www.proxy.example.com http://www.example.com/
> >> map https://www.proxy.example.com https://www.example.com/
> >>
> >>
> >> Proposed fix to my config:
> >>
> >> map http://www.proxy.example.com http://www.example.com/
> >> map https://www.proxy.example.com https://www.example.com/
> >> map http://www.example.com http://www.example.com/
> >> map https://www.example.com https://www.example.com/
> >>
> >> Is this the "right" way to fix this issue? The duplication feels like
> there must be a better way...
> >>
> >> Josh Gitlin
> >> Principal DevOps Engineer
> >> jgitlin@pinnacle21.com
> >>
> >> PINNACLE 21
> >> www.pinnacle21.com
> >>
> >>
> >>
> >>
>
>

-- 
John Rushford
jjrushford@gmail.com

Re: Remap causing parent servers to not have a match

Posted by Leif Hedstrom <zw...@apache.org>.


> On May 7, 2020, at 2:21 PM, Miles Libbey <ml...@apache.org> wrote:
> 
> Unfortunately, 9 is neither released nor stable yet :/

Correct. Possibly someone can back port these changes to their 8.x tree  (I know others have  :-) ).

> 
> Though we've not done it, I was thinking that you could use
> parent.config's "primary ring" to get the traffic to the peer, but,
> when down, sends it to itself through the secondary ring, and hitting
> its parent rule.
> 
> I'm also not entirely sure how the 9 feature works, but, I think the
> primary benefit is that you wouldn't have to generate different
> parent.config for each peer. (Whereas in my described version, peer1
> would need peer2 in the parent list, and vice-versa for peer2. Both
> could have 127.0.0.1 as their secondary ring)

Right. That’s the point of this feature, one parent.config for the “cluster”, and magic happens.

— Leif

> 
> On Thu, May 7, 2020 at 12:53 PM Josh Gitlin <jg...@pinnacle21.com> wrote:
>> 
>> Thanks Leif! I am on 8.0.7, so maybe I should switch to 9
>> 
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com
>> 
>> PINNACLE 21
>> www.pinnacle21.com
>> 
>> On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org> wrote:
>> 
>> 
>> 
>> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>> 
>> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)
>> 
>> 
>> 
>> In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.
>> 
>> — Leif
>> 
>> https://github.com/apache/trafficserver/pull/5544
>> 
>> 
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com
>> 
>> PINNACLE 21
>> www.pinnacle21.com
>> 
>> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>> 
>> Hello,
>> 
>> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>> 
>> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>> 
>> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>> 
>> Contents of remap.config now:
>> 
>> map http://www.proxy.example.com http://www.example.com/
>> map https://www.proxy.example.com https://www.example.com/
>> 
>> 
>> Proposed fix to my config:
>> 
>> map http://www.proxy.example.com http://www.example.com/
>> map https://www.proxy.example.com https://www.example.com/
>> map http://www.example.com http://www.example.com/
>> map https://www.example.com https://www.example.com/
>> 
>> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>> 
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com
>> 
>> PINNACLE 21
>> www.pinnacle21.com
>> 
>> 
>> 
>>

Re: Remap causing parent servers to not have a match

Posted by Miles Libbey <ml...@apache.org>.

Unfortunately, 9 is neither released nor stable yet :/

Though we've not done it, I was thinking that you could use
parent.config's "primary ring" to get the traffic to the peer, but,
when down, sends it to itself through the secondary ring, and hitting
its parent rule.

I'm also not entirely sure how the 9 feature works, but, I think the
primary benefit is that you wouldn't have to generate different
parent.config for each peer. (Whereas in my described version, peer1
would need peer2 in the parent list, and vice-versa for peer2. Both
could have 127.0.0.1 as their secondary ring)

On Thu, May 7, 2020 at 12:53 PM Josh Gitlin <jg...@pinnacle21.com> wrote:
>
> Thanks Leif! I am on 8.0.7, so maybe I should switch to 9
>
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com
>
> PINNACLE 21
> www.pinnacle21.com
>
> On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org> wrote:
>
>
>
> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>
> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :)
>
>
>
> In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.
>
> — Leif
>
> https://github.com/apache/trafficserver/pull/5544
>
>
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com
>
> PINNACLE 21
> www.pinnacle21.com
>
> On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
>
> Hello,
>
> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>
> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>
> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>
> Contents of remap.config now:
>
> map http://www.proxy.example.com http://www.example.com/
> map https://www.proxy.example.com https://www.example.com/
>
>
> Proposed fix to my config:
>
> map http://www.proxy.example.com http://www.example.com/
> map https://www.proxy.example.com https://www.example.com/
> map http://www.example.com http://www.example.com/
> map https://www.example.com https://www.example.com/
>
> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com
>
> PINNACLE 21
> www.pinnacle21.com
>
>
>
>

Re: Remap causing parent servers to not have a match

Posted by Josh Gitlin <jg...@pinnacle21.com>.

Thanks Leif! I am on 8.0.7, so maybe I should switch to 9

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

On May 7, 2020, at 3:52 PM, Leif Hedstrom <zw...@apache.org>> wrote:

On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com>> wrote:

In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.

— Leif

https://github.com/apache/trafficserver/pull/5544

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com<http://www.pinnacle21.com/>

On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com>> wrote:

Hello,

Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.

Contents of remap.config now:

map http://www.proxy.example.com<http://www.proxy.example.com/> http://www.example.com/
map https://www.proxy.example.com<https://www.proxy.example.com/> https://www.example.com/

Proposed fix to my config:

map http://www.proxy.example.com<http://www.proxy.example.com/> http://www.example.com/
map https://www.proxy.example.com<https://www.proxy.example.com/> https://www.example.com/
map http://www.example.com<http://www.example.com/> http://www.example.com/
map https://www.example.com<https://www.example.com/> https://www.example.com/

Is this the "right" way to fix this issue? The duplication feels like there must be a better way...

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com<http://www.pinnacle21.com/>

Re: Remap causing parent servers to not have a match

Posted by Leif Hedstrom <zw...@apache.org>.


> On May 7, 2020, at 1:05 PM, Josh Gitlin <jg...@pinnacle21.com> wrote:
> 
> The more I dig into this, the more I realize I have gone horribly wrong somewhere, as I seem to have just created an infinite parent proxy loop. So I may need to RTFM again to fix this broken design! :) 


In ATS 9.x, there is a “self” detection mechanism for this exact purpose (“cache cluster”). It prevents a box to parent a request that is hashing to itself.

— Leif

https://github.com/apache/trafficserver/pull/5544 <https://github.com/apache/trafficserver/pull/5544>

> 
> Josh Gitlin
> Principal DevOps Engineer
> jgitlin@pinnacle21.com <ma...@pinnacle21.com>
> 
> PINNACLE 21
> www.pinnacle21.com
> 
>> On May 7, 2020, at 1:51 PM, Josh Gitlin <jgitlin@pinnacle21.com <ma...@pinnacle21.com>> wrote:
>> 
>> Hello,
>> 
>> Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.
>> 
>> I am having an issue with remapping and parent caching. I have two Apache Traffic Server instances for HA, and each one has the other configured as its parent cache. The goal being a shared cache, because the two instances are behind a load balancer with leastconn distribution.
>> 
>> I am seeing an issue where cache misses on server B get forwarded to server A with the remapped URL and server A refuses to serve because it does not recognize the URL in it's remap config. (Error "ERR_INVALID_URL") I know I can resolve this by simply adding the original URL to the remap config, but that felt like the wrong fix.
>> 
>> Contents of remap.config now:
>> 
>> map http://www.proxy.example.com <http://www.proxy.example.com/> http://www.example.com/ <http://www.example.com/>
>> map https://www.proxy.example.com <https://www.proxy.example.com/> https://www.example.com/ <https://www.example.com/>
>> 
>> 
>> Proposed fix to my config:
>> 
>> map http://www.proxy.example.com <http://www.proxy.example.com/> http://www.example.com/ <http://www.example.com/>
>> map https://www.proxy.example.com <https://www.proxy.example.com/> https://www.example.com/ <https://www.example.com/>
>> map http://www.example.com <http://www.example.com/> http://www.example.com/ <http://www.example.com/>
>> map https://www.example.com <https://www.example.com/> https://www.example.com/ <https://www.example.com/>
>> 
>> Is this the "right" way to fix this issue? The duplication feels like there must be a better way...
>> 
>> Josh Gitlin
>> Principal DevOps Engineer
>> jgitlin@pinnacle21.com <ma...@pinnacle21.com>
>> 
>> PINNACLE 21
>> www.pinnacle21.com <http://www.pinnacle21.com/>
>

Re: Remap causing parent servers to not have a match

Posted by Josh Gitlin <jg...@pinnacle21.com>.

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com

On May 7, 2020, at 1:51 PM, Josh Gitlin <jg...@pinnacle21.com>> wrote:

Hello,

Apologies if this was covered in the docs or a previous message; I couldn't find an answer in my search.

Contents of remap.config now:

map http://www.proxy.example.com<http://www.proxy.example.com/> http://www.example.com/
map https://www.proxy.example.com<https://www.proxy.example.com/> https://www.example.com/

Proposed fix to my config:

Is this the "right" way to fix this issue? The duplication feels like there must be a better way...

Josh Gitlin
Principal DevOps Engineer
jgitlin@pinnacle21.com<ma...@pinnacle21.com>

PINNACLE 21
www.pinnacle21.com<http://www.pinnacle21.com>