You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@trafficserver.apache.org by GitBox <gi...@apache.org> on 2020/07/30 20:53:03 UTC

[GitHub] [trafficserver] SolidWallOfCode opened a new pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

SolidWallOfCode opened a new pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069


   to just before connection upstream.
   
   See #7052.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667459339


   Parent selection has build in loop prevention in that if you list the cache itself as a parent to itself in the parent list, parent selection will automatically mark it down for SELF DETECT and will never choose itself unless you disable self detection with ignore_self_detect.  I know that apple reported that PR #5767 broke parent selection and it was backed out. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] bryancall edited a comment on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

bryancall edited a comment on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-670229610


   @jp557198 Sounds like we might need to have another change to make parent proxy work with self.  
   
   What does your remap rule look like where you were getting proxy cycles?  If the from and the to are the same we should need this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford edited a comment on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford edited a comment on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667329221

@SolidWallOfCode we have a use case where we want a peering edge cache group. Say we have an edge cache group of cache1, cache2, and cache3. Upstream, there is a mid-tier cache group in front of the origins, mid-tier1, and mid-tier2. We want to send a client to an edge in the edge cache group where the parent's are an edge's peers and itself so as to cache a large video across the edge cache group and reduce lookups to the mid-tier using two remaps and two parent.config entries.

remap.config on cache1:
map http://cache1.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache2:
map http://cache2.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache3:
map http://cache3.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

parent.config on cache1, cache2, cache3
dest_domain=cdn-origin.foo.com parent="mid-tier1:80|1.0;mid-tier2:80|1.0" round_robin=consistent_hash go_direct=false
dest_domain=cdn-back-remap.foo.com parent="cache1:80|1.0;cache2:80|1.0;cache3:80|1.0" round_robin=consistent_hash go_direct=false ignore_self_detect=true

With the above parent.configs, each edge, cache1, cache2, and cache3 will mark themselves down for SELF DETECTION but we say ignore the SELF DETECTION because ignore_self_detect for the parent.config entry is true.

A request comes into cache1 on the first remap and is remapped to cdn-back-remap.foo.com. it misses and hashes to one of the parents in the local edge cache group, possibly to itself. when the request is forwarded to the back-remap, it gets remapped to the origin and if it misses, goes upstream to the mid-tiers and possibly the origin and gets cached locally on the edge. In the end, a large asset is spread across the 3 edge caches drastically reducing the necessity to go to the higher tier or origin and makes better use of the cache in the edge group. With the two remaps it does not cause any looping issues as long as you're careful. In 8.1 a host will get marked down by parent selection if it detects itself in the parent list so, that's the reasoning for the ignore_self_detect flag. Normally the self detection in parent selection prevents looping.

This scenario also required setting proxy.config.http.insert_request_via_str INT 0 to prevent loop detection in HttpTransact.

With these config settings, we're able to safely build a peering edge cache group to better utilize the edge's cache as a whole in ATS 8.1. Now with that said, this is experimental and we have nothing in production yet that utilizes it. We were taking advantage of the fact that we could loop safely one time in order to do this but with this PR, it will no longer be possible.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] bryancall commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

bryancall commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-670229610


   @jp557198 Sounds, like we might need to have another change to make parent proxy work with self.  
   
   What does your remap rule look like where you were getting proxy cycles?  If the from and the to are the same we should need this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford edited a comment on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford edited a comment on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667329221

@SolidWallOfCode we have a use case where we want a peering edge cache group. Say we have an edge cache group of cache1, cache2, and cache3. Upstream, there is a mid-tier cache group in from of the origins, mid-tier1, and mid-tier2. We want to send a client to an edge in the edge cache group where the parent's are an edge's peers and itself so as to cache a large video across the edge cache group and reduce lookups to the mid-tier using two remaps and two parent.config entries.

remap.config on cache1:
map http://cache1.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache2:
map http://cache2.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache3:
map http://cache3.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

This scenario also required setting proxy.config.http.insert_request_via_str INT 0 to prevent loop detection in HttpTransact.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] bryancall merged pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

bryancall merged pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford edited a comment on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford edited a comment on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667329221

@SolidWallOfCode we have a use case where we want a peering edge cache group. Say we have an edge cache group of cache1, cache2, and cache3. Upstream, there is a mid-tier cache group in from of the origins, mid-tier1, and mid-tier2. We want to send a client to an edge in the edge cache group where the parent's are an edge's peers and itself so as to cache a large video across the edge cache group and reduce lookups to the mid-tier using two remaps and two parent.config entries.

remap.config on cache1:
map http://cache1.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache2:
map http://cache2.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache3:
map http://cache3.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

A request comes into cache1 on the first remap and is remapped to cdi-back-remap.foo.com. it misses and hashes to one of the parents in the local edge cache group, possibly to itself. when the request is forwarded to the back-remap, it gets remapped to the origin and if it misses, goes upstream to the mid-tiers and possibly the origin and gets cached locally on the edge. In the end, a large asset is spread across the 3 edge caches drastically reducing the necessity to go to the higher tier or origin and makes better use of the cache in the edge group. With the two remaps it does not cause any looping issues as long as you're careful. In 8.1 a host will get marked down by parent selection if it detects itself in the parent list so, that's the reasoning for the ignore_self_detect flag. Normally the self detection in parent selection prevents looping.

This scenario also required setting proxy.config.http.insert_request_via_str INT 0 to prevent loop detection in HttpTransact.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667329221

@SolidWallOfCode we have a use case where we want a peering edge cache group. Say we have an edge cache group of cache1, cache2, and cache3. Upstream, there is a mid-tier cache group in from of the origins, mid-tier1, and mid-tier2. We want to send a client to an edge in the edge cache group where the parent's are an edge's peers and itself so as cache the video across the edge cache group and reduce lookups to the mid-tier using two remaps and two parent.config entries.

remap.config on cache1:
map http://cache1.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache2:
map http://cache2.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

remap.config on cache3:
map http://cache3.foo.com http://cdn-back-remap.foo.com
map http://cdn-back-remap.foo.com http://cdn-origin.foo.com

This scenario also required setting proxy.config.http.insert_request_via_str INT 0 to prevent loop detection in HttpTransact.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] bryancall commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

bryancall commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-670230189


   Cherry picked to 9.0.x


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jp557198 commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jp557198 commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-668258578


   ATS 7.1.4
   
   As a fyi..  we ran into this problem during our edge peering lab testing. This was my observation:
   
   "Proxy cycle is occurring because the proxy is making a request to itself.. ATS code compares the server and client via string values.
   See the below traffic.out and corresponding code.
   Maybe we can try playing with the via string(proxy_request_via_string)"
   
   To 'fix' this issue, we simply set proxy_request_via_string to 0.  Once set, a child was able to peer to itself.
   
   Side note: Edge/Child peering is currently soaking in prod and has worked to significantly increase HIT rate, as well as lower storage requirements(content/library sharding)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] SolidWallOfCode commented on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

SolidWallOfCode commented on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667443501


   Hrrrrm. I was specifically requested to detect and prevent direct looping on parent selection. A complaint about the original logic was it failed to detect exactly this situation. So it's breaking you by design (which in a sense is good, because that proves it solved one of the two problems the PR was addressing). 
   
   Since your stuff is currently experimental, and this fixes current production problems, I think we should merge it and then look for a fix for you. You'll have to hash it out with @vmamidi and his crew. I will ponder some approaches. Off the cuff,
   
   *  plugin API to disable the self loop check.
   *  configuration variable (which, frankly, will end up being the same as the previous).
   *  something in the HostDB/DNS record that says "OK to loop".
   *  something set by parent selection in this case to disable the check.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [trafficserver] jrushford edited a comment on pull request #7069: Move the direct self loop check later to HttpSM::do_http_server_open

Posted by GitBox <gi...@apache.org>.

jrushford edited a comment on pull request #7069:
URL: https://github.com/apache/trafficserver/pull/7069#issuecomment-667459339


   Since ATS 8, Parent selection has build in loop prevention in that if you list the cache itself as a parent to itself in the parent list, parent selection will automatically mark it down for SELF DETECT and will never choose itself unless you disable self detection with ignore_self_detect.  I know that apple reported that PR #5767 broke parent selection and it was backed out. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org