You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by "Gray, Jonathan" <Jo...@comcast.com> on 2018/09/05 00:16:07 UTC
Re: [EXTERNAL] Re: Pattern-Based Consistent Hashing in Traffic Router
I think that's correct. While using the ATS cachekey plugin allows you to collapse multiple requests to a single spindle block range, it may only do so for requests it was given by TR. This would allow a similar logic to exist in TR to collapse multiple requests to a single cache where normally they would be consistent hashed to multiple ATS instances. It would be nice if this functionality could mirror the capabilities of the cachekey plugin so we could consistently someday configure them together. At the moment the proposal is more like the deprecated ATS cache_url plugin.
Jonathan Gray
On 9/4/18, 5:14 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
Hey Jesse,
Just to clarify the use case for this, is the idea that the origin
serves the exact same content under variable paths that follow some
pattern, e.g. /foo/123/bar/index.m3u8 and /foo/321/bar/index.m3u8? And
pattern-based consistent hashing is meant to be used in concert with
the Cache Key Manipulation Plugin for ATS so that only a single copy
of the content is stored per cache?
Thanks,
Rawlin
On Tue, Sep 4, 2018 at 3:13 PM Rivas, Jesse <Je...@comcast.com> wrote:
>
> Hi Traffic Controllers,
>
>
>
> I am working on a feature that will allow for pattern-based consistent hashing in Traffic Router in order to ignore variable parts of a request path, such as a zip code, during cache selection to increase the likelihood of hashing similar content requests to the same caches.
>
>
>
> For the implementation, I would like to propose adding a delivery service regex field that would go into the CRConfig, and then be leveraged in Traffic Router to modify the request path before passing the value to the ConsistentHasher to select a cache. More specifically, I would like to use regex grouping in order to extract the desirable parts of a request path and construct the path to be used for cache selection. Here’s what this would look like:
>
>
>
> For the request paths:
>
> /foo/123/bar/index.m3u8
>
> /foo/321/bar/index.m3u8
>
>
>
> Provided the regex:
>
> (/.*?)/\d*?(/.*?)(/*.m3u8)
>
>
>
> Resulting path to use for cache selection:
>
> /foo/bar/index.m3u8
>
>
>
> Due to the potentially risky nature of modifying cache selection, I plan to include a front-end testing tool for verifying a provided regex against a provided request path, so that the user knows that the resulting path is what they expect. If the regex does not match against the request path, the original request path will be used for cache selection.
>
>
>
> Here is a summary of the component changes for my proposed implementation for the pattern based consistent hashing feature:
>
> -new optional field in the delivery_service table, delivery service form, crconfig
>
> -TR parsing and leveraging regex field from crconfig; unit tests; API endpoint for testing tool
>
> -TP testing tool that will take a regex and request path, return the resulting path to be used for cache selection
>
>
>
> Please respond with any questions, comments, or concerns.
>
>
>
> Thanks,
>
> Jesse Rivas
Re: [EXTERNAL] Re: Pattern-Based Consistent Hashing in Traffic Router
Posted by "Rivas, Jesse" <Je...@comcast.com>.
Rawlin,
The use case that you described is correct, although the current proposal is a standalone feature from the cachekey plugin; the idea behind the feature is the ability to tweak TR cache selection in those use cases in order to increase the likelihood of cache hits, as Jonathan described. I'll examine the cachekey plugin for similarities and see how this feature and the plugin could work together.
Jesse
On 9/4/18, 6:22 PM, "Gray, Jonathan" <Jo...@comcast.com> wrote:
I think that's correct. While using the ATS cachekey plugin allows you to collapse multiple requests to a single spindle block range, it may only do so for requests it was given by TR. This would allow a similar logic to exist in TR to collapse multiple requests to a single cache where normally they would be consistent hashed to multiple ATS instances. It would be nice if this functionality could mirror the capabilities of the cachekey plugin so we could consistently someday configure them together. At the moment the proposal is more like the deprecated ATS cache_url plugin.
Jonathan Gray
On 9/4/18, 5:14 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
Hey Jesse,
Just to clarify the use case for this, is the idea that the origin
serves the exact same content under variable paths that follow some
pattern, e.g. /foo/123/bar/index.m3u8 and /foo/321/bar/index.m3u8? And
pattern-based consistent hashing is meant to be used in concert with
the Cache Key Manipulation Plugin for ATS so that only a single copy
of the content is stored per cache?
Thanks,
Rawlin
On Tue, Sep 4, 2018 at 3:13 PM Rivas, Jesse <Je...@comcast.com> wrote:
>
> Hi Traffic Controllers,
>
>
>
> I am working on a feature that will allow for pattern-based consistent hashing in Traffic Router in order to ignore variable parts of a request path, such as a zip code, during cache selection to increase the likelihood of hashing similar content requests to the same caches.
>
>
>
> For the implementation, I would like to propose adding a delivery service regex field that would go into the CRConfig, and then be leveraged in Traffic Router to modify the request path before passing the value to the ConsistentHasher to select a cache. More specifically, I would like to use regex grouping in order to extract the desirable parts of a request path and construct the path to be used for cache selection. Here’s what this would look like:
>
>
>
> For the request paths:
>
> /foo/123/bar/index.m3u8
>
> /foo/321/bar/index.m3u8
>
>
>
> Provided the regex:
>
> (/.*?)/\d*?(/.*?)(/*.m3u8)
>
>
>
> Resulting path to use for cache selection:
>
> /foo/bar/index.m3u8
>
>
>
> Due to the potentially risky nature of modifying cache selection, I plan to include a front-end testing tool for verifying a provided regex against a provided request path, so that the user knows that the resulting path is what they expect. If the regex does not match against the request path, the original request path will be used for cache selection.
>
>
>
> Here is a summary of the component changes for my proposed implementation for the pattern based consistent hashing feature:
>
> -new optional field in the delivery_service table, delivery service form, crconfig
>
> -TR parsing and leveraging regex field from crconfig; unit tests; API endpoint for testing tool
>
> -TP testing tool that will take a regex and request path, return the resulting path to be used for cache selection
>
>
>
> Please respond with any questions, comments, or concerns.
>
>
>
> Thanks,
>
> Jesse Rivas